LECTURES on COMPUTATIONAL NUMERICAL ANALYSIS of PARTIAL DIFFERENTIAL EQUATIONS

Transcription

1 LECTURES on COMPUTATIONAL NUMERICAL ANALYSIS of PARTIAL DIFFERENTIAL EQUATIONS J. M. McDonough Departments of Mechancal Engneerng and Mathematcs Unversty of Kentucky c 1985, 00, 008

2 Contents 1 Introducton Basc Mathematcal Defntons Norms and related deas Convergence of sequences Consstency, stablty and convergence Classfcatons of Partal Dfferental Equatons Equaton type Form of nonlnearty Well Posedness of PDE Problems Dscretzaton and Grddng of PDE Problems Dscretzaton technques Grddng methods Summary Numercal Soluton of Ellptc Equatons 17.1 Background Iteratve soluton of lnear systems an overvew Basc theory of lnear teratve methods Successve Overrelaxaton Jacob teraton SOR theory Some modfcatons to basc SOR Alternatng Drecton Implct ADI) Procedures ADI wth a sngle teraton parameter ADI: the commutatve case ADI: the noncommutatve case Incomplete LU Decomposton ILU) Basc deas of ILU decomposton The strongly mplct procedure SIP) Precondtonng Conjugate-Gradent Acceleraton The method of steepest descent Dervaton of the conjugate-gradent method Relatonshp of CG to other methods Introducton to Multgrd Procedures Some basc deas The h-h two-grd algorthm l-grd multgrd methods The full multgrd method Some concludng remarks

3 CONTENTS.8 Doman-Decomposton Methods The alternatng Schwarz procedure The Schur complement Multplcatve and addtve Schwarz methods Multlevel doman-decomposton methods Summary Tme-Splttng Methods for Evoluton Equatons Alternatng Drecton Implct Methods Peaceman Rachford ADI Douglas Rachford ADI Implementaton of ADI schemes Locally One-Dmensonal Methods General Douglas Gunn Procedures D G methods for two-level dfference equatons D G methods for mult-level dfference equatons Summary Varous Mscellaneous Topcs Nonlnear PDEs The general nonlnear problem to be consdered Explct ntegraton of nonlnear terms Pcard teraton The Newton Kantorovch Procedure Systems of PDEs Example problem a generalzed transport equaton Quaslnearzaton of systems of PDEs Numercal Soluton of Block-Banded Algebrac Systems Block-banded LU decomposton how t s appled Block-banded LU decomposton detals Arthmetc operaton counts Cell-Re and Alasng Treatments The cell-re problem ts defnton and treatment Treatment of effects of alasng More Advanced Spatal Dscretzatons Basc approxmatons for mxed dervatves Mxed dervatves wth varable coeffcents Dscretzaton of self-adjont form second unmxed) dervatves Treatment of Advanced Boundary Condtons General lnear BCs Outflow BCs Summary Numercal Soluton of Partal Dfferental Equatons on Irregular Domans Grd Generaton Overvew of Grd Structures Unstructured grds Structured grds References 159

4 CONTENTS

5 Lst of Fgures 1.1 Schematc of a contracton mappng, [a, b] [fa), fb)] Methods for spatal dscretzaton of partal dfferental equatons; a) fnte dfference, b) fnte element and c) spectral N x N y pont grd and mesh star for dscretzatons of Eq..1) Sparse, banded matrces arsng from fnte-dfference dscretzatons of ellptc operators: a) 5-pont dscrete Laplacan; b) 9-pont general dscrete ellptc operator Qualtatve comparson of requred arthmetc for varous teratve methods for -D ellptc problems Qualtatve representaton of error reducton durng lnear fxed-pont teratons Dscretzaton of the Laplace/Posson equaton on a rectangular grd of N x N y ponts Band structure of Jacob teraton matrx for Laplace/Posson equaton Geometrc test of consstent orderng. a) consstent orderng, b) nonconsstent orderng Spectral radus of SOR teraton matrx vs. ω Red-black orderng for dscrete Laplacan Comparson of computatons for pont and lne SOR showng grd stencls and red-black ordered lnes Matrces arsng from decomposton of A: a) H matrx, b) V matrx, c) S matrx a) 7-band fnte-dfference matrx; b) correspondng mesh star Fnte-dfference grd for demonstratng structure of SIP matrces Level set contours and steepest-descent trajectory of -D quadratc form Level set contours, steepest-descent trajectory and conjugate gradent trajectory of -D quadratc form Comparson of h and h grds for multgrd mplementatons Multgrd V-cycles; a) l =, and b) l = Multgrd V-cycles wth l = 3 and dfferent values of γ; a) γ = 1, b) γ = and c) γ = Four-Level, V-cycle full multgrd schematc L-shaped grd depctng basc doman-decomposton approach Keyhole-shaped doman Ω 1 Ω consdered by Schwarz [41] Smple two-subdoman problem to demonstrate Schur complement Four-subdoman problem demonstratng types of ponts requred for Schur complement constructon Doman decomposton wth two overlappng subdomans; a) doman geometry, b) matrx structure Schematc depctng two-level doman decomposton and approxmate Schur complement Implementaton of lne-by-lne solves for tme splttng of tme-dependent problems Numercal Drchlet Neumann problem; ponts on dashed lnes are mage ponts needed for mplementaton of centered dscretzatons

6 v LIST OF FIGURES 4.1 Grd pont to grd pont oscllatons caused by cell-re problem Dependence of z + on cell Re Under-sampled oscllatory functon demonstratng effects of alasng Schematc of a dscrete mollfer Wavenumber response of Shuman flter for varous values of flter parameter Fnte-dfference grd wth unform grd spacngs h x h y n neghborhood of grd pont, j) Geometry of a) structured and b) unstructured grds Smple 1-D geometry for nonunform-grd constructon of second dervatve approxmatons. 157

7 Chapter 1 Introducton The purpose of these lectures s to present a set of straghtforward numercal methods wth applcablty to essentally any problem assocated wth a partal dfferental equaton PDE) or system of PDEs ndependent of type, spatal dmenson or form of nonlnearty. In ths frst chapter we provde a revew of elementary deas from pure mathematcs that usually are covered n a frst course n numercal analyss, but whch are often not ntroduced n engneerng computaton courses. Because many of these deas are essental to understandng correct numercal treatments of PDEs, we nclude them here. We note that these can all be found n varous sources, ncludng the elementary numercal analyss lecture notes of McDonough [1]. In Chap. we provde a qute thorough and reasonably up-to-date numercal treatment of ellptc partal dfferental equatons. Ths wll nclude detaled analyses of classcal methods such as successve overrelaxaton SOR) as well as varous modern technques, especally multgrd and doman decomposton methods. Chapter 3 presents a detaled analyss of numercal methods for tme-dependent evoluton) equatons and emphaszes the very effcent so-called tme-splttng methods. These can, n general, be equally-well appled to both parabolc and hyperbolc PDE problems, and for the most part these wll not be specfcally dstngushed. On the other hand, we wll note, va examples, some features of these two types of PDEs that make detals of ther treatment somewhat dfferent, more wth respect to the dscretzatons employed than wth specfc soluton technques. Chapter 4 s devoted to presentng a number of mscellaneous topcs and methods, many of whch could be applcable to any of the types of problems studed n earler chapters. These wll nclude smoothng technques, methods for treatng nonlnear equatons, approaches to be employed for systems of equatons, ncludng modfcatons to the requred numercal lnear algebra, some specal dscretzatons for specfc types of dfferental operators, and boundary condton mplementaton beyond that usually gven n elementary courses. Fnally, Chap. 5 wll provde an ntroducton to methods used for PDE problems posed on arbtrary spatal domans. There are many types of such problems and, correspondngly, many ways n whch to deal wth them. Heren, we wll begn wth a revew of advantages and dsadvantages of varous of the approaches used to treat such problems. Then we wll focus attenton on two partcular methods one very old and dependable, and the other farly new and stll the topc of much nvestgaton: namely, use of generalzed coordnates and so-called mmersed boundary methods, respectvely. At the concluson of these lectures t s hoped that readers wll have acqured a basc background permttng them to solve very general PDE problems, vz., systems of nonlnear partal dfferental equatons whose solutons may not be very regular) posed on essentally arbtrary geometrc domans ncludng those that are movng and/or deformng). Examples of such systems of equatons and ther assocated problems arse n many dfferent engneerng, physcal scences, and bologcal scences dscplnes and n varous combnatons of these. Our goal n these lectures s to prepare the reader to approach essentally any of these n straghtforward and computatonally-effcent ways. We remark that all analyses wll be presented n the context of fnte-dfference/fnte-volume dscretza- 1

8 CHAPTER 1. INTRODUCTION ton of the PDEs) nvolved on structured grds. We wll provde more detaled justfcaton for ths choce at varous junctures throughout the lectures, but we here note that ths combnaton represents the smplest and most effcent approach to the PDE problem, n general. Nevertheless, many n fact, most) of the algorthms we wll descrbe can easly be employed n the context of fnte-element methods, as well, and wth modfcaton possbly sgnfcant n some cases on unstructured grds. The remander of ths ntroductory chapter ncludes sectons n whch each of the followng topcs s dscussed: ) some basc mathematcal defntons, ) classfcatons of partal dfferental equatons, ) the noton of well posedness, and v) an overvew of methods for dscretzng PDEs and grddng ther domans. 1.1 Basc Mathematcal Defntons In ths secton we wll ntroduce some basc defntons along wth standard notaton correspondng to them. We wll not provde the detal here that s already avalable n [1] and elsewhere, but nstead manly supply a remnder of some of the mathematcal notons that are crucal n the study of numercal methods for PDEs. These wll nclude such mathematcal constructs as norm, the Cauchy Schwarz nequalty, convergence, a contracton mappng prncple, consstency, stablty, and the Lax equvalence theorem Norms and related deas One of the most fundamental propertes of any object, be t mathematcal or physcal, s ts sze. Of course, n numercal analyss we are always concerned wth the sze of the error of any partcular numercal approxmaton, or computatonal procedure. There s a general mathematcal object, called the norm, by whch we can assgn a number correspondng to the sze of varous mathematcal enttes. Defnton 1.1 Let S be a fnte- or nfnte-dmensonal) vector space, and let denote the mappng S R + {0} wth the followng propertes: ) v 0, v S wth v = 0 ff v 0, ) av = a v, v S, a R, ) v + w v + w v, w S. Then s called a norm for S. Note that we can take S to be a space of vectors, functons or even operators, and the above propertes apply. It s mportant to observe that for a gven space S there are, n general, many dfferent mappngs havng the propertes requred by the above defnton. We wll gve a few specfc examples. If S s a fnte-dmensonal space of vectors wth elements v = v 1, v,..., v N ) T then a famlar measure of the sze of v s ts Eucldean length, v = N v =1 ) ) The proof that, often called the Eucldean norm, or smply the -norm, satsfes the three condtons of the defnton s straghtforward, and s left to the reader. We note here that t s common n numercal

9 1.1. BASIC MATHEMATICAL DEFINITIONS 3 analyss to employ the subscrpt E to denote ths norm and use the subscrpt for the spectral norm of matrces. But we have chosen to defer to notaton more consstent wth pure mathematcs.) Another useful norm often encountered n practce s the max norm or nfnty norm defned as v = max 1 N v. 1.) If S s an nfnte-dmensonal space, then we can defne a partcular famly of norms as follows: Defnton 1. Let S be the space L p, 1 p <, on a doman Ω, and let f L p Ω). Then the L p norm of f on Ω s 1/p f L p f dx) p, 1.3) Ω where Ω s an arbtrary usually) spatal doman, possbly possessng nfnte lmts, and x denotes a correspondng coordnate vector wth the approprate dmenson. We remark that, n fact, the dfferental assocated wth ths coordnate vector s actually a measure a pont we wll not emphasze n these lectures), and the ntegral appearng n the defnton s that of Lebesgue hence, the L notaton for the assocated spaces rather than the famlar Remann ntegral from Freshman Calculus. These ponts wll occasonally arse n the sequel. Also, we note that by defnton f L p Ω) ff f L p < ;.e., the ntegral n Eq. 1.3) exsts. When p = we obtan a very specal case of the L p spaces known as a Hlbert space, and defned as follows: Defnton 1.3 A Hlbert space s a complete, normed, lnear space n whch the norm s nduced by an nner product. We note that completeness n ths context means that any convergent sequence n the space converges to an element of the space.e., to an element wthn the space). A lnear space s smply one n whch fnte lnear combnatons of elements of the space are also elements of the space. Fnally, the noton that the norm s nduced by an nner product can be understood n the followng way. Frst, we must defne nner product. Defnton 1.4 Let f and g be n L Ω). Then the nner product of f and g s defned as f, g fg dx, 1.4) where, agan Ω s a prescrbed doman. Ω Now observe that f f = g we have f, f f dx = f, 1.5) L Ω f f takes on values n R. The case when f has complex values s only slghtly dfferent and wll not be needed heren.) Hence, the nner product of f wth tself s just the square of the L norm, and we say ths norm s nduced by the nner product. Because of ths, the space L s often termed the canoncal Hlbert space. We menton that there are many other usually more complcated Hlbert spaces, but all, by defnton, have a norm nduced by an often more complcated) nner product. All such spaces are typcally referred to as nner-product spaces. We can now ntroduce a very mportant property that relates the nner product and the norm, the Cauchy Schwarz nequalty.

10 4 CHAPTER 1. INTRODUCTION Theorem 1.1 Cauchy Schwarz) Let f and g be n L Ω), Ω R d, d typcally 1, or 3. Then f, g f L g L. 1.6) Moreover, the L norms can generally be replaced by any norm nduced by an nner product, and the nequalty 1.6) wll hold. Many of the Sobolev spaces encountered n fnte-element analyss, and generally n studes of the Naver Stokes equatons, are of ths type. It s clear from 1.6) that f f and g are n L hence, ther norms are bounded), then ths must also be true of ther nner product. Ths smple observaton s extremely mportant for the theory of Fourer seres because t mples exstence of Fourer coeffcents for all functons n L and other Hlbert spaces). We next need to consder some correspondng deas regardng norms of operators. The general defnton of an operator norm s as follows. Defnton 1.5 Let A be an operator whose doman s D. Then the norm of A s defned as It s easy to see that ths s equvalent to A max Ax. 1.7) x =1 x DA) Ax A = max x 0 x, x DA) from whch follows an nequalty smlar to the Cauchy Schwarz nequalty for vectors or functons), Ax A x. 1.8) We should remark here that 1.8) actually holds only n the fnte-dmensonal case when the matrx and vector norms appearng n the expresson are compatble, and ths relatonshp s often used as the defnton of compatblty. We wll seldom need to employ ths concept n the present lectures, and the reader s referred to, e.g., Isaacson and Keller [] Chap. 1) for addtonal nformaton regardng ths noton. We observe that nether 1.7) nor the expresson followng t s sutable for practcal calculatons; we now present three norms that are readly computed, at least for M N matrces, and are wdely used n numercal lnear algebra. The frst of these s the -norm, gven n the matrx case by M,N A =,j=1 a j 1, 1.9) whch s completely analogous to the vector -norm of Eq. 1.1). Moreover, the same general constructon s used for norms of matrces of hgher dmenson. Two other norms are also frequently employed. These are the 1-norm and the nfnty norm A 1 A = max 1 j N = max 1 M M a j, 1.10) =1 N a j. 1.11) These too have generalzatons to hgher dmensons, but whch are no longer unque. We note that although the defnton of the operator norm gven above was not necessarly fntedmensonal, we have here gven only fnte-dmensonal practcal computatonal formulas. We wll see j=1

11 1.1. BASIC MATHEMATICAL DEFINITIONS 5 later that ths s not really a serous restrcton because problems nvolvng dfferental operators, one of the man nstances where norms of nfnte-dmensonal operators are needed, are essentally always solved va dscrete approxmatons leadng to fnte-dmensonal matrx representatons. There s a fnal, general comment that should be made regardng norms. It arses from the fact, mentoned earler, that n any gven vector space many dfferent norms mght be employed. A comparson of the formulas n Eqs. 1.1) and 1.), for example, wll show that the number one obtans to quantfy the sze of a mathematcal object, a vector n ths case, wll change accordng to whch formula s appled. Thus, a reasonable queston s, How do we decde whch norm to use? It turns out, for the fnte-dmensonal spaces we wll manly deal wth heren, that t really does not matter whch norm s used, provded only that the same one s used when makng comparsons between smlar mathematcal objects. Ths s the content of what s known as the norm equvalence theorem: all norms are equvalent on fnte-dmensonal spaces n the sense that f a sequence converges n one norm, t wll converge n any other norm see Ref. [], Chap. 1). Ths mples that n practcal numercal calculatons we should usually employ the norm that requres the least amount of floatng-pont arthmetc for ts evaluaton. But we note here that the stuaton s rather dfferent for nfnte-dmensonal spaces assocated wth, for example, analytcal studes of PDEs. In partcular, for problems nvolvng dfferental equatons, determnaton of the functon space n whch a soluton exsts and hence, the approprate norm) s a sgnfcant part of the overall problem. We wll at tmes need to deal wth ths later Convergence of sequences It wll be clear, as we proceed through these lectures, that convergence of sequences generated va teratve methods, and also by changng dscretzaton step szes, s a fundamental aspect of numercal soluton of PDEs. Hence, we present the followng elementary defntons and dscussons assocated wth ther applcaton. Defnton 1.6 Let {y m } m=1 be a sequence n RN. The sequence s sad to converge to the lmt y R N f ɛ > 0 M dependng on ɛ) m M, y y m < ɛ. We denote ths by lm y m = y. m We note here that the norm has not been specfed, and we recall the earler remark concernng equvalence of norms n fnte-dmensonal spaces. More nformaton on ths can be found, for example, n Apostol [3].) We also observe that when N = 1, we merely replace norm wth absolute value. It s farly obvous that the above defnton s not generally of practcal value, for f we knew the lmt y, whch s requred to check convergence, we probably would not need to generate the sequence n the frst place. To crcumvent such dffcultes mathematcans nvented the noton of a Cauchy sequence gven n the followng defnton. Defnton 1.7 Let {y m } m=1 RN, and suppose that ɛ > 0 M dependng on ɛ) m, n M, y m y n < ɛ. Then {y m } s a Cauchy sequence. By tself, ths defnton would not be of much mportance; but t s a farly easly proven fact from elementary analyss see, e.g., [3]) that every Cauchy sequence n a complete metrc space converges to an element of that space. It s also easy to show that R N s a complete metrc space N <. Thus, we need only demonstrate that successve terates, for example, form a Cauchy sequence, and we can then conclude that the sequence converges n the sense of the earler defnton. Although ths represents a consderable mprovement over tryng to use the basc defnton n convergence tests, t stll leaves much to be desred. In partcular, the defnton of a Cauchy sequence requres that y m y n < ɛ hold ɛ > 0, and m, n M, where M, tself, s not specfed, a pror. For computatonal purposes t s completely unreasonable to choose ɛ smaller than the absolute normalzed precson the machne ɛ) of the floatng-pont arthmetc employed. It s usually suffcent to use values of ɛ Oe/10), where e s the acceptable error for the computed results. The more dffcult part of the

12 6 CHAPTER 1. INTRODUCTION defnton to satsfy s m, n M. However, for well-behaved sequences, t s typcally suffcent to choose n = m + k where k s a specfed nteger between one and, say 100. Often k = 1 s used.) The great majorty of computer-mplemented teraton schemes test convergence n ths way; that s, the computed sequence {y m } s consdered to be converged when y m+1 y m < ɛ for some prescrbed and often completely fxed) ɛ. In many practcal calculatons ɛ 10 3 represents qute suffcent accuracy, but of course ths s problem dependent. Now that we have a means by whch the convergence of sequences can be tested, we wll study a systematc method for generatng these sequences n the context of solvng equatons. Ths method s based on a very powerful and basc noton from mathematcal analyss, the fxed pont of a functon, or mappng. Defnton 1.8 Let f: D D, D R N. Suppose x D, and x = fx). Then x s sad to be a fxed pont of f n D. We see from ths defnton that a fxed pont of a mappng s smply any pont that s mapped back to tself by the mappng. Now at frst glance ths mght not seem too useful some pont beng repeatedly mapped back to tself, over and over agan. But the expresson x = fx) can be rewrtten as x fx) = 0, 1.1) and n ths form we recognze that a fxed pont of f s a zero or root) of the functon gx) x fx). Hence, f we can fnd a way to compute fxed ponts, we automatcally obtan a method for solvng equatons. Indeed, ntuton suggests that we mght try to fnd a fxed pont of f va the followng teraton scheme: x 1 = fx 0 ) x = fx 1 ) x m = fx m 1 ) where x 0 s an ntal guess. Ths procedure generates the sequence {x m } of approxmatons to the fxed pont x, and we contnue ths untl x m+1 x m < ɛ. We remark that ths approach, often called successve approxmaton, wll be wdely used n subsequent chapters. The followng theorem utlzes ths basc dea of successve approxmaton to provde suffcent condtons for convergence of fxed-pont teratons n fnte-dmensonal spaces of dmenson N. Numerous smlar results are known for nfnte-dmensonal spaces as well, but we wll not consder these here. Theorem 1. Contracton Mappng Prncple) Let f be contnuous on a compact subset D R N wth f: D D, and suppose a postve constant L < 1 fy) fx) L y x x, y D. 1.13) Then a unque x D x = fx ), and the sequence {x m } m=0 generated by x m+1 = fx m ) converges to x from any and thus, every) ntal guess, x 0 D. The nequalty 1.13) s of suffcent mportance to mert specal attenton.

13 1.1. BASIC MATHEMATICAL DEFINITIONS 7 Defnton 1.9 The nequalty, fy) fx) L y x, x, y D, s called a Lpschtz condton, and L s the Lpschtz constant. Any functon f satsfyng such a condton s sad to be a Lpschtz functon. There are several thngs to note regardng the above theorem. The frst s that satsfacton of the Lpschtz condton wth L < 1 s suffcent, but not always necessary, for convergence of the correspondng teratons. In partcular, t mples the contractve propertes of the functon f, as llustrated n Fg. 1.1 for a smple scalar case. It s clear from ths fgure that fb) fa) < b a, so f we take D = [a, b] n the defnton, t follows that L < 1 must hold. In other words, the nterval [a, b] s contracted when t s mapped by the functon f. Second, for any set D, and mappng f wth 1.13) holdng throughout, x s f f b) ) f a a b x Fgure 1.1: Schematc of a contracton mappng, [a, b] [fa), fb)]. the unque fxed pont of f n D. Furthermore, the teratons wll converge to x usng any startng guess, whatever, so long as t s an element of the set D. In addton, the hypothess that f s contnuous n D s essental. It s easy to construct examples of teraton functons satsfyng all of the stated condtons except contnuty, and for whch the teratons fal to converge. On the other hand, the compactness requrement of the theorem statement s somewhat techncal, and t s needed manly for rgorous mathematcal analyses Consstency, stablty and convergence Here, we present some basc, farly smple, descrptons of these terms. By consstency we mean that the dfference approxmaton converges to the PDE as dscretzaton step szes approach 0, and by stablty we mean that the soluton to the dfference equaton does not ncrease wth tme at a faster rate than does the soluton to the dfferental equaton. Convergence mples that solutons to the dfference equaton approach those of the PDE, agan, as dscretzaton step szes are refned. If ths does not occur, the assocated numercal approxmatons are useless. Clearly, consstency and stablty are crucal propertes for a dfference scheme because of the followng theorem due to Lax see Rchtmyer and Morton [4]). Theorem 1.3 Lax Equvalence Theorem) Gven a well-posed lnear ntal-value problem, and a correspondng consstent dfference approxmaton, the resultng grd functons converge to the soluton of the

14 8 CHAPTER 1. INTRODUCTION dfferental equatons) as spatal and temporal step szes, respectvely, h, k 0 f and only f the dfference approxmaton s stable. It s mportant to understand the content of ths theorem. Frst, the noton of well posedness wll be treated n more detal later n ths chapter; here we can assocate t wth stuatons n whch the problem s guaranteed to have a well-defned soluton. Ths, of course, s not always the case.) One can deduce from the theorem the less than obvous fact that consstency of a dfference approxmaton s not a suffcent condton for guaranteeng that the grd functons produced by the scheme actually converge to the soluton of the orgnal dfferental equaton as dscretzaton step szes are refned. In partcular, both consstency and stablty are requred. As wll be evdent n what follows n later chapters, consstency of a dfference approxmaton s usually very straghtforward though sometmes rather tedous to prove, whle proof of stablty can often be qute dffcult. We observe here that although the Lax equvalence theorem s extremely mportant, t s also qute restrcted n scope. At the same tme, convergence of numercally-computed grd functons s one of the most fundamental propertes that must be demonstrated for any PDE approxmaton and, we remark that ths s too often gnored, especally by engneers and physcsts. Because of ts mportance, whch wll be further stressed n the sequel, we here present some detals of conductng such tests n a smple scalar envronment. The reader can fnd essentally the same materal n [1].) Clearly, f we know the soluton to the problem we are solvng ahead of tme we can always exactly determne the error of the numercal soluton. But, of course, f we already know the answer, we would not need a numercal soluton n the frst place, n general. An mportant excepton s the study of model problems when valdatng a new algorthm and/or computer code.) It turns out that a rather smple test for accuracy can and should always be performed on solutons represented by a grd functon. Namely, we employ a Cauchy convergence test on the grd functon as dscretzaton step szes are reduced. For grd functons we generally have avalable addtonal qualtatve nformaton, derved from the numercal method tself, about the theoretcal convergence rate of the grd functons generated by the method. In partcular, we almost always have the truncaton error expanson at our dsposal. For example, for any suffcently smooth u obtaned va a dscrete approxmaton usng a step sze h at a grd pont we would have and by changng the step sze to rh we have The domnant error n the frst case s and n the second case t s u h = ux ) + τ 1 h q 1 + = ux ) + O h q 1 ), u rh = ux ) + τ 1 r q 1 h q 1 +. e rh e h ux ) u h = τ 1h q 1, 1.14) = ux ) u rh = τ 1 r q 1 h q 1, 1.15) provded h s suffcently small to permt neglect of hgher-order terms n the expansons. theoretcal rato of the errors for two dfferent step szes s known to be smply Thus, the e rh e h = r q ) Hence, for a second-order method q 1 = ) a reducton n the step sze by a factor of two r = 1 ) leads to a reducton n error gven by ) r q 1 1 = = 1 4 ;

15 1.1. BASIC MATHEMATICAL DEFINITIONS 9.e., the error s reduced by a factor of four. In practcal problems we usually do not know the exact soluton, ux); hence we cannot calculate the true error. However, f we obtan three approxmatons to ux), say { u h } } { }, {u h/ and u h/4, we can make good estmates of τ 1, q 1 and ux ) at all ponts x for whch elements of all three grd functons are avalable. Ths merely nvolves solvng the followng system of three equatons for τ 1, q 1 and ux ): u h = ux ) + τ 1 h q 1, u h/ = ux ) + q 1 τ 1 h q 1, u h/4 = ux ) + 4 q 1 τ 1 h q 1. Now recall that u h, uh/, u h/4 and h are all known values. Thus, we can substract the second equaton from the frst, and the thrd from the second, to obtan and u h u h/ = 1 q 1 ) τ 1 h q 1, 1.17) u h/ u h/4 = q 1 1 q 1 ) τ 1 h q ) Then the rato of these s u h uh/ u h/ u h/4 = q 1, 1.19) whch s equvalent to the result 1.16) obtaned above now wth r = ) usng true error. Agan note that q 1 should be known, theoretcally; but n practce, due ether to algorthm/codng errors or smply to use of step szes that are too large, the theoretcal value of q 1 may not be attaned at all or possbly at any!) grd ponts x. Ths motvates us to solve Eq. 1.19) for the actual value of q 1 : [ ] u h log uh/ u h/ u h/4 q 1 =. 1.0) log Then from Eq. 1.17) we obtan τ 1 = uh uh/ 1 q ) h q ) Fnally, we can now produce an even more accurate estmate of the exact soluton equvalent to Rchardson extrapolaton) from any of the orgnal equatons; e.g., ux ) = u h τ 1 h q 1. 1.) In most practcal stuatons we are more nterested n smply determnng whether the grd functons converge and, f so, whether convergence s at the expected theoretcal rate. To do ths t s usually suffcent to replace ux ) n the orgnal expansons wth a value u computed on a grd much fner than any of the test grds, or a Rchardson extrapolated value obtaned from the test grds, say u. The latter s clearly more practcal, and for suffcently small h t leads to ẽ h = u uh = τ 1 h q 1, where ẽ h denotes an approxmaton to e h of Eq. 1.14) snce u only an approxmaton of the exact functon ux ). Smlarly, the result of Rchardson extrapolaton) s ẽ h/ = u u h/ = q 1 τ 1 h q 1,

16 10 CHAPTER 1. INTRODUCTION and the rato of these errors s ẽ h ẽ h/ = u uh u uh/ = q ) Yet another alternatve and n general, probably the best one when only grd functon convergence s the concern) s smply to use Eq. 1.19),.e., employ a Cauchy convergence test. As noted above we generally know the theoretcal value of q 1. Thus, the left sde obtaned from numercal computaton) can be compared wth the rght sde theoretcal). Even when q 1 s not known we can gan qualtatve nformaton from the left-hand sde alone. In partcular, t s clear that the rght-hand sde s always greater than unty. Hence, ths should be true of the left-hand sde. If the equalty n the approprate one of 1.19) or 1.3) s not at least approxmately satsfed, the frst thng to do s reduce h, and repeat the analyss. If ths does not lead to closer agreement between left- and rght-hand sdes n these formulas, t s farly certan that there are errors n the algorthm and/or ts mplementaton. We note that the above procedures can be carred out for arbtrary sequences of grd spacngs, and for mult-dmensonal grd functons. In both cases the requred formulas are more nvolved, but n fact generally occur n the PDE context. Fnally, we must recognze that e h or ẽ h ) s error at a sngle grd pont. In most practcal problems t s more approprate to employ an error norm computed wth the entre soluton vector. Then 1.16), for example, would be replaced wth for some norm, say, the vector -norm. e h = uh u h/ e h/ u h/ u h/4 = q 1, 1.4) 1. Classfcatons of Partal Dfferental Equatons There are many ways n whch PDEs can be classfed. Here, we consder what probably are the two most mportant: classfcaton by type for lnear equatons, and defnton of the forms of nonlnearty that arse for those whch are not lnear Equaton type The most general form of lnear second-order partal dfferental equatons, when restrcted to two ndependent varables and constant coeffcents, s au xx + bu xy + cu yy + du x + eu y + fu = gx, y), 1.5) where g s a known forcng functon; a, b, c,..., are gven constants, and subscrpts denote partal dfferentaton. In the homogeneous case,.e., g 0, ths form s remnscent of the general quadratc form from hgh school analytc geometry: ax + bxy + cy + dx + ey + f = ) Equaton 1.6) s sad to be an ellpse, a parabola or a hyperbola accordng as the dscrmnant b 4ac s less than, equal to, or greater than zero. Ths same classfcaton ellptc, parabolc, or hyperbolc s employed for the PDE 1.5), ndependent of the nature of gx, y). In fact, t s clear that the classfcaton of lnear PDEs depends only on the coeffcents of the hghest-order dervatves. Ths groupng of terms, au xx + bu xy + cu yy, s called the prncpal part of the dfferental operator n 1.5),.e., the collecton of hghest-order dervatve terms wth respect to each ndependent varable; and ths noton can be extended n a natural way to more complcated operators. Thus, the type of a lnear equaton s completely determned by ts prncpal part.

17 1.. CLASSIFICATIONS OF PARTIAL DIFFERENTIAL EQUATIONS 11 It should be mentoned that f the coeffcents of 1.5) are permtted to vary wth x and y, ts type may change from pont to pont wthn the soluton doman. Ths can pose sgnfcant dffcultes, both analytcal and numercal. We wll not specfcally deal wth these n the current lectures. We next note that correspondng to each of the three types of equatons there s a unque canoncal form to whch 1.5) can always be reduced. We shall not present the detals of the transformatons needed to acheve these reductons, as they can be found n many standard texts on elementary PDEs e.g., Berg and MacGregor [5]). On the other hand, t s mportant to be aware of the possblty of smplfyng 1.5), snce ths may also smplfy the numercal analyss requred to construct a soluton algorthm. Ellptc. It can be shown when b 4ac < 0, the ellptc case, that 1.5) collapses to the form u xx + u yy + Au = gx, y), 1.7) wth A = 0, ±1. When A = 0 we obtan Posson s equaton, or Laplace s equaton n the case g 0; otherwse, the result s usually termed the Helmholtz equaton. Parabolc. For the parabolc case, b 4ac = 0, we have u x u yy = gx, y), 1.8) whch s the heat equaton, or the dffuson equaton. We remark that b 4ac = 0 can also mply a degenerate form whch s only an ordnary dfferental equaton ODE). We wll not treat ths case n the present lectures. Hyperbolc. For the hyperbolc case, b 4ac > 0, Eq. 1.5) can always be transformed to u xx u yy + Bu = gx, y), 1.9) where B = 0 or 1. If B = 0, we have the wave equaton, and when B = 1 we obtan the lnear Klen Gordon equaton. Fnally, we note that determnaton of equaton type n dmensons greater than two requres a dfferent approach. The detals are rather techncal but bascally nvolve the fact that ellptc and hyperbolc operators have defntons that are ndependent of dmenson, and usual parabolc operators can then be dentfed as a combnaton of an ellptc spatal operator and a frst-order evoluton operator. 1.. Form of nonlnearty Nonlnear dfferental equatons can take on a number of dfferent forms. Here, we wll treat the ones that occur most often n practce. We begn by recallng the defnton of a lnear operator to provde a pont of reference. Defnton 1.10 Let S be a vector space defned on the real numbers R or the complex numbers C), and let L be an operator or transformaton) whose doman s S. Suppose for any u, v S and a, b R or C) we have Lau + bv) = alu + blv. 1.30) Then L s sad to be a lnear operator. The reader s encouraged to show that all of the dfferental operators of the prevous subsecton are lnear. The forms of nonlnearty we consder here are: semlnear, quaslnear and fully nonlnear. These forms are generally the same for both steady-state and evoluton equatons. Semlnear. Semlnear PDEs represent the smplest type of nonlnearty. They are lnear n all terms except those of zero th order; hence, they can be expressed as Lu + F u) = sx), where L s a lnear but possbly varable-coeffcent) operator, n general contanng both spatal dervatves and evoluton terms, and F s a nonlnear functon of u only;.e., there are no nonlneartes n terms contanng dervatves of

18 1 CHAPTER 1. INTRODUCTION u. The rght-hand sde term, s, s a prescrbed forcng functon. A typcal example of a semlnear PDE would be a reacton-dffuson equaton such as u t D u e C/u = 0, 1.31) wth denotng the Laplacan e.g., / x + / y n -D Cartesan coordnates); D s a constant dffuson coeffcent, and C s a constant reacton energy. Quaslnear. PDEs are called quaslnear when terms contanng ther lowest dervatves appear as a product of the dervatve and an undfferentated factor of the dependent varable. Note, of course, that f the equatons are only frst order, then ther lowest dervatves are also ther hghest ones.) Advectvedffusve equatons such as the Naver Stokes equatons are typcal examples: u t + uu x + vu y ν u = p x 1.3) for the x-momentum equaton. Observe that all terms are lnear except the advectve also called convectve) terms uu x + vu y. Clearly, the frst of these s nonlnear as the reader may check by usng the precedng defnton of lnearty t fals to satsfy ths defnton; n partcular, t s quaslnear. Moreover, we can wrte ths n so-called conservaton-law form as u ) x to obtan an alternate expresson for ths quaslnear form. But we emphasze that, n general, t s the frst form, shown n Eq. 1.3) that represents the defnton of quaslnearty. We also remark that the second advecton term s also formally quaslnear f v = vu) as would be the case for the ncompressble Naver Stokes equatons va the dvergence-free condton. But such terms are often classfed as blnear because u, tself, appears explctly n only one factor. Fully Nonlnear. The fully-nonlnear, or just nonlnear, case s characterzed by nonlneartes even n the hghest dervatves of the dfferental operator. An example mght be a heat equaton contanng a temperature-dependent thermal dffusvty, for example, n an advectve-dffusve equaton of the form T t + ut x + vt y DT )T ) = ) Another, often much more complcated, nstance of ths s use of eddy vscosty n Reynolds-averaged turbulence models of the Naver Stokes equatons. We comment that, n general, from a pure mathematcs perspectve, t s the fully-nonlnear PDEs whch are least understood, as mght be expected. 1.3 Well Posedness of PDE Problems Before proceedng to ntroduce numercal methods for solvng each of the three man classes of problems t s worthwhle to gve some consderaton to the queston of under what crcumstances these equatons do, or do not, have solutons. Ths s a part of the mathematcal concept of well posedness. There are many dfferent specfc defntons; the one gven here s commonly used n elementary settngs. Defnton 1.11 A problem consstng of a partal dfferental equaton and boundary and/or ntal condtons s sad to be well posed n the sense of Hadamard f t satsfes the followng condtons: ) a soluton exsts; ) the soluton s unque; ) the soluton depends contnuously on gven data. The well-posedness property s crucal n solvng problems by numercal methods because essentally all numercal algorthms embody the tact assumpton that problems to whch they apply are well posed. Consequently, a method s not lkely to work correctly on an ll-posed.e., a not well-posed) problem.

19 1.4. DISCRETIZATION AND GRIDDING OF PDE PROBLEMS 13 The result may be falure to obtan a soluton; but a more serous outcome may be generaton of numbers that have no assocaton wth realty n any sense. It behooves the user of numercal methods to suffcently understand the mathematcs of any consdered problem to be aware of the possble dffcultes and symptoms of these dffcultes assocated wth problems that are not well posed. We close ths bref dscusson of well posedness by descrbng a partcular problem that s not well posed. Ths s the so-called backward heat equaton problem. It arses n geophyscal studes n whch t s desred to predct the temperature dstrbuton wthn the Earth at some earler geologcal tme by ntegratng backward from the presumed-known) temperature dstrbuton of the present. To demonstrate the dffcultes that arse we consder a smple one-dmensonal ntal-value problem for the heat equaton: u t = κu xx, x, ), t [ T, 0), wth Formally, the exact soluton s ux, t) = ux, 0) = fx). 1 4πκt fξ)e x ξ) 4κt dξ, 1.34) the dervaton of whch see, e.g., Berg and MacGregor [5]) mposes no specfc restrctons on the sgn of t. But we mmedately see that f t < 0, ux, t), f t exsts at all, s magnary snce κ, the thermal dffusvty, s always greater than zero). In fact, unless f decays to zero faster than exponentally at ±, there s no soluton because the ntegral n 1.34) does not exst. It turns out that behavor of heat equaton solutons places restrctons on the form of dfference approxmatons that can be used to numercally solve the equaton. In partcular, schemes that are multlevel n tme wth a backward n tme) contrbuton can fal. An example of ths s the well-known second-order centered n tme) method due to Rchardson see [1]); t s uncondtonally unstable. 1.4 Dscretzaton and Grddng of PDE Problems Although n the sequel we wll consder only basc fnte-dfference methods for approxmatng solutons of partal dfferental equatons, n the present secton we shall provde bref dscussons of several of the most wdely-used dscretzaton technques. We note at the outset that temporal dscretzatons are almost always ether fnte-dfference or quadrature based, but many dfferent methods, and combnatons of these, may be employed for spatal approxmaton. In a second subsecton we present a bref overvew of technques that are wdely used to generate the dscrete grds on whch numercal approxmatons to PDEs are solved Dscretzaton technques Dscretzaton of any PDE conssts of convertng an nfnte-dmensonal soluton and the dfferental operators whch act on t to a fnte-dmensonal one, typcally but not always) n terms of grd functons and dfference operators to whch smple algebrac technques can be appled to produce approxmate solutons to the PDE. Fgure 1. depcts the man features of what are the most well-known classes of such methods: ) fnte dfference, ), fnte element and ) spectral. Fnte Dfference. As we have descrbed n consderable detal n [1], fnte-dfference methods are constructed by frst grddng the soluton doman as ndcated n Fg. 1.a), and then dervng systems of algebrac equatons for grd-pont values whch serve as approxmatons to the true soluton of the PDE at the dscrete set of ponts defned typcally) by ntersectons of the grd lnes. We remark that the regular structured grd shown here s not the only possblty. But we wll not treat fnte-dfference approxmatons on unstructured grds n the present lectures.

20 14 CHAPTER 1. INTRODUCTION y a) Ω y b) Ω,j) th grd pont Ω typcal mesh stencl Ω,j jth horzontal grd lne geometrc "element" th vertcal grd lne nodal pont x x y c) Ω Ω no geometrc doman dscretzaton requred Fgure 1.: Methods for spatal dscretzaton of partal dfferental equatons; a) fnte dfference, b) fnte element and c) spectral. x Wthn ths framework we wll often, n the sequel, use the followng dfference formula notatons ntroduced n [1] for frst-order partal dervatve approxmaton n -D. D +,x h x )u,j D,x h x )u,j u +1,j u,j h x u,j u 1,j h x D 0,x h x )u,j u +1,j u 1,j h x = u x + Oh x ), x,y j ) forward) 1.35a) = u x + Oh x ), x,y j ) backward) 1.35b) = u x + Oh x). x,y j ) centered) 1.35c) Smlarly, for second-order partal dervatves we wll employ the second centered dfference D 0,x h x)u,j = u 1,j u,j + u +1,j h x = u x + Oh x ), 1.36) x,y j ) where we note, as done n [1], that the notaton s formal, and n fact, we would actually construct ths approxmaton ether wth a one-half grd spacng, or by the composton of frst-order forward and

21 1.4. DISCRETIZATION AND GRIDDING OF PDE PROBLEMS 15 backward dfference operators, e.g., D 0,x h x) = D +,x h x )D,x h x ). Fnally, we wll typcally suppress the h x notaton n these operators when the nterpretaton of correspondng expressons s clear. Fnte Element. Fnte-element methods FEMs) are somewhat smlar to fnte-dfference methods, although they are typcally more elaborate, often somewhat more accurate, and essentally always more dffcult to mplement and less effcent. As can be seen from Fg. 1.b) the problem doman s subdvded nto small regons, often of trangular or, n 3-D, tetrahedral) shape. On each of these subregons a polynomal s used to approxmate the soluton, and varous degrees of smoothness of the approxmate soluton are acheved by requrng constructons that guarantee contnuty of a prescrbed number of dervatves across element boundares. Ths approxmaton plus the subregon on whch t apples s the element. It should be remarked that the mathematcs of FEMs s hghly developed and s based on varatonal prncples and weak solutons see, e.g., Strang and Fx [7]), n contrast to the Taylor-expanson foundatons of fnte-dfference approaches. In ths sense t bears some smlarty to Galerkn procedures. Spectral. Fgure 1.c) dsplays the stuaton for spectral methods. One should observe that n ths case there s no grd or dscrete pont set. Instead of employng what are essentally local polynomal approxmatons as done n fnte-dfference and fnte-element methods, assumed forms of the soluton functon are constructed as generalzed) Fourer representatons that are vald globally over the whole soluton doman. In ths case, the unknowns to be calculated consst of a fnte number of Fourer coeffcents leadng to what amounts to a projecton of the true soluton onto a fnte-dmensonal space. In modern termnology, these are examples of so-called grd-free methods. There are numerous other dscretzaton methods that have attaned farly wde acceptance n certan areas. In computatonal flud dynamcs CFD) fnte-volume technques are often used. These can be vewed as lyng between fnte-dfference and fnte-element methods n structure, but n fact they are essentally dentcal to fnte-dfference methods despte the rather dfferent vewpont ntegral formulaton of governng equatons) employed for ther constructon. Other less often-used approaches deservng of at least menton are spectral-element and boundary-element methods. As ther names suggest, these are also related to fnte-element methods, and partcularly n the latter case are applcable manly for only a very restrcted class of problems. Beyond these are a number of dfferent pseudo-spectral methods that are smlar to spectral methods, but for whch the bass functons employed are not necessarly egenfunctons of the prncpal part of the dfferental operators) of the problem beng consdered. In addton to all these ndvdual technques, varous calculatons have been performed usng two or more of these methods n combnaton. Partcular examples of ths nclude fnte-dfference/spectral and fnte-element/boundaryelement methods. Dscusson of detals of such schemes s beyond the ntended scope of the present lectures, and from ths pont onward our attenton wll be focused on basc fnte-dfference methods that have been wdely used because of ther nherent generalty and relatve ease of applcaton to PDE approxmaton Grddng methods As mpled above, there are two man types of grddng technques n wde use, correspondng to structured and unstructured grddng wth many dfferent varatons avalable, especally for the former of these. Here, we wll brefly outlne some of the general features of these approaches, and leave detals ncludng treatment of varous modfcatons and combnatons of these) to Chap. 5. Structured Grds. Use of structure grds nvolves labelng of grd ponts n such a way that f the ndces of any one pont are known, then the ndces of all ponts wthn the grd can be easly determned. For many years ths was the preferred n fact, essentally only) approach utlzed. It leads to very effcent, readly parallelzed numercal algorthms and straghtforward post processng. But generaton of structured grds for complcated problem domans, as arse n many engneerng applcatons, s very tme consumng n terms of human tme and thus, very expensve. Unstructured Grds. Human tme requred for grd generaton has been dramatcally reduced wth use of unstructured grds, but ths represents ther only advantage. Such grds produce solutons that are