9 Numerical Solution of Partial Differential Equations

9 Numerical Solution of Partial Differential Equations 9. Classification of Partial Differential Equations We now consider general second-order partial differential equations (PDEs) of the form Lu = au tt + bu xt + cu xx + f =, () where u is an unknown function of x and t, and a, b, c, and f are given functions. If these functions depend only on x and t, then the PDE () is called linear. If a, b, c, or f depend also on u, u x, or u t, then the PDE is called quasi-linear.. The notation used in () suggests that we think of one of the variables, t, as time, and the other, x, as space.. In principle, we could also have second-order PDEs involving more than one space dimension. However, we limit the discussion here to PDEs with a total of two independent variables. 3. Of course, a second-order PDE can also be independent of time, and contain two space variables only (such as Laplace s equation). There are three fundamentally different types of second-order quasi-linear PDEs: If b 4ac >, then L is hyperbolic. If b 4ac =, then L is parabolic. If b 4ac <, then L is elliptic. Examples:. The wave equation u tt = α u xx + f(x, t) is a second-order linear hyperbolic PDE since a, b, and c α, so that. The heat or diffusion equation b 4ac = 4α >. u t = ku xx is a second-order quasi-linear parabolic PDE since a = b, and c k, so that b 4ac =. 3. Poisson s equation (or Laplace s equation in case f ) u xx + u yy = f(x, y) is a second-order linear elliptic PDE since a = c and b, so that b 4ac = 4 <. Since a, b, and c may depend on x, t, u, u x, and u t the classification of the PDE may even vary from point to point.

9. Forward Differences for Parabolic PDEs We consider the following model problem u xx = u t, t, x, u(x, ) = g(x), x, u(, t) = α(t), t, u(, t) = β(t), t, () which can be interpreted as the model for heat conduction in a one-dimensional rod of length with prescribed end temperatures, α and β, and given initial temperature distribution g. Finite Differences We will proceed in a way very similar to our solution for two-point boundary value problems. However, now we are dealing with an initial-boundary value problem, and need to find the solution on the two-dimensional semi-infinite strip D = {(x, t) : x, t }. As before, we use a symmetric difference approximation for the second-order space derivative, i.e., for any given location x and fixed time t u xx (x, t) [u(x + h, t) u(x, t) + u(x h, t)]. (3) h For the first-order time derivative we use a forward difference approximation u t (x, t) [u(x, t + k) u(x, t)]. (4) k Next, we introduce a partition of the spatial domain x i = ih, i =,..., n +, with mesh size h = n+. Another partition is used for the time domain, i.e., where k is the time step. With the more compact notation t j = jk, j, v i,j = u(x i, t j ), v i+,j = u(x i + h, t j ), v i,j = u(x i h, t j ), v i,j+ = u(x i, t j + k). the heat equation () turns into the difference equation h [v i+,j v i,j + v i,j ] = k [v i,j+ v i,j ], i =,..., n, j =,,.... (5) For the initial condition we have v i, = g(x i ), i =,,..., n +,

whereas the boundary conditions become v,j = α(t j ), v n+,j = β(t j ), j =,,.... Remark: The discrete values v i,j will be our representation of the solution of the PDE. The values at the points along the boundaries of the domain are specified by the initial and boundary conditions, whereas the those at interior points are computed one row at a time using a stencil involving values at four mesh points as stated in (5). This is illustrated schematically in Figure. 5 4 3..4.6.8 Figure : Schematic illustration of the forward difference method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. Figure shows that the values of the approximate solution v are computed one row at a time, i.e., the new value v i,j+ at time level j + is computed from three old values at time level j. Therefore, the method we just described is a forward (in time) difference method, or an explicit time-stepping method. In order to implement formula (5) in an algorithm we solve (5) for the new value, and obtain or where s = k h v i,j+ = k h [v i+,j v i,j + v i,j ] + v i,j v i,j+ = sv i+,j + ( s)v i,j + sv i,j, i =,..., n, j =,,..., (6) is called the mesh ratio.. Note that this method is very simple since no linear systems have to be solved.. Moreover, all values at the new time level can be computed simultaneously. 3. An algorithm for this method is listed in the textbook on pages 68 69. The combined choice of mesh size h and time step k is critical for the performance of the algorithm. According to the finite difference approximations (3) and (4) of the partial derivatives we expect an accuracy of order O(k + h ). 3

We now investigate the effects of h and k on the stability of the given method. Stability Analysis There are two standard techniques used for the stability analysis of PDEs. We now illustrate them both. First we use the matrix form. We simplify the following discussion by introducing homogeneous boundary conditions, i.e., α(t) = β(t). Otherwise we would have more complicated first and last rows in the matrix below. We begin by writing (6) in matrix form, i.e., V j+ = AV j, (7) where V j = [v,j, v,j,..., v n,j ] T is a vector that contains the values of the approximate solution at all of the interior mesh points at any given time level j. The matrix A is tridiagonal, and of the form s s... s s s. A =........... s s s... s s Note, however, that A appears on the right-hand side of (7) so that no system is solved, only a matrix-vector product is formed. To investigate the stability of this method we assume that the initial data V are represented with some error E, i.e., V = Ṽ + E, where Ṽ denotes the exact initial data. Then V = AV = A(Ṽ + E ) = AṼ + AE. Advancing the solution further in time, i.e., iterating this formula, we get V j = AV j = A j Ṽ + A j E. Therefore, the initial error E is magnified by A j at the j-th time level. In order to have a stable and convergent method we need E j = A j E as j. Taking norms we get E j = A j E A j E, so that we know that stability will be ensured if the spectral radius ρ(a) <. (8) One can show (see textbook pages 6 6) that the eigenvalues of A are given by λ i = s( cos θ i ), θ i = iπ, i =,..., n. (9) n + 4

Combining (8) and (9) we require ρ(a) = max λ i = max s( cos θ i) < i=,...,n i=,...,n or < s < cos θ i for each i =,..., n. Now, the greatest restriction on s occurs if cos θ i =, in which case s <. Indeed, for i = n, θ n = nπ n+ π for n, and so cos θ i. Therefore, we have the stability requirement s /. This is summarized in Theorem 9. The forward difference method (6) is stable provided k h.. The special choice of boundary conditions makes the preceding analysis simpler, but has no effect on the stability of the method.. The stability requirement shows that a small mesh size h forces the time step k to be very small. This makes the forward difference method potentially very slow. Consider, e.g., a spatial domain [, ] with mesh size h =.. Then the time step k can be chosen no larger than 5 5. This means that in order to advance the solution from t = to t = we require time steps, or million mesh points. The second technique used for stability analyses was introduced by John von Neumann around 94 and is known as the von Neumann method or Fourier method. It is possible to show (using separation of variables) that solutions to the finite difference equation (6) are of the form v i,j = e σih Q j, () where σ could be determined by the boundary conditions. Using the principle of superposition it is clear that all solutions of (6) will contain terms with the factor Q j, for different values of j. This term will decay for large values of j (and thus indicate stability) only if Q <. We can insert the solution () into (6) and obtain e σih Q j+ = se σ(i+)h Q j + ( s)e σih Q j + se σ(i )h Q j, where, as before, s = k is the mesh ratio. Now we remove the factor e σih Q j from h the equation above which results in Q = se σh + s + se σh. 5

With the help of Euler s formula and the trigonometric formula e φ = cos φ + sin φ sin φ = ( cos φ) the expression for Q can be simplified as follows: Q = se σh + s + se σh = s + s cos σh = s ( cos σh) = 4s sin σh. From this it follows immediately that Q < (since both h and k are positive, and thus also s). To ensure also Q > we need 4s sin σh or s sin σh. Since the sine term can approach, we need to impose the stability condition s which is the same condition obtained earlier via the matrix method.. Once stability is ensured, the forward difference method converges with order O(k + h ) as stated earlier.. Computer Problem 9.. illustrates the effect the choice of mesh size h and time step k have on the stability of the forward difference method. 9. Implicit Methods for Parabolic PDEs As we learned in the context of multistep methods for initial value problems, one can expect a better stability behavior from an implicit method. Therefore, we discretize the first-order time derivative as u t (x, t) [u(x, t) u(x, t k)], () k and so the formula corresponding to (5) now becomes h [v i+,j v i,j + v i,j ] = k [v i,j v i,j ]. () 6

5 4 3..4.6.8 Figure : Schematic illustration of the backward difference method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. A typical stencil along with the values given by initial and boundary data is shown in Figure. Clearly, we now have an implicit or backward difference method. The new values of the approximate solution (at time level j) are no longer so easily found. Using the mesh ratio s = k as before, we can re-write () as h sv i,j + ( + s)v i,j sv i+,j = v i,j, i =,..., n, j =,,.... (3) If we make the simplification to homogeneous boundary conditions, i.e., α(t) = β(t), then we can write (3) in matrix form as AV j = V j, (4) where V j = [v,j, v,j,..., v n,j ] T as before, and + s s... s + s s. A =........... s + s s... s + s Since the matrix A now appears on the left-hand side in (4) we need to solve a tridiagonal linear system for each time step. However, since the mesh ratio s >, it is clear that A is diagonally dominant.. An algorithm for the backward difference method with a call to a tridiagonal solver is in the textbook on page 65.. Note that, even though the matrix A is constant for all time steps, it is more efficient to solve the triangular system anew for each time step instead of computing a factorization and using forward and back substitution. 7

Stability Analysis Under the same assumptions as for the forward difference method, the error now propagates according to V j = A V j = ( A )j Ṽ + ( A ) j E, with E the error present in the initial data. Therefore, this time we want ρ ( A ) < for stability. The eigenvalues of A are given by λ i = + s( cos θ i ), θ i = iπ, i =,..., n. n + Since < θ i < π we know that cos θ i <, and therefore λ i > for all i. Thus, ρ(a) >. Finally, since the eigenvalues of A are given by the reciprocals of the eigenvalues of A we know that ρ ( A ) <, and there is no stability restriction. Theorem 9. The backward difference method (3) is unconditionally stable.. Again, having non-homogeneous boundary conditions does not affect the stability of the method.. The accuracy of the backward difference method is O(k + h ), which implies that the time step needs to be chosen small for high accuracy. It has been suggested to use the symmetric difference approximation u t (x, t) [u(x, t + k) u(x, t k)], (5) k for the time derivative. This choice gives the following difference scheme an accuracy of order O(k + h ). Using the above discretization of the time derivative, the difference version of the heat equation is h [v i+,j v i,j + v i,j ] = k [v i,j+ v i,j ]. or v i,j+ = k h [v i+,j v i,j + v i,j ] + v i,j. (6) This method is known as Richardson s method. Richardson s method can be viewed as an explicit method that requires two rows of initial values. 8

5 4 3..4.6.8 Figure 3: Schematic illustration of Richardson s method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. The stencil for Richardson s method is shown in Figure 3. Remark: Even though Richardson s method seem like a very sensible approach, and is known to have convergence order O(k + h ) it is prone to serious stability problems. In fact, it turns out that this method is unstable for all values of the mesh ratio s = k h. Crank-Nicolson Method The most successful difference method for parabolic equations is due to John Crank and Phyllis Nicolson (947). It is based on a convex combination of the spatial terms of the forward and backward difference methods, i.e., θ h [v i+,j v i,j + v i,j ] + θ h [v i+,j v i,j + v i,j ] = k [v i,j v i,j ]. (7) This method is known as theta method or, in the case θ =, as the Crank-Nicolson method. For θ = we recover the forward difference method (with a shift in the time index j), and for θ = (7) yields the backward difference method. For the general case, the stencil contains six points at the two time levels j and j as shown in Figure 4. To get a better understanding of the Crank-Nicolson method we rewrite (7) for the case θ =. This leads to k h v i,j + ( + kh ) v i,j k h v i+,j = k h v i,j + ( kh ) v i,j + k h v i+,j. If we multiply by and introduce the mesh ratio s = k h we get sv i,j + ( + s) v i,j sv i+,j = sv i,j + ( s) v i,j + sv i+,j. (8) By restricting the discussion to the case of homogeneous boundary conditions we get a streamlined matrix form since, as before, α(t) = β(t) results in v,j = v n+,j = for all j. The resulting matrix form of (8) is AV j = BV j 9

5 4 3..4.6.8 Figure 4: Schematic illustration of the Crank-Nicolson method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. with V j = [v,j, v,j,..., v n,j ] T, + s s... s + s s. A =.........,. s + s s... s + s and s s... s s s. B =........... s s s... s s It is clear that A is a diagonally dominant tridiagonal matrix. Therefore, the implementation of the Crank-Nicolson method can be broken down into two phases:. Compute Ṽj = BV j. This can be done with an algorithm analogous to the forward difference method.. Solve AV j = Ṽj. For this we can basically use the backward difference algorithm.. Using a stability analysis as above, one can show that the Crank-Nicolson method is unconditionally stable. An outline of the argument is given in the textbook on page 66, and a detailed discussion can be found in the classical book Analysis of Numerical Methods by Isaacson and Keller (966).

. Since the averaging of the forward and backward difference methods used for the Crank-Nicolson can be interpreted as an application of the (implicit) trapezoid rule for the time discretization, it follows that the order of accuracy of the Crank- Nicolson method is O(k + h ). Therefore, this method is much more efficient (for the same accuracy) than the forward or backward difference methods by themselves. 3. If θ then the accuracy deteriorates to O(k + h ). It is possible to fix the Richardson method (6). To this end we replace the value v i,j by the average of v i,j+ and v i,j. We then end up with v i,j+ = k h + k [v i+,j + v i,j ] + h k h + k v i,j. (9) This method is known as the Dufort-Frankel method. It is stable for all values of s, and converges with order O(k + h ) as long as k/h. The stencil for the Dufort-Frankel method consists of four points, and is depicted in Figure 5. 5 4 3..4.6.8 Figure 5: Schematic illustration of the Dufort-Frankel method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil.. The Dufort-Frankel method is considered an explicit method, and therefore fast and simple to implement. There are no linear systems to solve. However, the condition k/h is essential for convergence.. Since the Dufort-Frankel method involves mesh points at three time levels, it requires a separate (possibly lower-order) method to get started.

9.3 Finite Differences for Elliptic PDEs We use Laplace s equation in the unit square as our model problem, i.e., u xx + u yy =, (x, y) = (, ), u(x, y) = g(x, y), (x, y) on. () This problem arises, e.g., when we want to determine the steady-state temperature distribution u in a square region with prescribed boundary temperature g. Of course, this simple problem can be solved analytically using Fourier series. However, we are interested in numerical methods. Therefore, in this section, we use the usual finite difference discretization of the partial derivatives, i.e., u xx (x, y) = h [u(x + h, y) u(x, y) + u(x h, y)] + O(h ) () and u yy (x, y) = h [u(x, y + h) u(x, y) + u(x, y h)] + O(h ). () The computational grid introduced in the domain = [, ] is now with mesh size h = n+. Using again the compact notation (x i, y j ) = (ih, jh), i, j =,..., n +, v i,j = u(x i, y j ), v i+,j = u(x i + h, y j ), etc., Laplace s equation () turns into the difference equation h [v i,j v i,j + v i+,j ] + h [v i,j v i,j + v i,j+ ] =. (3) This equation can be rewritten as 4v i,j v i,j v i+,j v i,j v i,j+ =. (4) This is the method referred to several times last semester. Example: Let s consider a computational mesh of 5 5 points, i.e., h = 4, or n = 3. Discretizing the boundary conditions in (), the values of the approximate solution around the boundary v,j, v 4,j j =,..., 4, v i,, v i,4 i =,..., 4, are determined by the appropriate values of g. There remain 9 points in the interior that have to be determined using the stencil (4). Figure 6 illustrates one instance of this task. By applying the stencil to each of the interior points, we obtain 9 conditions for the 9 undetermined values.

.8.6.4...4.6.8 Figure 6: Illustration for finite difference method for Laplace s equation on 5 5 grid. Interior mesh points are indicated with blue, green + correspond to given boundary values, and points marked with red form a typical stencil. Thus, we obtain the following 9 equations 4v, v, v, = v, + v, 4v, v, v 3, v, = v, 4v 3, v, v 3, = v 4, + v 3, 4v, v, v, v,3 = v, 4v, v, v 3, v, v,3 = 4v 3, v, v 3, v 3,3 = v 4, 4v,3 v,3 v, = v,4 + v,3 4v,3 v,3 v 3,3 v, = v,4 4v 3,3 v,3 v 3, = v 4,3 + v 3,4. The first equation corresponds to the stencil shown in Figure 6. The other equations are obtained by moving the stencil row-by-row across the grid from left to right. We can also write the above equations in matrix form. To this end we introduce the vector V = [v,, v,, v 3,, v,, v,, v 3,, v,3, v,3, v 3,3 ] T of unknowns. Here we have used the natural (row-by-row) ordering of the mesh points. Then we get AV = B with 4 4 4 A = 4 4 4 3 4 4 4

and B = v, + v, v, v 4, + v 3, v, v 4, v,4 + v,3 v,4 v 4,3 + v 3,4. We can see that A is a block-tridiagonal matrix of the form T I O A = I T I. O I T In general, for problems with n n interior mesh points, A will be of size n n (since there are n unknown values at interior mesh points), but contain no more than 5n nonzero entries (since equation (4) involves at most 5 points at one time). Thus, A is a classical example of a sparse matrix. Moreover, A still has a block-tridiagonal structure T I O... O I T I. A = O......... O. I T I O... O I T with n n blocks 4... 4. T =.......... 4... 4 as well as n n identity matrices I, and zero matrices O.. Since A is sparse (and symmetric positive definite) it lends itself to an application of Gauss-Seidel iteration (see Chapter 4.6 of the previous semester). After initializing the values at all mesh points (including those along the boundary) to some appropriate value (in many cases zero will work), we can simply iterate with formula (4), i.e., we obtain the algorithm fragment for M steps of Gauss-Seidel iteration 4

for k = to M do end for i = to n do end for j = to n do v i,j = (v i,j + v i+,j + v i,j + v i,j+ ) /4 end Note that the matrix A never has to be fully formed or stored during the computation.. State-of-the-art algorithms for the Laplace (or nonhomogeneous Poisson) equation are so-called fast Poisson solvers based on the fast Fourier transform, or multigrid methods. 9.4 Galerkin and Ritz Methods for Elliptic PDEs Galerkin Method We begin by introducing a generalization of the collocation method we saw earlier for two-point boundary value problems. Consider the PDE Lu(x) = f(x), (5) where L is a linear elliptic partial differential operator such as the Laplacian L = x + y + z, x = (x, y, z) IR3. At this point we will not worry about the boundary conditions that should be posed with (5). As with the collocation method we will obtain the approximate solution in the form of a function (instead of as a collection of discrete values). Therefore, we need an approximation space U = span{u,..., u n }, so that we are able to represent the approximate solution as n u = c j u j, u j U. (6) Using the linearity of L we have j= Lu = n c j Lu j. j= We now need to come up with n (linearly independent) conditions to determine the n unknown coefficients c j in (6). If {Φ,..., Φ n } is a linearly independent set of linear functionals, then n c j Lu j f =, i =,..., n, (7) Φ i j= 5

is an appropriate set of conditions. In fact, this leads to a system of linear equations Ac = b with matrix A = Φ Lu Φ Lu... Φ Lu n Φ Lu Φ Lu... Φ Lu n... Φ n Lu Φ n Lu... Φ n Lu n, coefficient vector c = [c,..., c n ] T, and right-hand side vector Φ f Φ f b =.. Φ n f Two popular choices are. Point evaluation functionals, i.e., Φ i (v) = v(x i ), where {x,..., x n } is a set of points chosen such that the resulting conditions are linearly independent, and v is some function with appropriate smoothness. With this choice (7) becomes n c j Lu j (x i ) = f(x i ), i =,..., n, j= and we now have a extension of the collocation method discussed in Section 8. to elliptic PDEs is the multi-dimensional setting.. If we let Φ i (v) = u i, v, an inner product of the basis function u i with an appropriate test function v, then (7) becomes n c j u i, Lu j = u i, f, i =,..., n. j= This is the classical Galerkin method. Ritz-Galerkin Method For the following discussion we pick as a model problem a multi-dimensional Poisson equation with homogeneous boundary conditions, i.e., u = f in, (8) u = on, with domain IR d. This problem describes, e.g., the steady-state solution of a vibrating membrane (in the case d = with shape ) fixed at the boundary, and subjected to a vertical force f. 6

The first step for the Ritz-Galerkin method is to obtain the weak form of (8). This is accomplished by choosing a function v from a space U of smooth functions, and then forming the inner product of both sides of (8) with v, i.e., u, v = f, v. (9) To be more specific, we let d = and take the inner product u, v = u(x, y)v(x, y)dxdy. Then (9) becomes (u xx (x, y) + u yy (x, y))v(x, y)dxdy = f(x, y)v(x, y)dxdy. (3) In order to be able to complete the derivation of the weak form we now assume that the space U of test functions is of the form U = {v : v C (), v = on }, i.e., besides having the necessary smoothness to be a solution of (8), the functions also satisfy the boundary conditions. Now we rewrite the left-hand side of (3): (u xx + u yy ) vdxdy = [(u x v) x + (u y v) y u x v x u y v y ] dxdy = [(u x v) x + (u y v) y ] dxdy By using Green s Theorem (integration by parts) (P x + Q y )dxdy = (P dy Qdx) the first integral on the right-hand side of (3) turns into [(u x v) x + (u y v) y ] dxdy = (u x vdy u y vdx). [u x v x u y v y ] dxdy.(3) Now the special choice of U, i.e., the fact that v satisfies the boundary conditions, ensures that this term vanishes. Therefore, the weak form of (8) is given by [u x v x + u y v y ] dxdy = fvdxdy. Another way of writing the previous formula is of course u vdxdy = fvdxdy. (3) 7

To obtain a numerical method we now need to require U to be finite-dimensional with basis {u,..., u n }. Then we can represent the approximate solution u h of (8) as u h = n c j u j. (33) j= The superscript h indicates that the approximate solution is obtained on some underlying discretization of with mesh size h.. In practice there are many ways of discretizing and selecting U. (a) For example, regular (tensor product) grids can be used. Then U can consist of tensor products of piecewise polynomials or B-spline functions that satisfy the boundary conditions of the PDE. (b) It is also possible to use irregular (triangulated) meshes, and again define piecewise (total degree) polynomials or splines on triangulations satisfying the boundary conditions. (c) More recently, meshfree approximation methods have been introduced as possible choices for U.. In the literature the piecewise polynomial approach is usually referred to as the finite element method. 3. The discretization of will almost always result in a computational domain that has piecewise linear (Lipschitz-continuous) boundary. We now return to the discussion of the general numerical method. Once we have chosen a basis for the approximation space U, then it becomes our goal to determine the coefficients c j in (33). By inserting u h into the weak form (3), and selecting as trial functions v the basis functions of U we obtain a system of equations u h u i dxdy = fu i dxdy, i =,..., n. Using the representation (33) of u h we get n c j u j u i dxdy = or by linearity j= n c j j= u j u i dxdy = fu i dxdy, i =,..., n, fu i dxdy, i =,..., n. (34) This last set of equations is known as the Ritz-Galerkin method and can be written in matrix form Ac = b, 8

where the stiffness matrix A has entries A i,j = u j u i dxdy.. The stiffness matrix is usually assembled element by element, i.e., the contribution to the integral over is split into contributions for each element (e.g., rectangle or triangle) of the underlying mesh.. Depending on the choice of the (finite-dimensional) approximation space U and underlying discretization, the matrix will have a well-defined structure. This is one of the most important applications driving the design of efficient linear system solvers. How does solving the Ritz-Galerkin equations (34) relate to the solution of the strong form (8) of the PDE? First, we remark that the left-hand side of (3) can be interpreted as a new inner product [u, v] = u vdxdy (35) on the space of functions whose first derivatives are square integrable and that vanish on. This space is a Sobolev space, usually denoted by H (). The inner product [, ] induces a norm v = [v, v] / on H (). Now, using this norm, the best approximation to u from H () is given by the function uh that minimizes u u h. Since we define our numerical method via the finite-dimensional subspace U of H (), we need to find uh such that u u h U or, using the basis of U, ] [u u h, u i =, i =,..., n. Replacing u h with its expansion in terms of the basis of U we have n u c j u j, u i =, i =,..., n, j= or n c j [u j, u i ] = [u, u i ], i =,..., n. (36) j= The right-hand side of this formula contains the exact solution u, and therefore is not useful for a numerical scheme. However, by (35) and the weak form (3) we have [u, u i ] = u u i dxdy 9

= fu i dxdy. Since the last expression corresponds to the inner product f, u i, (36) can be viewed as n c j [u j, u i ] = f, u i, i =,..., n, j= which is nothing but the Ritz-Galerkin method (34). The best approximation property in the Sobolev space H () can also be interpreted as an energy minimization principle. In fact, a smooth solution of the Poisson problem (8) minimizes the energy functional E(u) = udxdy fudxdy over all smooth functions that vanish on the boundary of. By considering the energy of nearby solutions u + λv, with arbitrary real λ we see that E(u + λv) = (u + λv) (u + λv)dxdy f(u + λv)dxdy = u udxdy + λ u vdxdy + λ v vdxdy fudxdy λ fvdxdy = E(u) + λ [ u v fv] dxdy + λ vdxdy The right-hand side is a quadratic polynomial in λ, so that for a minimum, the term [ u v fv] dxdy must vanish for all v. This is again the weak formulation (3). A discrete energy norm is then given by the quadratic form E(u h ) = ct Ac bc where A is the stiffness matrix, and c is such that the Ritz-Galerkin system (34) is satisfied. Ac = b

Example: One of the most popular finite element versions is based on the use of piecewise linear C polynomials (built either on a regular grid, or on a triangular partition of ). The basis functions u i are hat functions, i.e., functions that are piecewise linear, have value one at one of the vertices, and zero at all of its neighbors. This choice makes it very easy to satisfy the homogeneous Dirichlet boundary conditions of the model problem exactly (along a polygonal boundary). Since the gradients of piecewise linear functions are constant, the entries of the stiffness matrix essentially boil down to the areas of the underlying mesh elements. Therefore, in this case, the Ritz-Galerkin method is very easily implemented. We generate some examples with Matlab s PDE toolbox pdetool. It is not difficult to verify that the stiffness matrix for our example is symmetric and positive definite. Since the matrix is also very sparse due to the fact that the hat basis functions have a very localized support, efficient iterative solvers can be applied. Moreover, it is known that the piecewise linear FEM converges with order O(h ).. The Ritz-Galerkin method was independently introduced by Walther Ritz (98) and Boris Galerkin (95).. In the textbook this method is referred to as the Rayleigh-Ritz method. Many variations in terminology seem to exist indicating only slight (or no) differences in methodology. 3. The finite element method is one of the most-thoroughly studied numerical methods. Many textbooks on the subject exist, e.g., The Mathematical Theory of Finite Element Methods by Brenner and Scott (994), An Analysis of the Finite Element Method by Strang and Fix (973), or The Finite Element Method by Zienkiewicz and Taylor (). 4. Even though the use of piecewise linear finite elements is very simple, there are many problems that arise with higher-order methods, mesh refinement, or with the generation of finite element meshes in higher space dimensions. This is where meshfree approximation methods are trying to establish themselves as a viable alternative. 9.5 Characteristics We start with the simple advection equation u t (x, t) = cu x (x, t), < x <, t, (37) where c is some nonzero constant. This problem has a unique solution as soon as we specify an initial condition u(x, ) = f(x), < x <.

A pure initial value problem of this type is called a Cauchy problem. It is easily verified (using the chain rule) that the solution to this problem is given by u(x, t) = f(x ct). This shows us that the solution u is simply a shifted version of the initial profile f. In fact, the initial profile moves to the right with velocity c. Therefore, the PDE (37) is also referred to as the transport equation or one-way wave equation. A characteristic curve or simply characteristic is a curve in the (x, t)-plane on which the solution u of a PDE is constant. For the advection equation we have d dt u(x(t), t) = u dx(t) x + u t. (38) dt Since u t = cu x we see that the solution u is constant, i.e., d u(x(t), t) = dt if dx(t) = c. (39) dt This is an ODE that determines the characteristic x = x(t) of the advection equation. From (39) it is clear that the characteristics of the advection equation are the lines x(t) = ct + x. Remark: The left-hand side of equation (38) can be interpreted as the change as experienced by a moving observer, whereas the two parts on the right-hand side represent the change due to the fact that the the observer is moving into a region of possibly different u, and the change of the solution at a fixed point. Our discussion of characteristics shows that an observer moving along with speed c will see no change in the solution. Characteristics give us fundamental insight into the solution of a partial differential equation. Assume, e.g., that we have specified initial values for the advection equation (37) only on the finite interval x. If we consider the domain {(x, t) : x, t }, then the given initial values will determine the solution only in part of this domain. The left part of Figure 7 illustrates this fact for a positive value of c. The right part of Figure 7 shows how the solution propagates along characteristics. This is also clear when we recall that a characteristic curve is the location of points in the (x, t)-plane along which the solution of the PDE is constant. In order to obtain the solution on the remainder (unshaded part) of the domain we need to specify boundary values along the line x =. However, specifying values along the other boundary, x =, would possibly overdetermine the problem. If we want to study characteristics for general second-order quasi-linear PDEs then the situation is more complicated.

3.5 t.5.5 t.5..4.6.8.5..5.4.6 x.8 x Figure 7: Left: Characteristics (blue lines) and domain of influence (red area) for the advection equation. Right: Typical propagation of solution of the advection equation. If we consider the wave equation u tt (x, t) = α u xx (x, t), < x <, t, then we can factor the differential operator α t x = ( t α x ) ( t + α ). x Therefore, the wave equation will be satisfied if either of the two advection equations is satisfied. u t (x, t) = αu x (x, t) or u t (x, t) = αu x (x, t) Remark: This is the traveling wave solution of the wave equation found already by d Alembert in the 8th century. We illustrate this in the Maple worksheet 578 TravelingWaves.mws. Characteristics for more general second-order quasi-linear equations are discussed in Section 9.6 of the textbook. We just mention that if the PDE is of the form au tt + bu xt + cu xx + e = then the characteristic curves are determined by the nonlinear first-order ordinary differential equation ( ) dx(t) a b dx(t) + c =. dt dt Remark: It is the discriminant of this quadratic equation that gives rise to the classification of second-order quasi-linear PDEs into hyperbolic, elliptic, and parabolic types as discussed in Section 9.. Elliptic problems, that do not depend on time, i.e., describe steady-state problems, have no real characteristics. Parabolic equations, that are time-dependent (and generally evolve toward a steady-state) have one family of characteristics. And hyperbolic 3

PDEs, that are time-dependent (but do not evolve toward a steady-state) have two independent families of characteristics. If these characteristics cross, then the solution can develop discontinuities or shocks (even for smooth initial data). This is one of the major difficulties associated with the solution of hyperbolic PDEs. Since hyperbolic PDEs are conservative, i.e., the energy of the system remains constant over time (such as in the perpetual vibrations for the wave equation), it is important to ensure that numerical schemes are also conservative. We will follow up on this idea in Section 9.7. Remark: A numerical method, called the method of characteristics, asks the user to determine the ordinary differential equation that defines the characteristics of a given partial differential equations. Then a numerical ODE solver is used to solve this ODE, resulting in level curves for the solution. Once these level curves are found, it is possible to use another ODE solver to integrate the PDE (which is now an ODE) along the characteristic (see (39)). 9.6 Finite Differences for Hyperbolic PDEs Before we study specialized numerical methods for conservation laws, we take a look at a straightforward finite difference scheme. This material is touched on in Section 9.7 of the textbook. We now consider as a model problem the one-dimensional wave equation u tt (x, t) = α u xx (x, t), < x <, t, u(x, ) = f(x), x, u t (x, ) = g(x), x, u(, t) = u(, t) =, t. This model is used to describe the vibrations of a string of unit length fixed at both ends subject to an initial displacement f(x) and initial velocity g(x). In order to be able to formulate finite differences we need a discrete approximation of the second derivatives. We use the symmetric formulas u tt = k [u(x, t + k) u(x, t) + u(x, t k)] + O(k ) and u xx = h [u(x + h, t) u(x, t) + u(x h, t)] + O(h ). Here we have discretized the computational domain with x i = ih, i =,..., n +, t j = jk, j. As always, h = n+ is the mesh size and k is the time step. Also, we use the notation v i,j = u(x i, t j ), etc.. Inserting the finite difference approximations into the wave equation we get k [v i,j+ v i,j + v i,j ] = α h [v i+,j v i,j + v i,j ]. (4) 4

Finally, we can introduce the mesh ratio s = ( ) αk, h and solve (4) for the value at time level j +, i.e., v i,j+ = ( s)v i,j + s(v i,j + v i+,j ) v i,j. The stencil for this method is shown in Figure 8. 5 4 3..4.6.8 Figure 8: Finite difference stencil for wave equation. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil.. From the finite difference approximations employed it is clear that our method has accuracy O(k + h ).. As for Richardson s method and the Dufort-Frankel method studied earlier, there is a problem starting this method. Some additional procedure has to be applied to generate the second row of starting values. In the context of the Dufort-Frankel method we suggested that this was most likely accomplished using a lower-order method. However, now we have the advantage that not only the values of u are specified at the j = level, but also values of the time derivative. We should be able to use this to our advantage. We can attempt to use the backward Euler discretization for the first-order time derivative, i.e., u t (x i, ) u(x i, t ) u(x i, ), k or u(x i, t ) u(x i, ) + ku t (x i, ). Using the given initial velocity this becomes which gives us the numerical scheme u(x i, t ) u(x i, ) + kg(x i ) v i, = v i, + kg(x i ), i =,..., n. 5

This lets us advance the information given at the j = level to the first row of the computational mesh at j =. However, the backward Euler discretization is order O(k) only, so that the entire numerical method will be O(k + h ) only. In order to get a second-order accurate formula for this first step we employ a Taylor expansion of u with respect to the time variable, i.e., Now, the wave equation tells us that u(x i, t ) u(x i, ) + ku t (x i, ) + k u tt(x i, ). (4) u tt (x i, ) = α u xx (x i, ). Moreover, since u = f for time t = we get (provided f is smooth enough) and so the Taylor expansion (4) becomes u tt (x i, ) = α f (x i ), u(x i, t ) u(x i, ) + ku t (x i, ) + k α This allows us to express the values at the j = level as v i, = v i, + kg(x i ) + k α f (x i ). f (x i ). Using again a symmetric finite difference approximation for f we have v i, = v i, + kg(x i ) + k α Since f(x i ) = u(x i, ) = v i, we arrive at h [f(x i+) f(x i ) + f(x i )]. v i, = ( s)v i, + s [v i, + v i+, ] + kg(x i ), (4) where we have used the mesh ratio s = ( ) αk. h. Formula (4) is second-order accurate in time. Therefore, the finite difference method consisting of (4) for the first step, and (4) for subsequent steps is of order O(k + h ).. Once the first two rows of values have be obtained, the method proceeds in an explicit way, i.e., no linear systems have to be solved. 3. However, as with all explicit methods there are restrictions on the mesh ratio in order to maintain stability. For this method we have s. 6

9.7 Flux Conservation for Hyperbolic PDEs We mentioned earlier that hyperbolic partial differential equations are conservative. This was clearly illustrated by the simple advection equation (37). It is customary to re-write hyperbolic PDEs in the so-called flux conservative form. We illustrate this with the wave equation If we introduce the new variables then so that v t = αw x. Since we also have u tt (x, t) = α u xx (x, t), < x <, t. v = αu x, and w = u t v t = αu xt w x = u tx v x = αu xx w t = u tt we see that the wave equation is equivalent to a system of two coupled first-order partial differential equations v t = αw x w t = αv x. Remark: In physics these equations arise in the study of one-dimensional electromagnetic waves, and are known as Maxwell s equations. To get the standard flux conservative form of a hyperbolic conservation law we introduce matrix-vector notation [ ] [ ] v α u =, F (u) = u, w α so that u t = F (u) x. The function F (u) is called the flux of the problem.. If the system consists of only one equation, then we recover the advection equation (37).. An excellent book on hyperbolic conservation laws (covering theory and numerics) is Numerical Methods for Conservation Laws by Randy LeVeque (99). 7

We now investigate numerical methods for the simple conservation law u t (x, t) = cu x (x, t), < x <, t, u(x, ) = f(x), x. The most natural finite difference approach is based on the derivative approximations Using the standard discretization u t (x, t) = [u(x, t + k) u(x, t)] + O(k) k u x (x, t) = h [u(x + h, t) u(x h, t)] + O(h ). x i = ih, i =,..., n +, t j = jk, j, with mesh size h = n+ and time step k, as well as the notation v i,j = u(x i, t j ), we obtain k [v i,j+ v i,j ] = c h [v i+,j v i,j ] or v i,j+ = kc h [v i+,j v i,j ] + v i,j. (43). The method (43) is an explicit, and therefore fast method.. However, as we will show next, this method is completely useless since it is unconditionally unstable. 3. We use the advection equation with a smooth step function as initial condition to illustrate the performance of this method. The (slightly modified) Matlab program transport.m (with menu option forward Euler in time / centered differences in space ) originally written by Stefan Hüeber, University of Stuttgart, Germany, serves this purpose. We now perform the von Neumann stability analysis to verify this claim. assume that the solution of the model problem is of the form v i,j = e iβh e jλk = e ( iβh e λk) j Let s (44) obtained via separation of variables (the so-called plane wave solution). The question is what the long time behavior of this solution is, i.e., we are interested in lim j v i,j. Since (44) contains an oscillatory as well as an exponentially growing component, we will need e λk for stability. 8

In order to compute the amplification factor e λk we substitute (44) into our numerical method (43). This yields e iβh e (j+)λk = kc [ e (i+)βh e ] (i )βh e jλk + e iβh e jλk. h Division by e iβh e jλk gives us e λk = kc [ e ] βh e βh + h = kc sin βh. (45) h Here we have used the complex trigonometric identity From (45) it is clear that sin z = e z e z. e λk for all positive time steps k since e λk lies somewhere on the line R(z) = in the complex plane. Therefore the method is as claimed unconditionally unstable. Lax-Friedrichs Method Earlier, we were able to fix Richardson s method for parabolic PDEs by replacing v i,j by an average of neighboring points (in time). This led to the Dufort-Frankel method. Now we replace v i,j in (43) by an average of neighboring values in space, i.e., we insert v i,j = v i+,j + v i,j into (43). That results in v i,j+ = v i+,j + v i,j kc h [v i+,j v i,j ]. (46) This is known as the Lax-Friedrichs method due to Peter Lax and Kurt Friedrichs. Introducing, as always, the mesh ratio s = kc h (46) becomes ( ) ( ) v i,j+ = s v i+,j + + s v i,j The Lax-Friedrich method is clearly an explicit method, and its three point stencil is shown in Figure 9. The first part of the stability analysis for this method is as above. However, now we get an amplification factor of e λk = cos βh kc h sin βh, 9

5 4 3..4.6.8 Figure 9: Schematic illustration of the Lax-Friedrichs method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. and stability will be ensured if This condition is equivalent to e λk. or cos βh + ( ) kc sin βh h ( ) kc cos βh h sin =. βh Therefore, stability of the Lax-Friedrichs method is ensured if k c h, or (assuming positive c), s. This condition is known in the literature as the CFL condition (or Courant-Friedrichs- Levy condition).. As indicated in Section 9.6, the characteristics of a hyperbolic PDE define a domain of dependence, i.e., that part of the domain that influences the solution at a given point. The CFL condition says that for each mesh point the domain of dependence of the numerical scheme must contain the domain of dependence of the PDE.. The accuracy of the Lax-Friedrichs method is O(k + h ), i.e., only first-order accurate in time. 3

3. We use the Matlab program transport.m (with menu option Lax-Friedrichs ) on the advection equation with a smooth step function as initial condition to illustrate the performance of this method. We now present an alternate interpretation of the Lax-Friedrichs method. To this end we rewrite (46). By subtracting the term v i,j from both sides of (46) we get [ ] v i,j+ v i,j vi+,j v i,j = c + [ ] vi+,j v i,j + v i,j. (47) k h k We can now interpret (47) as a finite difference discretization of the PDE u t (x, t) = cu x (x, t) + h k u xx(x, t) using symmetric approximation for the space derivatives, and a forward difference in time. According to this interpretation we have implicitly added a diffusion term to the advection equation. This process is referred to as adding numerical viscosity (or artificial viscosity) to a hyperbolic problem. By doing so we introduce stability in the numerical method, but destroy its conservation properties. The effect is that sharp discontinuites in the solution (shocks) are smeared out. The diffusive effect of the Lax-Friedrichs method can also be observed by looking at the amplification factor. Since e λk { =, c k = h, <, c k < h, the amplitude of the wave decreases, i.e., energy is lost, if the time step is taken too small. Remark: For certain hyperbolic problems the diffusive effect is not much of an issue since for those problems one is interested in the case when βh, so that e λk. Lax-Wendroff Method We now present another finite difference scheme for conservation laws named after Peter Lax and Burt Wendroff. We start with the Taylor expansion u(x, t + k) = u(x, t) + ku t (x, t) + k u tt(x, t) +.... Using the conservation equation u t = cu x this becomes u(x, t + k) = u(x, t) cku x (x, t) + (ck) u xx(x, t) +.... If we keep the terms explicitly listed, then the truncation error in the time component obviously is O(k ). With the standard notation v i,j = u(x i, t j ) this expansion gives rise to the numerical scheme v i,j+ = v i,j ck v i+,j v i,j h 3 + (ck) v i+,j v i,j + v i,j h,

where we have used O(h ) approximations for both the first-order and second-order spatial derivatives. Therefore, introducing the mesh ratio s = ck h we get the Lax- Wendroff method v i,j+ = ( s s ) v i+,j + ( 4s ) v i,j + ( s + s ) v i,j. (48) This method enjoys O(k + h ) accuracy, is an explicit method, and involves a stencil as shown in Figure. 5 4 3..4.6.8 Figure : Schematic illustration of the Lax-Wendroff method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil. For the stability analysis we again begin with a plane wave solution of the form v i,j = e iβh ( e λk) j. Inserting this into the Lax-Wendroff scheme yields e iβh e (j+)λk = ( s s ) e (i+)βh e jλk + ( 4s ) e iβh e jλk + ( s + s ) e (i )βh e jλk. Next, we divide by e iβh e (j+)λk so that e λk = ( s s ) e βh + ( 4s ) + ( s + s ) ( e βh = s e ) ( βh + e βh s e ) βh e βh + 4s = 4s cos βh s sin βh + 4s, due to the trigonometric identities cos z = e z + e z Now, a straightforward (but lengthy) calculation yields e λk = 6s sin βh, sin z = e z e z. ( 4s sin βh ) βh + cos. 3

Stability will be ensured if the term in parentheses is non-negative, i.e., 4s sin βh + cos βh. It is easy to show that the expression on the left-hand side takes its minimum for βh = π. The minimum value is then 4s, and it will be nonnegative as long as s. This is the CFL condition encountered earlier. Remark: We use the Matlab program transport.m (with menu option Lax-Wendroff ) on the advection equation with a smooth step function as initial condition to illustrate the performance of this method. Leapfrog Method Another method that is second-order accurate in time and space for the advection equation can be found by using symmetric approximations to the partial derivatives, i.e., or v i,j+ v i,j k = c v i+,j v i,j h v i,j+ = ck h [v i+,j v i,j ] + v i,j. (49) This method has its name from the fact that the values used in the stencil (see Figure ) leapfrog over the missing central point v i,j. 5 4 3..4.6.8 Figure : Schematic illustration of the leapfrog method. Basic mesh points are indicated with blue, green + correspond to given values, and points marked with red form a typical stencil.. The leapfrog method requires more storage than the previous methods since the two most recent rows of values need to be kept in memory. 33

. Also, this method requires a special start-up step (which may be of lower order). The stability analysis of the leapfrog method involves the solution of a quadratic equation, and leads to the amplification factor so that e λk = ck h sin βh ± ( ) ck e λk = sin βh + h =. ( ) ck sin βh h ( ) ck sin βh h Therefore, the leapfrog method is stable for our model problem for all mesh ratios s = ck h, and does not involve any numerical diffusivity. However, for more complicated (nonlinear) problems this method becomes unstable. This instability is due to the fact that the stencil isolates the black from the white mesh points. This leads to so-called mesh drifting. Remark: By combining the leapfrog method with the Lax-Friedrichs method it is possible to get a two-step Lax-Wendroff type method which has accuracy O(k + h ) and is stable (i.e., does not suffer from mesh drifting). As the examples viewed earlier with the help of the Matlab program transport.m have shown, most of the methods presented so far are not capable of advecting a sharp transition accurately. The numerical solution of the test problem u t (x, t) = u x (x, t), x, t, with initial profile ( ) (.5 x) u(x, ) = arctan, x,. as displayed in Figure was not handled at all by the Euler/centered difference method (43), was significantly smeared out by the Lax-Friedrichs method (46) (see left part of Figure 3), and suffered from Gibbs-like oscillations with the Lax-Wendroff method (48) (see right part of Figure 3). Remark: Use of the leapfrog method (49) will result in oscillations similar to the Lax-Wendroff method. 34