Verifying Numerical Convergence Rates

1 Order of accuracy Verifying Numerical Convergence Rates We consider a numerical approximation of an exact value u. Te approximation depends on a small parameter, suc as te grid size or time step, and we denote it by ũ. If te numerical metod is of order p, we mean tat tere is a number C independent of suc tat ũ u C p, (1) at least for sufficiently small. We also say tat te convergence rate of te metod is p. (Te number C typically depends on te exact solution.) Often te error u ũ depends smootly on. Ten ũ u = C p + O ( p+1). (2) We will assume tis encefort. Example 1 In te trapezoidal rule we approximate te exact integral u = b a f(x)dx, by a sum ũ = N 1 2 f(a) + j=1 f(a + j) + 2 f(b), = b a N. For sufficiently smoot functions f(x) tis is a second order metod and ũ u = C 2 + O( 3 ). 2 Determining te order of accuracy We are often faced wit te problem of ow to determine te order p given a sequence of approximations ũ 1,ũ 2,... Tis is can be a good ceck tat a metod is correctly implemented (if p is known) and also a way to get a feeling for te credibility of an approximation ũ (ig p means ig credibility). We can eiter be in te situation tat te exact value u is known, or, more commonly, tat u is unknown. 2.1 Known u If te exact value u is known, it is quite obvious ow to do tis. Ten we just ceck te sequence log ũ u = log C + p log + O(), for 1, 2,...and fit it to a linear function of log to approximate p. A quick way to do tis is to plot ũ u as a function of in a loglog plot and determine te slope of te line tat 1 (9)

u u 10 2 2 10 3 10 4 10 2 10 1 Figure 1. Error in trapezoidal rule for f(x) = sin(x). Te dased line is 2 wic indicates te slope for a second order metod. appears. Te standard way to get a precise number for p is to alve te parameter and look at te ratios of te errors u ũ and u ũ /2, Hence ũ u ũ /2 u = C p + O( p+1 ) C(/2) p + O((/2) p+1 ) = 2p + O(). ũ u log 2 ũ /2 u = p + O(). Example 2 Te exact integral of sin(x) over [0,π] equals two. Computing ũ wit te trapezoidal rule and plotting ũ 2 in a loglog plot we get te result sown in Figure 1. 2.2 Unknown u Wen u is not known tere are two main approaces. Te first one is to compute a numerical reference solution wit a very small and ten proceed as in te case of a known u. Tis can be quite an expensive strategy if ũ is costly to compute. Using p to gauge te credibility of a ũ is also less relevant wen we already ave a good reference solution. Te second approac is to look at ratios of differences between ũ computed for different. Most commonly we compare solutions were is alved succesively. Ten we get 2 (9) ũ ũ /2 ũ /2 ũ /4 = Cp C(/2) p + O( p+1 ) C(/2) p C(/4) p + O( p+1 ) = 1 2 p + O() 2 p 2 2p + O() = 2p + O(). (3)

ũ ũ ũ ũ ũ /2 ũ /2 ũ /2 ũ /4 log ũ /2 2 ũ /2 ũ /4 π/5 1.933765598092805-0.049757939416650 4.024930251575880 2.008963782835339 π/10 1.983523537509455-0.012362435199260 4.006184396966857 2.002228827158397 π/20 1.995885972708715-0.003085837788350 4.001543117204195 2.000556454557076 π/40 1.998971810497066-0.000771161948770 4.000385593360853 2.000139066704584 π/80 1.999742972445836-0.000192771904301 4.000096386716427 2.000034763740606 π/160 1.999935744350136-0.000048191814813 π/320 1.999983936164949 Table 1. Table of values for te trapezoidal rule for f(x) = sin(x). Te last column is te final approximation of te order of accuracy p. Hence, after computing ũ for, /2 and /4 we can evaluate te expression above and get an estimate of p. We can do it similarly for oter grid sizes, e.g., α, α 2 gives ũ ũ α ũ α ũ α 2 = Cp C(α) p + O( p+1 ) C(α) p C(α 2 ) p + O( p+1 ) = 1 αp + O() α p α 2p + O() = α p + O(). (4) Example 3 Consider again Example 2. If te exact integral value was not known we would look at te values computed by te trapezoidal rule and ceck te ratios of differences as above. Te result is summarized in Table 1. 3 Asymptotic region We note tat te estimates of p in all te metods above gets better as 0 because of te O() term. (Te precise value is only given in te limit 0.) We say tat te metod is in its asymptotic region of accuracy wen is small enoug to give a good estimate of p ten te O( p+1 ) term in (2) is significantly smaller tan C p. Tis required size of can, owever, be quite different for different problems. To verify tat we are indeed in te asymptotic region, it can be valuable to make te estimate of p for several different and ceck tat we get approximately te same value. Usually one terefore computes ũ not just for tree values of, but for a longer sequence,,/2,/4,/8,/16,... and compares te corresponding ratios, ũ ũ /2 ũ /2 ũ /4, ũ /2 ũ /4 ũ /4 ũ /8, ũ /4 ũ /8 ũ /8 ũ /16,... Similarly, if u is known one considers u ũ for several decreasing values of wen fitting te line. Example 4 If we perform te same experiments as in Example 2 and Example 3 above, but wit f(x) = sin(31x) te constant C will be muc bigger, meaning tat te asymptotic region is sifted to smaller. Te results are sown in Figure 2 and Table 2. It is not until < π/40 10 1 tat te numbers start to look reasonable. Te general size of te error is also muc larger tan in Figure 2 because of te bigger C. 3 (9)

10 0 u u 2 10 1 10 2 10 3 10 2 10 1 Figure 2. Error in trapezoidal rule for f(x) = sin(31x). Te dased line is 2 wic indicates te slope for a second order metod. ũ ũ ũ ũ ũ /2 ũ /2 ũ /2 ũ /4 log ũ /2 2 ũ /2 ũ /4 π/5 1.933765598092808 1.983523537509458 14.784906442999516 3.886053209184444 π/10-0.049757939416650 0.134158680351247-0.630173999781565 π/20-0.183916619767896-0.212891487744257 7.778391902691306 2.959471924644287 π/40 0.028974867976361-0.027369601635860 4.437830912882666 2.149854700028653 π/80 0.056344469612220-0.006167337641551 4.096338487974619 2.034334932805155 π/160 0.062511807253771-0.001505573247830 π/320 0.064017380501601 Table 2. Table of values for te trapezoidal rule for f(x) = sin(31x). Te last column is te final approximation of te order of accuracy p. 4 (9)

/4 /2 x * x * x 1 x* (a) (b) Figure 3. Pointwise approximations. 4 Grid functions Wen solving differential equations te numerical solution in question is often a grid function u j wic approximates a continuous function on a grid {x j }. We assume tat te grid is uniform, wit x j = + j for some and fixed, cf. Figure 3a. Te grid size is restricted to values suc tat te grid fits te boundary, typically = d/n for a fixed domain size d and integer N of our coice. We will write u j () to indicate te dependence on. To ceck convergence rates for tese problems it is very important tat we compare wit te same ting wen we cange. Tis can be a bit tricky. 4.1 Pointwise values In a finite difference sceme we would approximate pointwise values, u j () = u(x j ) + C(x j ) p + O( p+1 ), were C now depends on te spatial location x j. Suppose we want to ceck pointwise convergence. Wen computing te ratios of differences in (3) te solutions for different grids must be compared in exactly te same points. Let x, x and x be te grid points were te solution is compared for te grid sizes, and, respectively. Suppose x = + j. If we alve we must ten precisely double j to stay at te same grid point. Hence, if = /2 and = /2 = /4, ten x = + 2j /2 = x and x = + 4j /4 = x, see Figure 3b. We obtain u j () u 2j (/2) u 2j (/2) u 4j (/4) = u(x ) + C(x ) p u(x ) C(x ) u(x ) + C(x )( ) p u(x ) C(x 2 ( ) p 2 + O( p+1 ) )( 4 ) p + O( p+1 ) = 2p + O(), as before in Section 2.2. However, if we are not careful and te grid points used are just one index off, an error will be introduced wic completely ruins te estimate. Example 5 To apply Neumann boundary conditions one often sifts te entire grid by alf a cell, giving = /2 as in Figure 4a. Hence, now depends on. As before we let te measuring point be x = + j = /2 + j. Ten if we do te same ting ere, alving and doubling j, we get tree sligtly different grid points were te solutions are compared: x, x = /4 + 2j /2 = x /4 and x = /8 + 4j /4 = x 3/8 (see Figure 4b). Tis gives an 5 (9)

/4 /2 x * x * x 1 x* (a) (b) Figure 4. Pointwise approximations, sifted grid. error in te ratio of differences wic prevents it from predicting te convergence rate: u j +1() u 2j (/2) u 2j (/2) u 4j (/4) = u(x ) + C(x ) p u(x ) C(x ) ( ) p 2 + O( p+1 ) u(x ) + C(x ) ( ) p 2 u(x ) C(x ) ( p 4) + O( p+1 ) = u(x ) u(x ) + C(x ) p (1 2 p ) + O( p+1 ) u(x ) u(x ) + C(x ) p (2 p 2 2p ) + O( p+1 ) = (x x )u x (x ) + O( 2 ) (x x )u x (x ) + O( 2 ) = 2 + O(). Hence, regardless of te actual order p, te estimate would just indicate first order convergence. Example 6 In time stepping metods for ODEs we often want to ceck convergence at a fixed time T. Te time step t must ten be cosen suc tat T is exactly a multiple of t. Oterwise we get te same problem as in Example 5, and ratios of differences will always indicate first order convergence. One sould terefore avoid setting t in te code, but rater set te number of time steps and compute t from tis. One particular pitfall is te common practice of doubling te number of unknowns N in a problem, rater tan alving te grid size. Often = d/n and te two approaces are equivalent. However, depending on boundary conditions we can also ave for instance = d/(n + 1) or = d/(n 1). Ten if N is doubled, we get te wrong order of convergence from our tests. Remark 1 Using interpolation is one way to avoid te problems associated wit coosing te rigt grid sizes. Ten we can evaluate any grid function at an arbitrary point, and tese values can ten readily be compared. However, it is important tat te order of interpolation is at least as ig as te order of te metod we are studying. Oterwise te interpolation error will dominate. 4.2 Local averages In a finite volume sceme we would approximate local averages over grid cells [x j /2,x j +/2], u j () = 1 xj +/2 x j /2 u(x)dx + C(x j ) p + O( p+1 ). See Figure 5. In tis case also te exact value tat we compare wit canges as we refine te grid. We can deal wit tis in two ways. 6 (9)

x 1 (a) x 1 (b) Figure 5. Local averages. First, if p 2 we can use te fact tat te value in te mid point of te cell is a second order approximation of te average wen te function is smoot, and terefore 1 xj +/2 x j /2 u(x)dx = u(x j ) + C (x j ) 2 + O( 3 ). u j () = u(x j ) + C(x j ) p + O( p+1 ), possibly wit a different C(x). Hence, tis takes us back to te same considerations as for pointwise values in Section 4.1. Note tat also ere te grid can be sifted, see Figure 5b. Second, wen p > 2 we must really compare local averages instead of pointwise values. Tis means tat u j () must be compared wit te average of two computed values wen is alved, 1 2 (u j (/2) + u j (/2)) for some j, and four values wen is alved again. In tese comparisons it is important ten tat we consider te same interval in every grid. For instance, let x, x and x be te left edges of tree cells in te grids wit cell sizes, /2 and /4. Suppose we ave te sifted grid in Figure 5b and tat x = x j /2 = /2+j /2 = j. As in te pointwise case we can double te index, taking 2j and 4j, suc tat all tree points are equal x = x = x. Our first value I for grid size is ten I = u j () = 1 For grid size /2 we need two values, x + I /2 = u 2j (/2) + u 2j +1(/2) 2 = 1 + 1 = 1 x +/2 x u(x)dx + C(x + /2) p + O( p+1 ). x u(x)dx + 1 2 C(x + /4)(/2) p + O( p+1 ) x + x +/2 x + Finally, for grid size /4 one can ceck tat I /4 = 1 4 u(x)dx + 1 2 C(x + 3/4)(/2) p + O( p+1 ) x u(x)dx + C(x + /2)(/2) p + O( p+1 ). 3 u 4j +k(/4) = 1 k=0 It follows as before in (3) tat x + x u(x)dx + C(x + /2)(/4) p + O( p+1 ). I I /2 I /2 I /4 = 2 p + O(). 7 (9)

ũ ũ ũ ũ ũ /2 ũ /2 ũ /2 ũ /4 log ũ /2 2 ũ /2 ũ /4 0.2 0.302842712474619 0.009289321881345 26.142135623725615 4.708305098603142 0.1 0.293553390593274 0.000355339059327 1.999999999999688 0.999999999999775 0.05 0.293198051533946 0.000177669529664 2.000000000001875 1.000000000001352 0.025 0.293020382004283 0.000088834764832 2.635450714080436 1.398049712285012 0.0125 0.292931547239451 0.000033707617584 12.589489353884787 3.654147861537719 0.00625 0.292897839621867 0.000002677441208 0.003125 0.292895162180659 Table 3. Table of values for te trapezoidal rule for f(x) = x α wit α = 1/ 2. Te last column is te final approximation of te order of accuracy p, wic fails for tis case. 5 Non-smoot error So far we ave assumed tat te error depends smootly on te parameter. Ten te error is of te form in (2). Tis is, owever not always te case. Te error can, for instance, depend discontinuously on, eventoug it is bounded as in (1). Te reason for tis can be discontinuities in te metod itself (e.g. case switces) or non-smoot functions in te problem (e.g. solutions, sources, integrands). Wen te error is non-smoot one cannot ceck convergence rates by looking at ratios of differences as in Section 2.2. Oter metods must be used. Example 7 Consider te trapezoidal rule applied to te integral 1 0 x α dx, for some value 0 < α < 1. Te trapezoidal rule is exact everywere except at te grid cell wic contains α. Te error tere depends crucially on te distance between α and te nearest grid point. More precisely, if x j α < x j+1 and x j+1 x j =, u ũ = = xj+1 x α dx x j α + x j+1 α x j 2 α x j (α x)dx + xj+1 α (x α)dx x j+1 x j 2 = β(β 1) 2 2, were β = β() = (α x j )/, i.e. te fractional part of α/, wic is a discontinuous function of. Te metod is still second order accurate since β() 1 and (1) terefore olds wit C = 1/8. However, te results presented in Figure 6 and Table 3 clearly sows te non-smootness of te error and te failure of te ratios of te differences to predict te order of convergence. 8 (9)

10 3 u u 2 10 4 10 5 10 6 10 7 10 2 10 1 Figure 6. Error in trapezoidal rule for f(x) = x α wit α = 1/ 2. Te dased line is 2 wic indicates te slope for a second order metod. 9 (9)