Implementation of the Bulirsch Stöer extrapolation method Sujit Kirpekar Department of Mechanical Engineering kirpekar@newton.berkeley.edu December, 003 Introduction This paper outlines the pratical implementation of extrapolation methods. It is often claimed that extrapolation methods are the best known ways to obtain high-accuracy solutions to ordinary differential equations (odes) with minimal computational efforts. In particular, we implement and then review the performance of the Bulirsch Stöer algorithm. We implement the method is fine detail, paying particular attention to the control of the step size and the order (obtained from higher and higher degrees of interpolation). We test the performance of the method on a simple ode and draw useful conclusions about the step- and order- control strategy of the algorithm. Asymptotic error expansions We have the following theorem due to Gragg (965): Theorem. Let f F N+ (a, b), and y(t) be the exact solution of the initial value problem y = f(t, y), y(t 0 ) = y 0, t 0 [a, b]. ()
ASYMPTOTIC ERROR EXPANSIONS For t R h = (t 0 + ih i = 0,,,...) let u n (t) be defined by the explicitly started (multi-step) midpoint rule as, u(t 0 ) = u 0 u(t 0 + h) = u 0 + hf(t 0, u 0 ) () u n+ = u n + hf(t, u n ) then u(t) has an asymptotic expansion of the form, u n = y(t n ) + N h ( k e k (t) + ( ) (tn t 0)/hẽ k (t) ) + h N+ E N+ (h, t n ) (3) k= valid for x [a, b] and all h = (t n t 0 )/n, n =,,... The functions e k (t) and ẽ k (t) are independant of h. The remainder term for fixed t n remains bounded for all h = ( t n t 0 ) / n, n =,,... From the above theorem, we notice that the midpoint rule has asymptotic expansion in h and not h. Thus, by systematically extrapolating errors away, one can improve the order of accuracy of the solution by h in each iteration. From above, the error in first approximation is π(t) = u n y(t n ) = h [ e (t n ) + ( ) (tn t 0)/hẽ (t) ] (4) In view of the term ( ) (tn t 0)/hẽ (t) the error shows oscillatory behaviour, π(t + h) = h [ e (t n ) + ( ) (t n+ t 0 )/hẽ (t) ] = h [ e (t n ) ( ) (tn t 0)/hẽ (t) ] (5) The oscillations may be removed due to a trick due to Gragg (965), as follows. Define, v(t n ) = v n = [u n + u n + hf(t, u n )] (6) And by Eqn. we know, u n+ = u n + hf(t, u n ) hence also, v(t n ) = v n = [u n + u n+ hf(t, u n )] (7) and by adding Eqn. 6 and qn. 7 gives us,
3 THE MODIFIED MIDPOINT RULE 3 v n = [ u n + ] u n u n+ (8) and by use of Eqn. 3 one can obtain the expansion for v(t n ) v(t n ) = ( y(t n ) + [y(t n + h) + y(t n h)] (9) N h k( e k (t) + [e k(t n + h) + e k (t n h)] k= + ( ) (tn t 0)/h (ẽ k (t) [ẽ k(t n + h) + ẽ k (t n h)]) )) + O(h N+ ) Expanding y(t n ± h) and e k, ẽ k in a Taylor series in h, one finally obtains for v(t n ) an expansion of the form, v(t n ) =y(t n ) + h [e (t n ) + ] 4 y (t n ) (0) + N h k( e k (t) + ( ) (tn t 0)/hẽ k (t) ) + O(h N+ ) k= in which the leading error term no longer contains an oscillatory factor. 3 The modified midpoint rule The procedure decribed above naturally leads us to use the modified midpoint rule as described by: Eqn. and Eqn. 6 combined. We use this procedure to advance the solution from u N at t to u N+ at t + H by taking a set of n steps each of size h. The method can be summarized as, h j = H/n j () z 0 =u N () z =u N + hf(t N, u N ) z m+ =z m + hf(t + mh, z m ) for m =,,..., n u N+ = [z n + z n + hf(t + H, z n )]
4 THE BULIRSCH STÖER METHOD 4 Here the z s are intermediate approximations which march along in steps of h, while u N+ is the final approximation to y(t + H) 4 The Bulirsch Stöer Method The basic idea of the Bulirsch Stoer method is the same as any other extrapolation method. We compute the solution u N+ at t + H using the modified midpoint method for various values of n. Once we know a few desired values, we fit the solution to an analytic form and evaluate it at h = 0. This idea is nothing but Richardson s deferred approach to the limit. The sequence of separate attempts of the modified midpoint rule is made with increasing values of n. Bulirsch and Stöer originally proposed, n =, 4, 6, 8,, 6, 4, 3, 48, 64, 96... n j = n j and Deufelhard suggests that n =, 4, 6, 8, 0,, 4... n j = j is more efficient. For extrapolation we use the Aitkens-Neville algorithm of rational function interpolation which is consicely described by the following tableau, T T T T 3 T 3 T 33......... T kk The first column of the tableau is formed by use of the modified midpoint rule for increasing substeps. T j, (h) = u n j N+ (t) (3) and successive columns can be filled by using the recurrence, T j,k (h) = (h h j k+)t j,k (h) (h h j )T j,k (h) h j h j k+ (4)
5 STEP-SIZE AND ORDER CONTROL 5 and we are really interested in the extrapolated value at h = 0, hence, T j,k (0) = h j k+t j,k (h) + h j )T j,k (h) h j h j k+ = T j,k + T j,k T j,k ( nj n j k+ ) (5) After each successive n is tried, polynomial interpolation is attempted. Adjacent diagonal entries of the tableau are used for error estimates and if the error is more than the tolerance, the next n in the sequence is attempted. 5 Step-size and order control To implement error control, we define an error estimate of the form, and enforce the requirement that ɛ k+,k = T k+,k T k+,k+ (6) ɛ k+,k < ε (7) where ε is the user specified tolerance. Error estimates from the Aitken- Neville s algorithm can be shown to be of O(H k ). This has been subsequently verified in a numerical experiment in this paper. Thus, for an even sequence of n the order of the method is k and a simple estimate of a new stepsize would be, ( ) ε k H k = H (8) ɛ k+,k subject to conditions of maximum and minimum change from the previous value. The question now remains which column k we aim to achieve convergence in. Suppose A k is the work to obtain row k of the tableau, A k+ is the work to obtain column k. Defining the work in terms of the function evaluations f(t, u), we use the following recurrence relations, A = n + see We note that error estimates from the diagonal entries only, as described by, ɛ k+,k = T k+,k+ T k,k has been demonstrated to be too conservative. By relaxing this criteria computations usually speed up by 0% in non-critical applications The number of right-hand side evaluations required by the modified midpoint method is n +.
5 STEP-SIZE AND ORDER CONTROL 6 A k+ = A k + n k+ The work per unit step is equal to A k+ /H k which we non-dimensionalize by H and write as, W k = A ( ) k+ ε k H = A k+ (9) H k ɛ k+,k A vector of the quantities W k are recorded during the integration and the optimal column index q is then defined by, W q = min{w, W,...W kf } where k f is the final column in which the error criteria was satisfied. The q thus determined defines the step size to be used as the next large step so that we can expect to get convergence in the optimal column q. After some numerical experimentation, we propose the two important refinements:. If the current H is too small, then k f will be too small and the order q will remain too small. It may be desirable to increase H and select a higher order.. If the current H is too big the required tolerance may never be achieved and we may need to decrease H To deal with these issues, we compute the following table along with the integration α(k, q) = ε A k+ Aq+ (k )(A q+ A +) for k < q The idea is that if we attain convergence in the qth column of the extrapolation table, we aim to increase the step size to H q+. Since we dont have a value for H q+ we estimate it as, H q+ = H q α(q, q + ) However this increase in step size is efficient only if the work per unit step reduces, which reqiures, A q+ α(q, q + ) > A q+
6 NUMERICAL EXPERIMENTS 7 The problem of stepsize reduction can be handled by computing stepsize estimates H k H k α(k, q). If the any H k is too small, we abandon the current step and restart using H k. The criterion for smallness is taken as, This check is only restricted to, H k α(k, q + ) < H max(q, q ) k min(k max, q + ) 6 Numerical experiments. To test our code, we use a simple non-linear ode, which has the exact solution, y (t, y) = 00ty (0) y(0) = ; y( 3) = 90 y(t) = or with + 00t We first verify the order of accuracy of the modified midpoint rule, as we derived in Eqn. 0. The results are shown below in Fig.. It is seen that the error does scale as O(h ), and thus our basic implementation is verified.. Next we test, the convergence of our interpolation scheme. Our claim from Section 5 is: Use of kth column interpolation results in an error estimate of O(h k ). This is successfully verified by our code and shown in Fig. 3. Next we test our algorithm for its choice of step-size. The basic idea is that, for a fixed order of interpolation, the algorithm should choose a step size such that the error remains within the tolerance. We note that the error in the first step depends on the size of the first step and the column of the interpolation we choose. Also, there may be
6 NUMERICAL EXPERIMENTS 8 cases, when we are never able to achieve convergence to the specified tolerance because of too low-order choice of interpolation. Taking into account these effects, we successfully show that our algorithm functions well to keep the error below the required tolerance. This is shown in Fig. 3 where the algorithm is forced to choose a stepsize for a fixed order of interpolation. The user specified tolerance was set to 0 7. 4. Our next experiment is with different seqeunces of steps to be taken during the modified midpoint rule. We experiment with the following three: n = [ 4 8 6 3 64 8 5 04] Romberg Sequence () n = [ 4 6 8 6 4 3 48 64 96] n = [ 4 6 8 0 4 6 8 0 4] Bulirsch Sequence Deufelhard Sequence We test the above three sequences with a tolerance of 0 3, and report that the Deufelhard seqeunce performed the best (3693 function evalutions) compared to the Romberg (379) and Bulirsch (5487) sequences. 5. Finally, we run our code with full freedom, allowing the algorithm to choose order and step-size and report the cost of the method as a function of tolerance. A typical solution, with the programs adopted strategy is plotted in Fig. 4. We notice that the size of the interpolation table is usually not allowed to increase beyond 5. The algorithm does not increase the value of q for reasons of cost (work evaluted in W ). Also, the step-size increases progressively as the solution becomes more and more flat (magnitude of function derivatives decrease). This is a good sign, but the oscillations we see in the step-size are not. They are probably because we tend to be slightly ambitious in estimating stepsize and need to reduce immediately it to keep the errors incontrol. 6. Our final figure (Fig. 5) is a display of the cost of our method compared to a standard Runge Kutta Fehlberg method. For comparison, we use the Merson s 4-5 method, in which we use the simple strategy of doubling the step-size if the error is bounded by ε/0 for two consequtive steps and rescaling the current step by 0.5 for a failed step. The figure clearly shows that extrpolation methods are more efficient at higher tolerances, but more expensive for lower tolerances. The Merson s method
7 CONCLUSION 9 is actually a four-stage and five-stage RK, with both methods of order four. (Hence 4-5 is actually misleading). The error estimate is correct for linear odes, but is taken as an approximation for non-linear odes like the one we are working with. The Butcher array is, 0 0 3 3 3 6 6 3 0 8 8 0 3 b 0 3 ˆb 0 0 6 3 e 5 0 3 0 4 5 6 30 7 Conclusion From our results, we conclude that extrapolation methods are indeed efficient. For a higher orders of accuracy, the cost (in terms of the number of function evaluations) of our method is smaller than traditional Runge-Kutta methods. We have also demonstrated that our algorithm is successful in adjusting both step-size and order (i.e. the column of the interpolation polynomial). The algorithm successfully displays the quality of taking smaller steps when the function derivative is large and taking larger steps when the function derivative is smaller. However, the technique presented above may not be suitable to odes whose right hand sides are evaluated by interpolation or from a table lookup. Also, the technique would be unsuitable for odes that contain a singular point inside the region of integration. All codes used in this paper are available at http://www.me.berkeley.edu/~kirpekar/
REFERENCES 0 Figure : Convergence of the modified midpoint rule References Figure : Convergence of Aitken-Neville s algorithm [] Deufelhard,P. Order and step-size control in Extrapolation methods 983, Numerische Mathematik, vol. 4, pp. 399-4 [] Deufelhard,P. Recent Progress in Extrapolation methods for ordinary differential equtions 985, Siam Review, vol. 7, pp. 505-535 [3] Gragg, W. B. Repeated extrapolation to the limit in the numerical solution of ordinary differential equations, SIAM J. Numer. Anal. (965) pp. 384-403 [4] Press, et. al. Numerical Recipes in Fortran 77: the art of scientific computing, Cambridge University Press. [5] Stoer, J. and Bulirsch, R 980, Introduction to Numerical Analysis, Springer-Verlag. 8 Figures Figure 3: Step-size control ascpect of our algorithm
8 FIGURES Figure 4: Typical Solution using our algorithm Figure 5: Cost of our method compared to Merson s 4-5 RKF method