

 Clyde Kennedy
 1 years ago
 Views:
Transcription
1 Chapter 3 OneStep Methods 3. Introduction The explicit and implicit Euler methods for solving the scalar IVP y = f(t y) y() = y : (3..) have an O(h) global error which istoolow to make themofmuch practical value. With such a low order of accuracy, they will be susceptible to round o error accumulation. Additionally, the region of absolute stability of the explicit Euler method is too small. Thus, we seek higherorder methods which should provide greater accuracy than either the explicit or implicit Euler methods for the same step size. Unfortunately, there is generally a trade o between accuracy and stability and we will typically obtain one at the expense of the other. Since we were successful in using Taylor's series to derive a method, let us proceed along the same lines, this time retaining terms of O(h k ). y(t n )=y(t n; )+hy (t n; )+ h y (t n; )+:::+ hk k! y(k) (t n; )+O(h k+ ): (3..) Clearly, methods of this type will be explicit. Using (3..) y (t n; )=f(t n; y(t n; )): (3..3a) Dierentiating (3..) y (t n; )=[f t + f y y ] (t n; y(tn;)) =[f t + f y f] (t n; y(tn;)): (3..3b)
2 Continuing in the same manner y (t n; )=[f tt +f ty f + f t f y + f yy f + f y f] (t n; y(tn;)) (3..3c) etc. Specic methods are obtained by truncating the Taylor's series at dierent values of k. For example, if k =we get the method y n = y n; + hf(t n; y n; )+ h [f t + f y f] (t n; yn;): (3..4a) From the Taylor's series expansion (3..), the local error of this method is d n = h3 6 y ( n ): (3..4b) Thus, we succeeded in raising the order of the method. Unfortunately, methods of this type are of little practical value because the partial derivatives are dicult to evaluate for realistic problems. Any software would also have to be problem dependent. By way of suggesting an alternative, consider the special case of (3..) when f is only a function of t, i.e., y = f(t) y() = y : This problem, which is of little interest, can be solved by quadrature to yield y(t) =y + Z t f()d: We can easily construct highorder approximate methods for this problem by using numerical integration. Thus, for example, the simple leftrectangular rule would lead to Euler's method. The midpoint rule with a step size of h would give us y(h) =y + hf(h=) + O(h 3 ): Thus, by shifting the evaluation point to the center of the interval we obtained a higherorder approximation. Neglecting the local error term and generalizing the method to the interval t n; <t t n yields y n = y n; + hf(t n; + h=):
3 Runge [] sought to extend this idea to true dierential equations having the form of (3..). Thus, we might consider y n = y n; + hf(t n ; h= y n;= ) as an extension of the simple midpoint rule to (3..). The question of how to dene the numerical solution y n;= at the center of the interval remains unanswered. A simple possibility that immediately comes to mind is to evaluate it by Euler's method. This gives y n;= = y n; + h f(t n; y n; ) however, we must verify that this approximation provides an improved order of accuracy. After all, Euler's method has an O(h ) local error and not an O(h 3 ) error. Let's try to verify that the combined scheme does indeed have an O(h 3 ) local error by considering the slightly more general scheme y n = y n; + h(b k + b k ) (3..5a) where k = f(t n; y n; ) (3..5b) k = f(t n; + ch y n; + hak ): (3..5c) Schemes of this form are an example of RungeKutta methods. We see that the proposed midpoint scheme is recovered by selecting b =, b =, c = =, and a = =. We also see that the method does not require any partial derivatives of f(t y). Instead, (the potential) highorder accuracy is obtained by evaluating f(t y) atanadditional time. The coecients a, b, b, and c will be determined so that a Taylor's series expansion of (3..5) using the exact ODE solution matches the Taylor's series expansion (3.., 3..3) of the exact ODE solution to as high a power in h as possible. To this end, recall the formula for the Taylor's series of a function of two variables F (t + y + ) =F (t y)+[f t + F y ] ( t y) + [ F tt +F ty + F yy ] ( t y) + ::: (3..6) 3
4 The expansion of (3..5) requires substitution of the exact solution y(t) into the formula and the use of (3..6) to construct an expansion about (t n; y(t n; )). The only term that requires any eort is k, which, upon insertion of the exact ODE solution, has the form k = f(t n; + ch y(t n; )+haf(t n; y(t n; ))): To construct an expansion, we use (3..6) with F (t y) =f(t y), t = t n;, y = y(t n; ), = ch, and = haf(t n; y(t n; )). This yields k = f + chf t + haff y + [(ch) f tt +ach ff ty +(ha) f f yy ]+O(h 3 ): All arguments of f and its derivatives are (t n; y(t n; )). We have suppressed these to simplify writing the expression. Substituting the above expansion into (3..5a) while using (3..5b) with the exact ODE solution replacing y n; yields y(t n )=y(t n; )+h[b f + b (f + chf t + haff y + O(h ))]: (3..7) Similarly, substituting (3..3) into (3..), the Taylor's series expansion of the exact solution is y(t n )=y(t n; )+hf + h (f t + ff y )+O(h 3 ): (3..8) All that remains is a comparison of terms of the two expansions (3..7) and (3..8). The constant terms agree. The O(h) terms will agree provided that b + b =: (3..9a) The O(h ) terms of the two expansions will match if cb = ab ==: (3..9b) A simple analysis would reveal that higher order terms in (3..7) and (3..8) cannot be matched. Thus, we have three equations (3..9) to determine the four parameters a, b, b, and c. Hence, there is a one parameter family of methods and we'll examine two specic choices. 4
5 . Select b =, then a = c = = and b =. Using (3..5), this RungeKutta formula is y n = y n; + hk (3..a) with k = f(t n; y n; ) k = f(t n; + h= y n; + hk =): (3..b) Eliminating k and k,we can write (3..) as y n = y n; + hf(t n; + h= y n; + hf(t n; y n; )=) (3..a) or ^y n;= = y n; + h f(t n; y n; ) (3..b) y n = y n; + hf(t n; + h= ^y n;= ): (3..c) This is the midpoint rule integration formula that we discussed earlier. The ^ on y n;= indicates that it is an intermediate rather than a nal solution. As shown in Figure 3.., we can regard the twostage process (3..b,c) as the result of two explicit Euler steps. The intermediate solution ^y n;= is computed at t n + h= in the rst (predictor) step and this value is used to generate an approximate slope f(t n + h= ^y n;= ) for use in the second (corrector) Euler step. According to Gear [5], this method has been called the EulerCauchy, improved polygon, Heun, or modied Euler method. Since there seems to be some disagreement about its name and because of its similarity to midpoint rule integration, we'll call it the midpoint rule predictorcorrector.. Select b = =, then a = c = and b = =. According to (3..5), this Runge Kutta formula is y n = y n; + h (k + k ) (3..a) 5
6 with k = f(t n; y n; ) k = f(t n; + h y n; + hk ): (3..b) Again, eliminating k and k, y n = y n; + h [f(t n; y n; )+f(t n y n; + hf(t n; y n; ))]: (3..3a) This too, can be written as a twostage formula ^y n = y n; + hf(t n; y n; ) (3..3b) y n = y n; + h [f(t n; y n; )+f(t n ^y n )]: (3..3c) The formula (3..3a) is reminiscent of trapezoidal rule integration. The combined formula (3..3b,c) can, once again, be interpreted as a predictorcorrector method. Thus, as shown in Figure 3.., the explicit Euler method is used to predict a solution at t n and the trapezoidal rule is used to correct it there. We'll call (3.., 3..3) the trapezoidal rule predictorcorrector however, it is also known as the improved tangent, improved polygon, modied Euler, or EulerCauchy method ([5], Chapter ). Using Denition..3, we see that the Taylor's series method (3..4) and the Runge Kutta methods (3..) and (3..3) are consistent to order two since their local errors are all O(h 3 ) (hence, their local discretization errors are O(h )). Problems. Solve the IVP y = f(t y) =y ; t + y() = = on < t using the explicit Euler method and the midpoint rule. Use several step sizes and compare the error at t = as a function of the number of evaluations of f(t y). The midpoint rule has twice the number of function evaluations of the Euler method but is higher order. Which method is preferred? 6
7 y n y n slope f(t + h/,y ^ ) n n/ ^y n/ h/ h/ t t t n n Figure 3..: Midpoint rule predictorcorrector (3..b,c) for one time step. average slope y n slope f(t,y ^ ) n n y n y^ n h t t t n n Figure 3..: Trapezoidal rule predictorcorrector (3..3b,c) for one time step. 3. Explicit RungeKutta Methods We would like to generalize the second order RungeKutta formulas considered in Section 3. to higher order. As usual, we will apply them to the scalar IVP (3..). RungeKutta methods belong to a class called onestep methods that only require information about the solution at time t n; to calculate it at t n. This being the case, it's possible to write 7
8 them in the general form y n = y n; + h(t n; y n; h): (3..) This representation is too abstract and we'll typically consider an sstage RungeKutta formula for the numerical solution of the IVP (3..) in the form where y n = y n; + h k i = f(t n; + c i h y n; + h j= i= b i k i (3..a) a ij k j ) i = ::: s: (3..b) These formulas are conveniently expressed as a tableau or a \Butcher diagram" or more compactly as where We can also write (3..) in the form Y i = y n; + h c a a a s c a a a s c s a s a s a ss b b b s y n = y n; + h j= c i= A b b i f(t n; + c i h Y i ) (3..3a) a ij f(t n; + c j h Y j ) i = ::: s: (3..3b) In this form, Y i, i = ::: s, are approximations of the solution at t = t n + c i h that typically do not have as high an order of accuracy as the nal solution y n. An explicit RungeKutta formula results when a ij = for j i. Historically, all RungeKutta formulas were explicit however, implicit formula are very useful for sti systems and problems where solutions oscillate rapidly. We'll study explicit methods in this section and take up implicit methods in the next. RungeKutta formulas are derived in the same manner as the secondorder methods of Section 3.. Thus, we 8
9 . expand the exact solution of the ODE in a Taylor's series about, e.g., t n;. substitute the exact solution of the ODE into the RungeKutta formula and expanded the result in ataylor's series about, e.g., t n; and 3. match the two Taylor's series expansions to as high an order as possible. The coecients are usually not uniquely determined by this process thus, there are families of methods having a given order. A RungeKutta method that is consistent to order k (or simply of order k) will match the terms of order h k in both series. Clearly the algebra involved in obtaining these formulas increases combinatorically with increasing order. A symbolic manipulation system, such as MAPLE or MATHEMATICA, can be used to reduce complexity. Fortunately, the derivation is adequately demonstrated by the secondorder methods presented in Section 3. and, for the most part, we will not need to present detailed derivations of higherorder methods. There are three oneparameter families of threestage, thirdorder explicit Runge Kutta methods [3, 6]. However, the most popular explicit methods are of order four. Their tableau has the general form c a c 3 a 3 a 3 c 4 a 4 a 4 a 43 b b b 3 b 4 The Taylor's series produce eleven equations for the thirteen nonzero parameters listed above. The classical RungeKutta method has the following form: y n = y n; + h 6 (k +k +k 3 + k 4 ) (3..4a) where k = f(t n; y n; ) (3..4b) k = f(t n; + h= y n; + hk =) (3..4c) 9
10 Order, k Min. Fn. Evals Table 3..: Minimum number of function evaluations for explicit RungeKutta methods of various orders. k 3 = f(t n; + h= y n; + hk =) (3..4d) k 4 = f(t n; + h y n; + hk 3 ): (3..4e) Some observations about this method follow:. The local error of (3..4) is O(h 5 ). In order to get an a priori estimate of the local error we have to subtract the two Taylor's series representations of the solution. This is very tedious and typically does not yield a useful result. RungeKutta methods do not yield simple a priori error estimates.. Four function evaluations are required per time step. 3. In the (unlikely) case when f is a function of t only, (3..4) reduces to y n = y n; + h 6 [f(t n;)+4f(t n;= )+f(t n )] which is the same as Simpson's rule integration. Our limited experience with RungeKutta methods would suggest that the number of function evaluations increases linearly with the order of the method. Unfortunately, Butcher [8] showed that this is not the case. Some key results are summarized in Table 3... The popularity of the fourstage, fourthorder RungeKutta methods are now clear. From Table 3.., we see that a fth order RungeKutta method requires an additional two function evaluations per step. Additionally, Butcher [8] showed that an explicit sstage RungeKutta method will have an order of at least s ;. Although RungeKutta formulas are tedious to derive, we can make a few general observations. An order one formula must be exact when the solution of the ODE is a linear polynomial. Were this not true, it wouldn't annihilate the constant and linear
11 terms in a Taylor's series expansion of the exact ODE solution and, hence, could not have the requisite O(h ) local error to be rstorder accurate. Thus, the RungeKutta method should produce exact solutions of the dierential equations y = and y =. The constantsolution condition is satised identically by construction of the Runge Kutta formulas. Using (3..3a), the latter (linearsolution) condition with y(t) = t and f(t y) = implies t n = t n; + h or i= b i =: i= b i (3..5a) If we also require the intermediate solutions Y i to be rst order, then the use of (3..3b) with Y i = t n; + c i h gives c i = j= a ij i = ::: s: (3..5b) This condition does not have to be satised for loworder RungeKutta methods [6] however, its satisfaction simplies the task of obtaining order conditions for higherorder methods. Methods that satisfy (3..5b) also treat autonomous and nonautonomous systems in a symmetric manner (Problem ). We can continue this process to higher orders. Thus, the RungeKutta method will be of order p if it is exact when the dierential equation and solution are y =(t ; t n; ) l; y(t) = l (t ; t n;) l l = ::: p: (The use of t ; t n; as a variable simplies the algebraic manipulations.) Substituting these solutions into (3..3a) implies that or i= h l l = h j= b i (c i h) l; b i c l; i = l = ::: p: (3..5c) l
12 Conditions (3..5c) are necessary for a method to be order p, butmay not be sucient. Note that there is no dependence on the coecients a ij i j = ::: s, in formulas (3..5a,c). This is because our strategy of examining simple dierential equations is not matching all possible terms in a Taylor's series expansion of the solution. This, as noted, is a tedious operation. Butcher developed a method of simplifying the work by constructing rooted trees that present the order conditions in a graphical way. They are discussed in many texts (e.g., [, 6]) however, they are still complex and we will not pursue them here. Instead, we'll develop additional necessary order conditions by considering the simple ODE y = y: Replacing f(t y) in(3..3) by y yields y n = y n; + h Y i = y n; + h It's simpler to use vector notation j= a ij Y j i= b i Y i i = ::: s: y n = y n; + hb T Y Y = y n; l + hay where Y =[Y Y ::: Y s ] T (3..6a) A = l =[ ::: ] T 6 4 a a ::: a s a a ::: a s a s a s ::: a ss 3 7 (3..6b) 5 (3..6c) and b =[b b ::: b s ] T : (3..6d)
13 Eliminating Y, we have Y = y n; (I ; ha) ; l and y n = y n; + hy n; b T (I ; ha) ; l: Assuming that y n; is exact, the exact solution of this test equation is y n = e h y n;. Expanding this solution and (I ; ha) ; in series +h + :::+ hk k! + :::=+hbt (I + ha + :::+ h k A k + :::)l: Equating like powers of h yields the order condition b T A k; l = k = ::: p: (3..7) k! We recognize that this condition with k = is identical to (3..5a). Letting c = [c c ::: c s ] T, we may write (3..5c) with l = in the form b T c = =. The vector form of (3..5b) is Al = c. Thus, b T Al ==, which is the same as (3..7) with k =. Beyond k =,the order conditions (3..5c) and (3..7) are independent. Although conditions (3..5) and (3..7) are only necessary for a method to be of order p, they are sucient in many cases. The actual number of conditions for a RungeKutta method of order p are presented in Table 3.. [6]. These results assume that (3..5b) has been satised. Order, p No. of Conds Table 3..: The number of conditions for a RungeKutta method of order p [6]. Theorem 3... The necessary and sucient conditions for a RungeKutta method (3..3) to be of second order are (3..5c), l =, and (3..7), k =. If (3..5b) is satised then (3..5), k =, are necessary and sucient for secondorder accuracy. Proof. We require numerous Taylor's series expansions. To begin, we expand f(t n; + c i h Y i ) using (3..6) to obtain f(t n; + c i h Y i )=f + f t c i h + f y (Y i ; y(t n; )) + [f tt(c i h) +f ty (c i h)(y i ; y(t n; ))+ 3
14 f yy (Y i ; y(t n; )) ]+O(h 3 ): All arguments of f and its derivatives are at (t n; y(t n; )). They have been suppressed for simplicity. Substituting the exact ODE solution and the above expression into (3..3a) yields y(t n )=y(t n; )+h i= b i [f + f t c i h + f y (Y i ; y(t n; )) + O(h )]: The expansion of Y i ; y(t n; ) will, fortunately, only require the leading term thus, using (3..3b) Y i ; y(t n; )=h Hence, we have y(t n )=y(t n; )+h j= a ij f + O(h ): b i [f + f t c i h + hff y i= j= a ij + O(h )]: Equating terms of this series with the Taylor's series (3..8) of the exact solution yields (3..5c) with l =, (3..5c) with l =,and (3..7) with k =. We have demonstrated the equivalence of these conditions when (3..5b) is satised. Remark. The results of Theorem 3.. and conditions (3..5) and (3..7) apply to both explicit and implicit methods. Let us conclude this section with a brief discussion of the absolute stability of explicit methods. We will present a more detailed analysis in Section 3.4 however, the present material will serve to motivate the need for implicit methods. Thus, consider an sstage explicit RungeKutta method applied to the test equation y = y: (3..8) Using (3..8) in (3..3) with the simplication that a ij =, j i, for explicit methods yields y n = y n; + z i= b i Y i = y n; + zb T Y 4 (3..9a)
15 where Y i = y n; + z i; X j= a ij Y j i = ::: s (3..9b) and z = h: (3..9c) The vector form of (3..9) is Y = y n; l + zay: (3..9d) Using this to eliminate Y in (3..9a), we have y n = y n; [ + zb T (I ; za) ; l]: Expanding the inverse y n = y n; [ + zb T (I + za + :::+ z k A k + :::)l] Using (3..7) y n = R(z)y n; (3..a) where R(z) =+z + z + :::+ zp p! + X j=p+ z j b T A j; l: The matrix A is strictly lower triangular for an sstage explicit RungeKutta method thus, A j; =, j >s. Therefore, R(z) =+z + z + :::+ zp p! + j=p+ z j b T A j; l: In particular, for explicit sstage methods with p = s 4, we have (3..b) R(z) =+z + z zp + :::+ s = p 4: (3..c) p! The exact solution of the test equation (3..8) is y(t n )=e h y(t n; ) 5
16 thus, as expected, a pthorder RungaKutta formula approximates a Taylor's series expansion of the exact solution through terms of order p. Using Denition..7 and (3..), the region of absolute stability of an explicit RungeKutta method is jr(z)j = z +z + + :::+ zp p! + j=p+ z j b T A j; l : (3..a) In particular, jr(z)j = z +z + zp + :::+ p! s = p 4: (3..b) Since no RungeKutta coecients appear in (3..b), we have the interesting result. Lemma 3... All pstage explicit RungeKutta methods of order p 4 have the same region of absolute stability. Since je i j =,, we can determine the boundary of the absolute stability regions (3..a,b) by solving the nonlinear equation R(z) =e i : (3..) Clearly, (3..) implies that jy n =y n; j =. For p = (i.e., for Euler's method), the boundary of the absolutestability region is determined as +z = e i which can easily be recognized as the familiar unit circle centered at z = ; + i. For real values of z the intervals of absolute stability formethods with p = s 4 are shown in Table Absolute stability regions for complex values of z are illustrated for the same methods in Figure 3... Methods are stable within the closed regions shown. The regions of absolute stability grow with increasing p. When p = 3 4, they also extend slightly into the right half of the complex zplane. Problems. Instead of solving the IVP (3..), many software systems treat an autonomous ODE y = f(y). Nonautonomous ODEs can be written as autonomous systems 6
17 Order, p Interval of Absolute Stability (,) (,) 3 (.5,) 4 (.78,) Table 3..3: Interval of absolute stability for pstage explicit RungeKutta methods of order p = Im(z) Re(z) Figure 3..: Region of absolute stability for pstage explicit RungeKutta methods of order p = 3 4 (interiors of smaller closed curves to larger ones). 7
18 by letting t be a dependent variable satisfying the ODE t =. A RungeKutta method for an autonomous ODE can be obtained from, e.g., (3..3) by dropping the time terms, i.e., y n = y n; + h with Y i = y n; + h j= i= a ij f(y j ) b i f(y i ) i = ::: s: The RungeKutta evaluation points c i, i = ::: s, do not appear in this form. Show that RungeKutta formulas (3..3) and the one above will handle autonomous and nonautonomous systems in the same manner when (3..5b) is satised. 3.3 Implicit RungeKutta Methods We'll begin this section with a negative result that will motivate the need for implicit methods. Lemma No explicit RungeKutta method can have an unbounded region of absolute stability. Proof. Using (3..), the region of absolute stability of an explicit RungeKutta method satises jy n =y n; j = jr(z)j z = h where R(z) is a polynomial of degree s, thenumber of stages of the method. Since R(z) is a polynomial, jr(z)j!as jzj!and, thus, the stability region is bounded. Hence, once again, we turn to implicit methods as a means of enlarging the region of absolute stability. Necessary order conditions for sstage implicit RungeKutta methods are given by (3..5c, 3..7) (with sucient conditions given in Hairer et al. [6], Section II.). A condition on the maximum possible order follows. Theorem The maximum order of an implicit sstage RungeKutta method iss. 8
19 Proof. cf. Butcher [7]. The derivations of implicit RungeKutta methods follow those for explicit methods. We'll derive the simplest method and then give afew more examples. Example Consider the implicit stage method obtained from (3..3) with s = as y n = y n; + hb f(t n; + c h Y ) (3.3.a) Y = y n; + ha f(t n; + c h Y ): (3.3.b) To determine the coecients c, b, and a, we substitute the exact ODE solution into (3.3.a,b) and expand (3.3.a) in a Taylor's series y(t n )=y(t n; )+hb [f + c hf t + f y (Y ; y(t n; )) + O(h )] where f := f(t n; y(t n; )), etc. Expanding (3.3.b) in a Taylor's series and substituting the result into the above expression yields y(t n )=y(t n; )+hb [f + c hf t + ha ff y + O(h )]: Comparing the terms of the above series with the Taylor's series of the exact solution yields y(t n )=y(t n; )+hf + h (f t + ff y )+O(h 3 ) b = a = c = : Substituting these coecients into (3.3.), we nd the method to be an implicit midpoint rule y n = y n; + hf(t n; + h= Y ) (3.3.a) Y = y n; + h f(t n; + h= Y ): (3.3.b) The tableau for this method is 9
20 The formula has similarities to the midpoint rule predictorcorrector (3..) however, there are important dierences. Here, the backward Euler method (rather than the forward Euler method) may be regarded as furnishing a predictor (3.3.b) with the midpoint rule providing the corrector (3.3.a). However, formulas (3.3.a) and (3.3.b) are coupled and must be solved simultaneously rather than sequentially. Example The twostage method having maximal order four presented in the following tableau was developed by Hammer and Hollingsworth [8]. ; p p p ; p This method is derived in Gear [5], Section.5. Example Let us examine the region of absolute stability of the implicit midpoint rule (3.3.). Thus, applying (3.3.) to the test equation (3..8) we nd Y = y n; + h Y and y n = y n; + hy : Solving for Y Y = y n; ; h= and eliminating it in order to explicitly determine y n as y n = + y n; = h ; h= +h= ; h= Thus, the region of absolute stability is interior to the curve +z= ; z= = ei z = h: y n; : Solving for z z = ; ei ;ei= +e = ; e ;i= i e i= + e ;i= = ;i tan =:
21 Im(h λ) Re(h λ) Figure 3.3.: Region of absolute stability for the implicit midpoint rule (3.3.). Since z is imaginary, the implicit midpoint rule is absolutely stable in the entire negative half of the complex z plane (Figure 3.3.). Let us generalize the absolute stability analysis presented in Example before considering additional methods. This analysis will be helpful since we will be interested in developing methods with very large regions of absolute stability. Thus, we apply the general method (3..3) to the test equation (3..8) to obtain y n = y n; + zb T Y (3.3.3a) where Y, l, A, and b and are dened by (3..6) and z = h. Eliminating Y in (3.3.3a) by using (3.3.3b) we nd (I ; za)y = y n; l (3.3.3b) y n = R(z)y n; (3.3.4a) where R(z) =+zb T (I ; za) ; l: (3.3.4b) The region of absolute stability is the set of all complex z where jr(z)j. While R(z) is a polynomial for an explicit method, it is a rational function for an implicit method.
22 Hence, the region of absolute stability can be unbounded. As shown in Section 3., a method of order p will satisfy R(z) =e z + O(z p+ ): Rationalfunction approximations of the exponential are called Pade approximations. Denition The (j k) Pade approximation R jk (z) is the maximumorder approximation of e z having the form R jk (z) = P k(z) Q j (z) = p + p z + :::+ p k z k q + q z + :::+ q j z j (3.3.5a) where P k and Q j have no common factors, Q j () = q = (3.3.5b) and R jk (z) =e z + O(z k+j+ ): (3.3.5c) With R jk normalized by (3.3.5b), there are k + j + undetermined parameters in (3.3.5a) that can be determined by matching the rst k + j +terms in the Taylor's series expansion of e z. Thus, the error of the approximation should be O(z k+j+ ). Using (3.3.5c), we have k+j X i= z i i! = P k i= p iz i P j i= q iz i + O(zk+j+ ) (3.3.6) Equating the coecients of like powers of z determines the parameters p i, i = ::: k and q i, i = ::: j. Example Find the (,) Pade approximation of e z. Setting j =andk =in (3.3.6) gives ( + z + z )( + q z + q z )=p : Equating the coecients of z i, i =, gives p = +q = + q + q =
23 Thus, p = q = ; q ==: Using (3.3.5), the (,) Pade approximation is R (z) = ; z + z = : Additionally, e z = R (z)+o(z 3 ): Some other Pade approximations are presented in Table We recognize that the (,) approximation corresponds to Euler's method, the (,) method corresponds to the backward Euler method, and the (,) approximation corresponds to the midpoint rule. (The (,) approximation also corresponds to the trapezoidal rule.) Methods corresponding to the (s s) diagonal Pade approximations are Butcher's maximum order implicit RungeKutta methods (Theorem 3.3.). k = j = +z +z + z = ;z +z= ;z= +z=3+z =6 ;z=3 ;z+z = +z=3 ;z=3+z =6 +z=+z = ;z=+z = Table 3.3.: Some Pade approximations of e z. Theorem There is one and only one sorder sstage implicit RungeKutta formula and it corresponds to the (s s) Pade approximation. Proof. cf. Butcher [7]. We'll be able to construct several implicit RungeKutta methods having unbounded absolutestability regions. We'll want to characterize these methods according to their behavior as jzj!and this requires some additional notions of stability. Denition A numerical method is Astable if its region of absolute stability includes the entire lefthalf plane Re(h). 3
24 The relationship between Astability and the Pade approximations is established by the following theorem. Theorem Methods that lead to a diagonal or one of the rst two subdiagonals of the Pade table for e z are Astable. Proof. The proof appears in Ehle [3]. Without introducing additional properties of Pade approximations, we'll make some observations using the results of Table We have shown that the regions of absolute stability of the backward Euler method and the midpoint rule include the entire lefthalf of the h plane hence, they are Astable.. The coecients of the highestorder terms of P s (z) and Q s (z) are the same for diagonal Pade approximations R ss (z) hence, jr ss (z)j! as jzj! and these methods are Astable (Table 3.3.). 3. For the subdiagonal (,) and (,) Pade approximations, jr(z)j! as jzj! and these methods will also be Astable. It is quite dicult to nd highorder Astable methods. Implicit RungeKutta methods provide the most viable approach. Examining Table 3.3., we see that we canintro duce another stability notion. Denition A numerical method is Lstable if it is Astable and if jr(z)j! as jzj!. The backward Euler method and, more generally, methods corresponding to subdiagonal Pade approximations in the rst two bands are Lstable ([7], Section IV.4). Lstable methods are preferred for sti problems where Re() but methods where jr(z)j! are more suitable when Re() but jim()j, i.e., when solutions oscillate rapidly. Explicit RungeKutta methods are easily solved, but implicit methods will require an iterative solution. Since implicit methods will generally be used for sti systems, 4
25 Newton's method will be preferred to functional iteration. To emphasize the diculty, we'll illustrate RungeKutta methods of the form (3..3) for vector IVPs y = f(t y) y() = y (3.3.7) where y, etc. are mvectors. The application of (3..3) to vector systems just requires the use of vector arithmetic thus, Y i = y n; + h j= a ij f(t n; + c j h Y j ) i = ::: s (3.3.8a) y n = y n; + h i= b i f(t n; + c i h Y i ): (3.3.8b) Once again, y n etc. are mvectors. To use Newton's method, we write the nonlinear system (3.3.8a) in the form F i (Y Y ::: Y s )=Y i ; y n; ; h and get 6 4 j= I ; a J () ;ha J () ;ha s J () s ;ha J () I ; ha J () ;ha s J () s ;ha s J () ;ha s J () I ; ha ss J () s a ij f(t n; + hc j Y j )= Y () Y (). Y () s = ; 6 4 F () F (). F () s j = ::: s 3 7 (3.3.9a) 5 (3.3.9b) Y (+) i = Y () i + Y () i i = ::: s = ::: (3.3.9c) where J () j = f y (t n; + hc j Y () j ) F () j = F j (Y () Y () ::: Y s () ) j = ::: s: (3.3.9d) For an sstage RungeKutta method applied to an mdimensional system (3.3.7), the Jacobian in (3.3.9b) has dimension sm sm. This will be expensive for highorder methods and highdimensional ODEs and will only be competitive with, e.g., implicit 5
26 multistep methods (Chapter 5) under special conditions. Some simplications are possible and these can reduce the work. For example, we can approximate all of the Jacobians as J = f y (t n; y n; ): (3.3.a) In this case, we can even shorten the notation by introducing the Kronecker or direct product of two matrices as A J = 6 4 Then, (3.3.9b) can be written concisely as 3 7 a J a J a s J a J a J a s J : (3.3.b) a s J a s J a ss J (I ; ha J)Y () = ;F () (3.3.c) where A was given by (3..6c) and Y () = 6 4 Y () Y (). Y () s F() = 6 4 F () F (). F () s : (3.3.d) The approximation of the Jacobian does not change the accuracy of the computed solution, only the convergence rate of the iteration. As long as convergence remains good, the same Jacobian can be used for several time step and only be reevaluated when convergence of the Newton iteration slows. Even with this simplication, with m ranging into the thousands, the solution of (3.3.) is clearly expensive and other ways of reducing the computational cost are necessary. Diagonally implicit RungeKutta (DIRK) methods oer one possibility. A DIRK method is one where a ij =, i < j and at least one a ii 6=, i j = ::: s. If, in addition, a = a = :::= a ss = a, the technique is known as a singly diagonally implicit RungeKutta (SDIRK) method. Thus, the coecient matrix of an SDIRK method has 6
27 the form A = 6 4 a a a..... a s a s a : (3.3.) Thus, with the approximation (3.3.), the system Jacobian in (3.3.c) is (I ; ha J) = I ; haj ;ha J I ; haj : ;ha s J ;ha s J I ; haj The Newton system (3.3.) is lower block triangular and can be solved by forward substitution. Thus, the rst block of (3.3.c) is solved for Y (). Knowing Y the second equation is solved for Y (), etc. The Jacobian J is the same for all stages thus, the diagonal blocks need only be factored once by Gaussian elimination and forward and backward substitution may be used for each solution. The implicit midpoint rule(3.3.) is a onestage, secondorder DIRK method. We'll examine a twostage DIRK method momentarily, but rst we note that the maximum order of an sstage DIRK method is s + []. Example A twostage DIRK formula has the tableau c a c a a b b and it could be of third order. According to Theorem 3.., the conditions for secondorder accuracy are (3..5c) with l = when(3..5b) is satised, i.e., b + b = c = a c = a + a b c + b c ==: (As noted earlier, satisfaction of (3..5b) is not necessary, but it simplies the algebraic manipulations.) We might guess that the remaining conditions necessary for third order accuracy are (3..5c) with l =3and (3..7) with k =3,i.e., b c + b c ==3 and b T A l = bac = b a c + b (a c + a c )==6 7
28 where (3..5b) was used to simplify the last expression. After some eort, this system of six equations in seven unknowns can be solved to yield c = ; 3c b = c ; = b = = ; c 3 ; 6c c ; c c ; c a = c a = =6 ; b c b (c ; c ) a = c ; a : As written, the solution is parameterized by c. Choosing c ==3 gives Using (3..3), the method is /3 /3 / / 3/4 /4 Y = y n; + h 3 f(t n; + h 3 Y ) Y = y n; + h [f(t n; + h 3 Y )+f(t n Y )] y n = y n; + h 4 [3f(t n; + h 3 Y )+f(t n Y )]: We can check by constructing a Taylor's series that this method is indeed third order. Hairer et al. [6], Section II., additionally show that our necessary conditions for thirdorder accuracy are also sucient in this case. The computation of Y can be recognized as the backward Euler method for onethird of the time step h. The computation of Y and y n are not recognizable in terms of simple quadrature rules. Since the method is thirdorder, its local error is O(h 4 ). We can also construct an SDIRK method by insisting that a = a. Enforcing this condition and using the previous relations gives two methods having the tableau where ; ; = = = ( p 3 ): The method with = ( + = p 3)= is Astable while the other method has a bounded stability region. Thus, this would be the method of choice. 8
29 Let us conclude this Section by noting a relationship between implicit RungeKutta and collocation methods. With u(t) a polynomial of degree s in t for t t n;,acollocation method for the IVP y = f(t y) y(t n; )=y n; (3.3.a) consists of solving u(t n; )=y n; (3.3.b) u (t n; + c i h)=f(t n; + c i h u(t n; + c i h)) i = ::: s (3.3.c) where c i, i = ::: s, are nonnegative parameters. Thus, the collocation method consists of satisfying the ODE exactly at s points. The solution u(t n; + h) maybeused as the initial condition y n for the next time step. Usually, the collocation points t n; + c i h are such that c i [ ], i = ::: s,but this need not be the case [6,, ]. Generally, the c i, i = ::: s, are distinct and we shall assume that this is the case here. (The coecients need not be distinct when the approximation u(t) interpolates some solution derivatives, e.g., as with Hermite interpolation.) t t n;, by a Lagrange interpolating polynomial of degree s ;, we have u (t) = j= Approximating u (t), k j L j ( t ; t n; ) (3.3.3a) h where L j () = sy i= i6=j ; c i c j ; c i (3.3.3b) = t ; t n; : (3.3.3c) h The polynomials L j (), j = ::: s, are a product of s ; linear factors and are, hence, of degree s ;. They satisfy L j (c i )= ji j i = ::: s (3.3.3d) 9
30 where ji is the Kronecker delta. Using (3.3.3a), we see that u (t) satises the interpolation conditions u (t n; + c i h)=k i i = ::: s: (3.3.3e) Transforming variables in (3.3.3a) using (3.3.3c) u(t n; + h)=y n; + h Z u (t n; + h)d: (3.3.4) By construction, (3.3.4) satises (3.3.b). Substituting (3.3.3e) and (3.3.4) into (3.3.c), we have k i = f(t n; + c i h y n; + h Z c i k j L j ()d): j= This formula is identical to the typical RungeKutta formula (3..b) provided that a ij = Z c i L j ()d: (3.3.5a) Similarly, using (3.3.3a) in (3.3.4) and evaluating the result at = yields u(t n; + h) =y n = y n; + h This formula is identical to (3..a) provided that Z k j L j ()d: j= b j = Z L j ()d: (3.3.5b) This view of a RungeKutta method as a collocation method is useful in many situations. Let us illustrate one result. Theorem A RungeKutta method with distinct c i, i = ::: s, and of order at least s is a collocation method satisfying (3.3.), (3.3.5) if and only if it satises the order conditions j= a ij c q; j = cq i i q = ::: s: (3.3.6) q Remark. The order conditions (3.3.5) are related to the previous conditions (3..5c, 3..7) (cf. [6], Section II.7). 3
31 Proof. We use the Lagrange interpolating polynomial (3.3.3) to represent any polynomial P () of degree s ; as P () = j= P (c j )L j (): Regarding P () as u (t n + h), integrate to obtain u(t n; + c i h) ; y n; = Z c i P ()d = P (c j ) Z c i j= L j ()d i = ::: s: Assuming that (3.3.5a) is satised, we have Z c i P ()d = j= a ij P (c j ) i = ::: s: Now choose P () = q;, q = ::: s, to obtain (3.3.6). The proof of the converse follows the same arguments (cf. [6], Section II.7). Now, we might ask if there is an optimal way of selecting the collocation points. Appropriate strategies would select them so that accuracy and/or stability are maximized. Let's handle accuracy rst. The following theorems discuss relevant accuracy issues. Theorem (Alekseev and Grobner) Let x, y, and z satisfy x (t z()) = f(t x(t z())) x( z()) = z() (3.3.7a) y (t) =f(t y(t)) y() = y (3.3.7b) with f y (t y) C, t>. Then, z (t) =f(t z(t)) + g(t z(t)) z() = y (3.3.7c) z(t) ; y(t) = Z z()) g( (3.3.7d) Remark. Formula (3.3.7d) is often called the nonlinear variation of parameters. Remark 3. The parameter identies the time that the initial conditions are applied in (3.3.7a). A prime, as usual, denotes t dierentiation. Remark 4. Observe that y(t) =x(t y ). 3
32 Proof. cf. Hairer et al. [6], Section I.4, and Problem. Theorem makes it easy for us to associate the collocation error with a quadrature error as indicated below. Theorem Consider the quadrature rule where Z t n tn; F (t)dt = h Z F (t n; + h)d = h i= b i F (t n; + c i h)+e p (3.3.8a) E p = Ch p+ F (p) ( n ) n (t n; t n ) (3.3.8b) F C p (t n; t n ),andc is a constant. Then the collocation method (3.3.) has order p. Proof. Consider the identity u = f(t u)+[u ; f(t u)] and use Theorem on [t n; t n ] with z(t) =u(t) andg(t u) =u ; f(t u) to obtain u(t n ) ; y(t n )= Z t n x u (t n u())[u () ; f( u())]d: tn; Replace this integral by the quadrature rule (3.3.8) to obtain u(t n ) ; y(t n )=h i= b i x u (t n t n; + c i h u(t n; + c i h))[u (t n; + c i h); f(t n; + c i h u(t n; + c i h))] + E p : All terms in the summation vanish upon use of the collocation equations (3.3.) thus, ju(t n ) ; y(t n )j = je p jjcjh p+ max [tn; p x u(t n u())[u () ; f( u)]j: It remains to show that the derivatives in the above expression are bounded as h!. We'll omit this detail which is proven in Hairer et al. [6], Section II.7. Thus, and the collocation method (3.3.) is of order p. jy(t n ) ; u(t n )j ^Ch p+ (3.3.9) 3
33 At last, our task is clear. We should select the collocation points c i, i = ::: s, to maximize the order p of the quadrature rule (3.3.8). We'll review some of the details describing the derivation of (3.3.8). Additional material appears in most elementary numerical analysis texts [4]. Let ^F () =F (t n; + h) and approximate it by a Lagrange interpolating polynomial of degree s ; to obtain ^F () = j= ^F (c j )L j ()+ M s() s! ^F (s) () ( ) (3.3.a) where M s () = sy i= ( ; c i ): (3.3.b) (Dierentiation in (3.3.a) is with respect to, not t.) Integrate (3.3.a) and use (3.3.5b) to obtain Z ^F ()d = j= b j ^F (cj )+ ^E s (3.3.a) where ^E s = s! Z M s () ^F (s) (())d = s! Z sy i= ( ; c i ) ^F (s) (())d: (3.3.b) In NewtonCotes quadrature rules, such as the trapezoidal and Simpson's rules, the evaluation points c i, i = ::: s, are specied a priori. With Gaussian quadrature, however, the points are selected to maximize the order of the rule. This can be done by expanding ^F (s) (()) in a Taylor's series and selecting the c i, i = ::: s, to annihilate as many terms as possible. Alternatively, and equivalently, the quadrature rule can be designed to integrate polynomials exactly to as high a degree as possible. The actual series expansion is complicated by the fact that ^F (s) is evaluated at () in (3.3.b). Isaacson and Keller [9] provide additional details on this matter however, we'll sidestep the subtleties by assuming that all derivatives of () are bounded so that ^F (s) has an expansion in powers of of the form ^F (s) () = + + :::+ r; r; + O( r ): 33
34 d P d (x) x x ; 3 3 x 3 ; 3x 5 4 x 4 ; 6x x 5 ; + 5x x3 9 Table 3.3.: Legendre polynomials P d (x) of degree d [ 5] on ; x. The rst r terms of this series will be annihilated by (3.3.b) if M s () is orthogonal to polynomials of degree r ;, i.e., if Z M() q; d = q = ::: r: (3.3.) Under these conditions, were we to transform the integrals in (3.3.) and (3.3.) back to t dependence using (3.3.3c), we would obtain the error of (3.3.8b) with p = s + r. With the s coecients c i, i = ::: s, we would expect the maximum value of r to be s. According to Theorem 3.3.6, this choice would lead to a collocation method of order s, i.e., a method having p = r + s = s and an O(h s+ ) local error. These are Butcher's maximal order formulas (Theorem 3.3.) corresponding to the diagonal Pade approximations. The maximumorder coecients identied above are the roots of the s thdegree Legendre polynomial scaled to the interval ( ). The rst six Legendre polynomials are listed in Table Additional polynomials and their roots appear in Abromowitz and Stegun [], Chapter. Example According to Table 3.3., the roots of P (x) are x = = p 3 on [; ]. Mapping these to [ ] by the linear transformation = ( + x)=, we obtain the collocation points for the maximalorder twostage method as c = ( ; p 3 ) c = ( + p 3 ): 34
35 Since this is our rst experience with these techniques, let us verify our results by a direct evaluation of (3.3.) using (3.3.b) thus, Integrating Z ( ; c )( ; c )d = 3 ; c + c + c c = Z ( ; c )( ; c )d =: 4 ; c + c + c c 3 =: These may easily be solved to conrm the collocation points obtained by using the roots of P (x). In this case, we recognize c and c as the evaluation points of the Hammer Hollingsworth formula of Example With the collocation points c i, i = ::: s, determined, the coecients a ij and b j, i j = ::: s, may be determined from (3.3.5a,b). These maximal order collocation formulas are Astable since they correspond to diagonal Pade approximations (Theorem 3.3.3). We may not want to impose the maximal order conditions to obtain, e.g., better stability and computational properties. With Radau quadrature, we x one of the coef cients at an endpoint thus, we set either c =or c s =. The choice c =leads to methods with bounded regions of absolute stability. Thus, the methods of choice have c s =. They correspond to the subdiagonal Pade approximations and are, hence, A and Lstable (Theorem 3.3.3). They have orders of p = s ; [7], Section IV.5. Such excellent stability and accuracy properties makes these methods very popular for solving sti systems. The Radau polynomial of degree s on ; x is R s (x) =P s (x) ; s s ; P s;(x): The roots of R s transformed to [ ] (using =(+x)=) are the c i, i + ::: s. All values of c i, i = ::: s, are on ( ] with, as designed, c s =. The onestage Radau method is the backward Euler method. The tableau of the twostage Radau method is (Problem )
36 We'll conclude this Section with a discussion of singly implicit RungeKutta (SIRK) methods. These methods are of order s, which is less than the Legendre (s), Radau (s ; ), and DIRK (s + ) techniques. They still have excellent A and Lstability properties and, perhaps, oer a computational advantage. A SIRK method is one where the coecient matrix A has a single sfold real eigenvalue. These collocation methods were Originally developed by Butcher [9] and have been subsequently extended [5,, 6, ]. Collocating, as described, leads to the system ( ). The intermediate solutions Y i, i = ::: s,have the vector form specied by (3..9d) with the elements of A given by (3.3.5a). Multiplying (3..9d) by a nonsingular matrix T ;,we obtain T ; Y = y n T ; l + ht ; ATT ; f where Y, l, A, and f are, respectively, given by (3..6ac) and Let f = 6 4 f(t n; + c h) f(t n; + c h). f(t n; + c s h) : (3.3.3) ^Y = T ; Y ^l = T ; l ^A = T ; AT ^f = T ; f: (3.3.4) Butcher [9] chose the collocation points c i = i, i = ::: s, where i is the i th root of the sthdegree Laguerre polynomial L s (t) and is chosen so that the numerical method has favorable stability properties. butcher also selected T to have elements T ij = L i; ( j ): Then ^A = : (3.3.5) 36
37 Thus, ^A is lower bidiagonal with the single eigenvalue. The linearized system (3.3.9) is easily solved in the transformed variables. (A similar transformation also works with Radau methods [7].) Butcher [9] and Burrage [5] show that it is possible to nd A stable SIRK methods for s 8. These methods are also Lstable with the exception of the sevenstage method. Problems. Verify that (3.3.7d) is correct when f(t y) =ay with a a constant.. Consider the method y n = y n; + h[( ; )f(t n; y n; )+f(t n y n )] with [ ]. The method corresponds to the Euler method with =, the trapezoidal rule with ==, and the backward Euler method and when =... Write the RungeKutta tableau for this method... For what values of is the method Astable? Justify your answer. 3. Radau or Lobatto quadrature rules have evaluation points at one or both endpoints of the interval of integration, respectively. Consider the twotwostage RungeKutta methods based on collocation at Radau points. In one, the collocation point c = and in the other the collocation point c =. In each case, the other collocation point (c for the rst method and c for the second method) is to be determined so that the resulting method has as high an order of accuracy as possible. 3.. Determine the parameters a ij, b j, and c i, i j = for the two collocation methods and identify their orders of accuracy. 3.. To which elements of the Pade table do these methods correspond? 3.3. Determine the regions of absolute stability for these methods? Are the methods A and/or Lstable? 37
38 3.4 Convergence, Stability, Error Estimation The concepts of convergence, stability, anda priori error estimation introduced in Chapter readily extend to a general class of (explicit or implicit) onestep methods having the form y n = y n; + h(t n; y n; h): (3.4.a) Again, consider the scalar IVP y = f(t y) y() = y (3.4.b) and, to begin, we'll show that onestep methods are stable when satises a Lipschitz condition on y. Theorem If (t y h) satises a Lipschitz condition on y then the onestep method (3.4.a) is stable. Proof. The analysis follows the lines of Theorem... Let y n and z n satisfy method (3.4.) and z n = z n; + h(t n; z n; h) z = y + (3.4.) respectively. Subtracting (3.4.) from (3.4.) y n ; z n = y n; ; z n; + h[(t n; y n; h) ; (t n; z n; h)]: Using the Lipschitz condition jy n ; z n j( + hl)jy n; ; z n; j: Iterating the above inequality leads to jy n ; z n j( + hl) n jy ; z j: Using (..) jy n ; z n je nhl j je LT k since nh T and j j. 38
39 Example The function satises a Lipschitz condition whenever f does. Consider, for example, the explicit midpoint rule which has the form of (3.4.a) with (t y h) =f(t + h= y+ hf(t y)=): Then, j(t y h) ; (t z h)j = jf(t + h= y+ hf(t y)=) ; f(t + h= z+ hf(t z)=)j Using the Lipschitz condition on f j(t y h) ; (t z h)j Ljy + hf(t y)= ; z ; hf(t z)=j or or j(t y h) ; (t z h)j L[jy ; zj +(h=)jf(t y) ; f(t z)j] j(t y h) ; (t z h)j L( + hl=)jy ; zj: Thus, we can take the Lipschitz constant for to be L( + ^hl=) for h ( ^h]. In addition to a Lipschitz condition, convergence of the onestep method (3.4.a) requires consistency. Recall (Denition..3), that consistency implies that the local discretization error lim h! n =. Consistency is particularly simple for a onestep method. Lemma The onestep method (3.4.a) is consistent with the ODE y = f(t y) if (t y ) = f(t y): (3.4.3) Proof. The local discretization error of (3.4.a) satises Letting h tend to zero n = y(t n) ; y(t n; ) h ; (t n; y(t n; ) h): lim n = y (t n; ) ; (t n; y(t n; ) ): h! Using the ODE to replace y yields the result. 39
40 Theorem Let (t y h) be a continuous function of t, y, and h on t T, ; < y <, and h ^h, respectively, and satisfy a Lipschitz condition on y. Then the onestep method (3.4.a) converges to the solution of (3.4.b) if and only if it is consistent. Proof. Let z(t) satisfy the IVP z = (t z ) z() = y (3.4.4) and let z n, n, satisfy z n = z n; + h(t n; z n; h) n z = y : (3.4.5) Using the mean value theorem and (3.4.4) z(t n ) ; z(t n; )=hz (t n; + h n )=h(t n; + h n z(t n; + h n ) ) (3.4.6) where n ( ). Let e n = z(t n ) ; z n (3.4.7) and subtract (3.4.5) from (3.4.6) to obtain e n = e n; + h[(t n; + h n z(t n; + h n ) ) ; (t n; z n; h)]: Adding and subtracting similar terms e n = e n; + h[(t n; + h n z(t n; + h n ) ) ; (t n; z(t n; ) ) + (t n; z(t n; ) h) ; (t n; z n; h) + (t n; z(t n; ) ) ; (t n; z(t n; ) h)]: (3.4.8a) Using the Lipschitz condition j(t n; z(t n; ) h) ; (t n; z n; h)j Lje n j: (3.4.8b) Since (t y h) C, it is uniformly continuous on the compact set t [ T], y = z(t), h [ ^h] thus, (h) = max j(t n; z(t n; ) ) ; (t n; z(t n; ) h)j = O(h): t[ T ] (3.4.8c) 4
41 Similarly, (h) = max j(t n; + h n z(t n; + h n ) ) ; (t n; z(t n; ) )j = O(h): t[ T ] (3.4.8d) Substituting (3.4.8b,c,d) into (3.4.8a) je n jje n; j + h[lje n; j + (h)+(h)]: (3.4.9) Equation (3.4.9) is a rst order dierence inequality with constant (independent of n) coecients having the general form je n jaje n; j + B (3.4.a) where, in this case, A =+hl (3.4.b) B = h[(h)+(h)]: (3.4.c) The solution of (3.4.a) is je n ja n je j + A n ; B n : A ; Since e =,we have ( + hl) n ; je n j h[(h)+(h)] hl or, using (..) e LT ; je n j [(h)+(h)]: L Both (h) and (h) approach zero as h! therefore, lim z n = z(t n ): h! n! Nh=T Thus, z n converges to z(t n ), where z(t) is the solution of (3.4.4). If the onestep method satises the consistency condition (3.4.3), then z(t) =y(t). Thus, y n converges to y(t n ), n. This establishes suciency of the consistency condition for convergence. 4
42 In order to show that consistency is necessary for convergence, assume that the onestep method (3.4.a) converges to the solution of the IVP (3.4.b). Then, y n! y(t n ) for all t [ T] as h! and N!. Now, z n, dened by (3.4.5), is identical to y n, so z n must also converge to y(t n ). Additionally, we have proven that z n converges to the solution z(t) of the IVP (3.4.4). Uniqueness of the solutions of (3.4.4) and (3.4.b) imply that z(t) = y(t). This is impossible unless the consistency condition (3.4.3) is satised. Global error bounds for general onestep methods (3.4.) have the same form that we saw in Chapter for Euler's method. Thus, a method of order p will converge globally as O(h p ). Theorem Let satisfy the conditions of Theorem 3.4. and let the onestep method be of order p. Then, the global error e n = y(t n ) ; y n is bounded by je n j Chp L (elt ; ): (3.4.) Proof. Since the onestep method is of order p, there exists a positive constant C such that the local error d n satises jd n jch p+ : The remainder of the proof follows the lines of Theorem.... Prove Theorem Problems 3.5 Implementation: Error and Step Size Control We would like to design software that automatically adjusts the step size so that some measure of the error, ideally the global error, is less than a prescribed tolerance. While automatic variation of the step size is easy with onestep methods, it is very dicult to compute global error measures. A priori bounds, such as (3.4.), tend to be too conservative and, hence, use very small step sizes (cf. [6], Section II.3). Other more accurate procedures (cf. [5], pp. 34) tend to be computationally expensive. Controlling a 4
43 measure of the local (or local discretization) error, on the other hand, is fairly straight forward and this is the approach that we shall study in this section. A pseudocode segment illustrating the structure of a onestep method y n = y n; + h(t n; y n; h) (3.5.a) that performs a single integration step of the vector IVP y = f(t y) y() = y (3.5.b) is shown in Figure On input, y contains an approximation of the solution at time t. On output, t is replaced by t + h and y contains the computed approximate solution at t + h. The step size must be dened on input, but may be modied each time the computed error measure fails to satisfy the prescribed tolerance. procedure onestep (f: vector function : real var t, h: real var y: vector) begin repeat Integrate (3.5.b) from t to t + h using (3.5.a) Compute error measure at t + h if error measure >then Calculate a new step size h until error measure t = t + h Suggest a step size h for the next step end Figure 3.5.: Pseudocode segment of a onestep numerical method with error control and automatic step size adjustment. In addition to supplying a onestep method, the procedure presented in Figure 3.5. will require routines to compute an error measure and to vary the step size. concentrate on the error measure rst. We'll Example Let us calculate an estimate of the local discretization error of the midpoint rule predictorcorrector. We do this by subtracting the Taylor Taylor's series expansion of the exact solution (3.., 3..3) from the expansion of the RungeKutta formula (3..7) with a = c ==, b =,and b =. The result is d n = h3 6 [3(f tt +ff ty + f f yy ) ; (f t f y + ff y )] (t n; y ( tn;) + O(h 4 ): 43
44 Clearly this is too complicated to be used as a practical error estimation scheme. Two practical approaches to estimating the local and local discretization errors of RungeKutta methods are (i) Richardson's extrapolation (or step doubling) and (ii) embedding. We'll study Richardson's extrapolation rst. For simplicity, consider a scalar onestep method of order p having the following form and local error y n = y n; + h(t n; y n; h) (3.5.a) d n = C n h p+ + O(h p+ ): (3.5.b) The coecient C n may depend on t n; and y(t n; ) but is independent of h. Typically, C n is proportional to y (p+) (t n; ). Of course, the ODE solution must have derivatives of order p + for this formula to exist. Let y h n be the solution obtained from (3.5.a) using a step size h. Calculate a second solution y h= n at t = t n using two steps with a step size h= and an \initial condition" of y n; at t n;. (We'll refer to the solution computed at t n;= = t n; + h= as y h= n;=. Assuming that the error after two steps of size h= is twice that after one step (i.e., C n;= C n ), the local errors of both solutions are y h n ; y(t n )=C n h p+ + O(h p+ ) and y h= n ; y(t n )=C n (h=) p+ + O(h p+ ) Subtracting the two solutions to eliminate the exact solution gives y h n ; y h n= =C n h p+ ( ; ;p )+O(h p+ ): Neglecting the O(h p+ ) term, we estimate the local error in the solution of (3.5.a) as jd n jjc n jh p+ = jyh n ; yn h= j : (3.5.3a) ; ;p Computation of the error estimate requires s additional function evaluations (to compute y n;= and y n ) for an sstage RungeKutta method. If s p then approximately 44
45 p extra function evaluations (for scalar systems). This cost for mdimensional vector problems is approximately pm function evaluations per step. Richardson's extrapolation is particularly expensive when used with implicit methods because the change of step size requires another Jacobian evaluation and (possible) factorization. It may, however, be useful with DIRK methods because of their lower triangular coecient matrices. It's possible to estimate the error of the solution y h= n as jd h= n j jc njh p+ = jyh n ; yn h= j : (3.5.3b) p p ; Proceeding in this manner seems better than accepting y h n as the solution however, it is a bit risky since we do not have an estimate of the error of the intermediate solution y h= n;=. Finally, the local error estimate (3.5.3a) or (3.5.3b) may be added to y h n or y h= n, respectively, to obtain a higherorder method. For example, using (3.5.3b), Thus, we could accept y(t n )=yn h= + yh n ; yn h= p ; + O(hp+ ): ^y n h= = yn h= + yh n ; yn h= p ; as an O(h p+ ) approximation of y(t n ). This technique, called local extrapolation, is also a bit risky since we do not have an error estimate of ^y n h=. We'll return to this topic in Chapter 4. Embedding, the second popular means of estimating local (or local discretization) errors, involves using two onestep methods having dierent orders. Thus, consider calculating two solutions using the p th and p + storder methods and y p n = y n; + h p (t n; y n; h) d p n = C p nh p+ (3.5.4a) y p+ n = y n; + h p+ (t n; y n; h) d p+ n = C p+ n h p+ : (3.5.4b) (The superscripts on y n and d n are added to distinguish solutions of dierent order.) The local error of the porder solution is jd p nj = jyn p ; y(t n )j = jyn p ; yn p+ + yn p+ ; y(t n )j: 45
46 Using the triangular inequality jd p njjyn p ; yn p+ j + jyn p+ ; y(t n )j: The last term on the right is the local error of the order p +method (3.5.4b) and is O(h p+ ) thus, jd p njjy p n ; y p+ n j + jd p+ n j: The higherorder error term on the right may be neglected to get an error estimate of the form jd p njjy p n ; y p+ n j: (3.5.5) Embedding, like Richardson's extrapolation, is also an expensive way of estimating errors. If the number of RungeKutta stages s p, then embedding requires approximately m(p + )additional function evaluations per step for a system of m ODEs. The number of function evaluations can be substantially reduced by embedding the pthorder method within an (s+)stage method of order p+. For explicit RungeKutta methods, the tableau of the (s + )stage method would have the form c a c 3 a 3 a c s+ a s+ a s+ a s+ s ^b ^b ^bs ^bs+ (Zero's on an above the diagonal in A are not shown.) Assuming that the p thorder RungeKutta method has s stages, it would be required to have the form c a c 3 a 3 a c s a s a s a s s; b b b s+ b s With this form, only one additional function evaluation is needed to estimate the error in the (lower) p thorder method. However, the derivation of such formula pairs is 46
47 not simple since the order conditions are nonlinear. Additionally, it may be impossible to obtain a p + order method by adding a single stage to an sstage method. Formulas, nevertheless, exist. Example The forward Euler method is embedded in the trapezoidal rule predictorcorrector method. The tableaux for these methods are The two methods are / / k = f(t n; y n; ) k = f(t n; + h y n; + hk ) y n = y n; + hk y n = y n; + h (k + k ): Example There is a threestage, secondorder method embedded in the classical fourthorder RungeKutta method. Their tableaux are These formulas are / / / / /6 /3 /3 /6 / / / / k = f(t n; y n; ) k = f(t n; + h= y n; + hk =) k 3 = f(t n; + h= y n; + hk =) k 4 = f(t n; + h y n; + hk 3 ) y n = y n; + hk 3 47
48 y 4 n = y n; + h 6 (k +k +k 3 + k 4 ): Example Fehlberg [4] constructed pairs of explicit RungeKutta formulas for nonsti problems. His fourth and fthorder formula pair is ; ; ; 8 ; ; ; 44 5 ^ ; The ^ denotes the coecients in the higher fthorder formula. Thus, after determining k i, i = ::: 6, the solutions are calculated as y 4 n = y n; + h[ 5 6 k k k 4 ; 5 k 5] and y 5 n = y n; + h[ 6 35 k k k 4 ; 9 5 k k 6]: Hairer et al. [6], Section II.4 give several Fehlberg formulas. Their fourth and fthorder pair is slightly dierent than the one presented here. Example Dormand and Prince [] develop another fourth and fthorder pair that has been designed to minimize the error coecient of the higherorder method so that it may beused with local extrapolation. Its tableau follows. 48
49 ; ; ; ; ; ; ; ^ ; Having procedures for estimating local (or local discretization) errors, we need to develop practical methods of using them to control step sizes This will involve the selection of an appropriate (i) error measure, (ii) error test, and (iii) renement strategy. As indicated in Figure 3.5., we will concentrate on step changing algorithms without changing the order of the method. Techniques that automatically vary the order of the method with the step size are more dicult and are not generally used with RungeKutta methods (cf., however, Moore and Flaherty []). For vector IVPs (3.5.b), we will measure the \size" of the solution or error estimate by using avector norm. Many such metrics are possible. Some that suit our needs are. the maximum norm ky(t)k = max im jy i(t)j (3.5.6a). the L or sum norm ky(t)k = mx jy i (t)j (3.5.6b) i= 3. and the L or Euclidean norm " mx i= # = ky(t)k = jy i (t)j : (3.5.6c) 49
50 The two most common error tests are control of the absolute and relative errors. An absolute error test would specify that the chosen measure of the local error be less than a prescribed tolerance thus, k d ~ n k A where the ~ signies the local error estimate rather than the actual error. Using a relative error test, we would control the error measure relative tothe magnitude of the solution, e.g., k d ~ n k R ky n k: It is also common to base an error test on a combination of an absolute and a relative tolerance, i.e., k d ~ n k R ky n k + A : When some components of the solution are more important than others it may be appropriate to use a weighted norm with y i (t) in (3.5.6) replaced by y i (t)=w i, where w =[w w ::: w m ] T (3.5.7a) is a vector of positive weights. As an example, consider the weighted maximum norm of the local error estimate k ~ d n k w = max im ~d n i w i where ~ d n i denotes the local error estimate of the i th component ofd n. Use of a weighted test such as (3.5.7b) k ~ d n k w (3.5.7c) adds exibility to the software. Users may assign weights prior to the integration in proportion to the importance of a variable. The weighted norm may also be used to simulate a variety of standard tests. Thus, for example, an absolute error test would be obtained by setting w i =, i = ::: m, and = A. A mixed error test where the integration step is accepted if the local error estimate of the ithode does not exceed R jy n i j + A 5
51 may be specied by using the maximum norm and selecting =max( A R ) and Present RungeKutta software controls: w i =( R jy n i j + A )=:. the local error k ~ d n k w (3.5.8a). the local error perunitstep k ~ d n k w h (3.5.8b) 3. or the indirect (extrapolated) local error per unit step k ~ d n k w Ch (3.5.8c) where C is a constant depending on the method. The latter two formulas are attempts to control a measure of the global error. Let us describe a step size selection process for controlling the local error per unit step in a p th order RungeKutta method. Suppose that we have just completed an integration from t n; to t n. We have computed an estimate of the local error d ~ n using either Richardson's extrapolation or order embedding. We compare k d ~ n k w with the prescribed tolerance and. if k d ~ n k w >we reject the step and repeat the integration with a smaller step size,. otherwise we accept the step and suggest a step size for the subsequent step. In either case, k ~ d n k w h C n h p : 5
52 Ideally, we would like to compute a step size h OPT so that C n h p OPT : Eliminating the coecient C n between the two equations or h OPT h k ~ d n k w hp OPT h p+ h k ~ d n k w =p : (3.5.9a) The error estimates are based upon an asymptotic analysis and are, thus, not completely reliable. Therefore, it is best to include safety factors such as ( " h OPT = h min MAX max MIN s k ~ d n k w =p #) : (3.5.9b) The factors MAX and MIN limit the maximum step size increase and decrease, respectively, while s tends to make step size changes more conservative. Possible choices of the parameters are MAX = 5, MIN = :, and s = :9. Step size control based on either (3.5.8a) or (3.5.8c) works similarly. In general, the user must also provide a maximum step size h MAX so that the code does not miss interesting features in the solution. Selection of the initial step size is typically left to the user. This can be somewhat problematical and several automatic initial step size procedures are under investigation. One automatic procedure that seems to be reasonably robust is to select the initial step size as h = =T p + kf( y())k p =p where T is the nal time and p = p + for local error control and p = p for local error per unit step control. Example ([6], Section II.4). We report results when several explicit fourthorder explicit RungeKutta codes were applied to y =ty log(max(y ; 3)) y () = y = ;ty log(max(y ; 3)) y () = e: 5
53 ; ; ; ; ; 6 36 ; ; 8 43 ; Table 3.5.: Butcher's sevenstage sixthorder explicit RungeKutta method The exact solution of this problem is y (t) =e sin t y (t) =e cos t : Hairer et al. [6] solved the problem on t 5 using tolerances ranging from ;7 to ;3. The results presented in Figure 3.5. compare the base logarithms of the maximum global error and the number of function evaluations. The several methods that are not identied in Figure 3.5. are the more traditional formulas, including the classical RungeKutta method (solid line). All of these are listed in Hairer et al. [6], Section II.. \Fehlberg's method" is the fourth and fthorder pair given in Example The \DormandPrince" method is the fourth and fthorder pair of Example \Butcher's method" is the sixthorder sevenstage formula shown in Table It is the only formula that is beyond fourth or fth order. Results in the lower graph of Figure 3.5. use local extrapolation thus, the higherorder solution of the pair is kept, even though it has no local error estimate. Of all the methods shown in Figure 3.5., the DormandPrince and Fehlberg methods appear to have the greatest accuracy for a given cost. The higherorder Butcher formula gains appeal as accuracy increases. The DormandPrince method has a distinct advantage relative to the Fehlberg method when local extrapolation is used. As noted, the Dormand Prince method was designed for this purpose. For this problem, which has a smooth 53
54 Figure 3.5.: Accuracy vs. eort for several explicit RungeKutta methods [6]. 54
Orthogonal Bases and the QR Algorithm
Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries
More informationON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION. A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT
ON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT A numerical study of the distribution of spacings between zeros
More informationThe Backpropagation Algorithm
7 The Backpropagation Algorithm 7. Learning as gradient descent We saw in the last chapter that multilayered networks are capable of computing a wider range of Boolean functions than networks with a single
More informationSpaceTime Approach to NonRelativistic Quantum Mechanics
R. P. Feynman, Rev. of Mod. Phys., 20, 367 1948 SpaceTime Approach to NonRelativistic Quantum Mechanics R.P. Feynman Cornell University, Ithaca, New York Reprinted in Quantum Electrodynamics, edited
More informationTHE PROBLEM OF finding localized energy solutions
600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Reweighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,
More informationHow to Use Expert Advice
NICOLÒ CESABIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, California
More informationOn the Optimality of the Simple Bayesian. given the class, but the question of whether other sucient conditions for its optimality exist has
,, 1{30 () c Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. On the Optimality of the Simple Bayesian Classier under ZeroOne Loss PEDRO DOMINGOS pedrod@ics.uci.edu MICHAEL PAZZANI
More informationA Modern Course on Curves and Surfaces. Richard S. Palais
A Modern Course on Curves and Surfaces Richard S. Palais Contents Lecture 1. Introduction 1 Lecture 2. What is Geometry 4 Lecture 3. Geometry of InnerProduct Spaces 7 Lecture 4. Linear Maps and the Euclidean
More informationOptimization by Direct Search: New Perspectives on Some Classical and Modern Methods
SIAM REVIEW Vol. 45,No. 3,pp. 385 482 c 2003 Society for Industrial and Applied Mathematics Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods Tamara G. Kolda Robert Michael
More informationSubspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity
Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity Wei Dai and Olgica Milenkovic Department of Electrical and Computer Engineering University of Illinois at UrbanaChampaign
More informationCOSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES
COSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES D NEEDELL AND J A TROPP Abstract Compressive sampling offers a new paradigm for acquiring signals that are compressible with respect
More informationFrom Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
SIAM REVIEW Vol. 51,No. 1,pp. 34 81 c 2009 Society for Industrial and Applied Mathematics From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein David
More informationIndexing by Latent Semantic Analysis. Scott Deerwester Graduate Library School University of Chicago Chicago, IL 60637
Indexing by Latent Semantic Analysis Scott Deerwester Graduate Library School University of Chicago Chicago, IL 60637 Susan T. Dumais George W. Furnas Thomas K. Landauer Bell Communications Research 435
More informationRECENTLY, there has been a great deal of interest in
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 47, NO. 1, JANUARY 1999 187 An Affine Scaling Methodology for Best Basis Selection Bhaskar D. Rao, Senior Member, IEEE, Kenneth KreutzDelgado, Senior Member,
More informationControllability and Observability of Partial Differential Equations: Some results and open problems
Controllability and Observability of Partial Differential Equations: Some results and open problems Enrique ZUAZUA Departamento de Matemáticas Universidad Autónoma 2849 Madrid. Spain. enrique.zuazua@uam.es
More informationFast Greeks by algorithmic differentiation
The Journal of Computational Finance (3 35) Volume 14/Number 3, Spring 2011 Fast Greeks by algorithmic differentiation Luca Capriotti Quantitative Strategies, Investment Banking Division, Credit Suisse
More informationONEDIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK
ONEDIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK Definition 1. A random walk on the integers with step distribution F and initial state x is a sequence S n of random variables whose increments are independent,
More informationIntellectual Need and ProblemFree Activity in the Mathematics Classroom
Intellectual Need 1 Intellectual Need and ProblemFree Activity in the Mathematics Classroom Evan Fuller, Jeffrey M. Rabin, Guershon Harel University of California, San Diego Correspondence concerning
More informationSemiSimple Lie Algebras and Their Representations
i SemiSimple Lie Algebras and Their Representations Robert N. Cahn Lawrence Berkeley Laboratory University of California Berkeley, California 1984 THE BENJAMIN/CUMMINGS PUBLISHING COMPANY Advanced Book
More informationIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289. Compressed Sensing. David L. Donoho, Member, IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289 Compressed Sensing David L. Donoho, Member, IEEE Abstract Suppose is an unknown vector in (a digital image or signal); we plan to
More informationMesh Parameterization Methods and Their Applications
Foundations and Trends R in Computer Graphics and Vision Vol. 2, No 2 (2006) 105 171 c 2006 A. Sheffer, E. Praun and K. Rose DOI: 10.1561/0600000011 Mesh Parameterization Methods and Their Applications
More informationHighRate Codes That Are Linear in Space and Time
1804 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 7, JULY 2002 HighRate Codes That Are Linear in Space and Time Babak Hassibi and Bertrand M Hochwald Abstract Multipleantenna systems that operate
More informationBootstrap Methods and Their Application
Bootstrap Methods and Their Application cac Davison and DV Hinkley Contents Preface i 1 Introduction 1 2 The Basic Bootstraps 11 21 Introduction 11 22 Parametric Simulation 15 23 Nonparametric Simulation
More informationScalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights
Seventh IEEE International Conference on Data Mining Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights Robert M. Bell and Yehuda Koren AT&T Labs Research 180 Park
More informationFast Solution of l 1 norm Minimization Problems When the Solution May be Sparse
Fast Solution of l 1 norm Minimization Problems When the Solution May be Sparse David L. Donoho and Yaakov Tsaig October 6 Abstract The minimum l 1 norm solution to an underdetermined system of linear
More informationLearning Deep Architectures for AI. Contents
Foundations and Trends R in Machine Learning Vol. 2, No. 1 (2009) 1 127 c 2009 Y. Bengio DOI: 10.1561/2200000006 Learning Deep Architectures for AI By Yoshua Bengio Contents 1 Introduction 2 1.1 How do
More informationBootstrap con"dence intervals: when, which, what? A practical guide for medical statisticians
STATISTICS IN MEDICINE Statist. Med. 2000; 19:1141}1164 Bootstrap con"dence intervals: when, which, what? A practical guide for medical statisticians James Carpenter* and John Bithell Medical Statistics
More informationDiscreteTime Signals and Systems
2 DiscreteTime Signals and Systems 2.0 INTRODUCTION The term signal is generally applied to something that conveys information. Signals may, for example, convey information about the state or behavior
More informationhow to use dual base log log slide rules
how to use dual base log log slide rules by Professor Maurice L. Hartung The University of Chicago Pickett The World s Most Accurate Slide Rules Pickett, Inc. Pickett Square Santa Barbara, California 93102
More informationCommunication Theory of Secrecy Systems
Communication Theory of Secrecy Systems By C. E. SHANNON 1 INTRODUCTION AND SUMMARY The problems of cryptography and secrecy systems furnish an interesting application of communication theory 1. In this
More information