Numerical Solutions to Differential Equations

Transcription

1 Numerical Solutions to Differential Equations Lecture Notes #7 Linear Multistep Methods Peter Blomgren, Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research Center San Diego State University San Diego, CA Spring 25 Peter Blomgren, Linear Multistep Methods (/3)

2 Outline Introduction and Recap Linear Multistep Methods, Historical Overview Zero-Stability 2 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule 3 Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

3 Linear Multistep Methods, Historical Overview Zero-Stability Quick Review, Higher Order Methods for y (t) = f(t,y) Taylor When the Taylor series for f(t,y) is available, we can use the expansion to build higher accurate methods. RK If the Taylor series is not available (or too expensive), but f(t,y) easily can be computed, then RK-methods are a good option. RK-methods compute / sample / measure f(t,y) in a neighborhood of the solution curve and use those a combination of the values to determine the final step from (t n,y n ) to (t n+,y n+ ). LMM If the Taylor series is not available, and f(t,y) is expensive to compute (could be a lab experiment?), then LMMs are a good idea. Only one new evaluation of f(t,y) needed per iteration. LMMs use more of the history {(t n k,y n k ); k =,...,s} to build up the step. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

4 Linear Multistep Methods, Historical Overview Zero-Stability Chronology Methods 883 Adams and Bashforth introduce the idea of improving the Euler method by letting the solution depend on a longer history of computed values. (Now known as Adams-Bashforth schemes) 925 Nyström proposes another class of LMM methods, ρ(ζ) = ζ k ζ k 2, explicit. 926 Moulton developed the implicit version of Adams and Bashforth s idea. (Now known as Adams-Moulton schemes) 952 Curtiss and Hirschenfelder Backward difference methods. 953 Milne s methods, ρ(ζ) = ζ k ζ k 2, implicit. Modern Theory 956 Dahlquist 962 Henrici Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (4/3)

5 Linear Multistep Methods, Historical Overview Zero-Stability Introducing Zero-Stability (Review) Consider the LMM applied to a noise-free problem: k k α j y n+j = h β j f n+j j= j= j= y µ = η µ (h), µ =,,...,k and the same LMM applied to a slightly perturbed system k k α j y n+j = h β j f n+j +δ n+k j= y µ = η µ (h)+δ µ, µ =,,...,k Perturbations are typically due to discretization and round-off. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (5/3)

6 Linear Multistep Methods, Historical Overview Zero-Stability Defining Zero-Stability (Review) Definition (Zero-stability) Let {δ n,n =,,...,N} and {δ n,n =,,...,N} be any two perturbations of the LMM, and let {y n,n =,,...,N} and {y n,n =,,...,N} be the resulting solutions. If there exists constants S and h such that, for all h (,h ], whenever y n y n Sǫ, n N δ n δ n ǫ, n N the method is said to be zero stable. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (6/3)

7 Interpreting Zero-Stability Linear Multistep Methods, Historical Overview Zero-Stability (Formalized) Applying the LMM to z n = y n yn, δn = δ n δn gives: k α j z n+j = δ n+k j= z µ = δ µ, µ =,,...,k Interpretation That is, zero-stability guarantees that a zero-forced system (with zero starting-values) produces errors bounded by the round-off noise. In infinite precision, the solution stays at zero. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (7/3)

8 Linear Multistep Methods, Historical Overview Zero-Stability A Simple Criterion for Zero-Stability (Review) If the roots of the characteristic polynomial k α j y n+j =, ρ(ζ) = j= satisfies the root criterion then the method is zero-stable. Theorem (Convergence) r j, j =,2,...,k The method is convergent if and only if it is consistent and zero-stable. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (8/3)

9 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule The First Dahlquist Barrier, I/III Theorem (Germund Dahlquist, 956) No zero-stable s-step method can have order exceeding (s + ) when s is odd, and (s +2) when s is even. Definition A zero-stable s-step method is said to be optimal if it is of order (s +2). Statement Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (9/3)

10 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule The First Dahlquist Barrier, I/III Theorem (Germund Dahlquist, 956) No zero-stable s-step method can have order exceeding (s + ) when s is odd, and (s +2) when s is even. Definition A zero-stable s-step method is said to be optimal if it is of order (s +2). Observation Simpson s rule is optimal (to be shown...) y n+2 y n = h ] [f n+2 +4f n+ +f n 3 Note: Zero-stability does not give us the whole picture; see absolute stability... (coming right up!) Statement Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (9/3)

11 The First Dahlquist Barrier, II/III The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Newton-Cotes Errors The first Dahlquist barrier reminds us of something from Math 54: Theorem (Errors for Newton-Cotes Integration Formulas) Suppose that n i= a if(x i ) denotes the (n+) point closed Newton-Cotes formula with x = a, x n = b, and h = (b a)/n. Then there exists ξ (a,b) for which b a f(x)dx = n i= a i f(x i )+ hn+3 f (n+2) (ξ) (n+2)! n t 2 (t ) (t n)dt, if n is even and f C n+2 [a,b], and b a f(x)dx = n i= a i f(x i )+ hn+2 f (n+) (ξ) (n+)! n t(t ) (t n)dt, if n is odd and f C n+ [a,b]. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (/3)

12 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule The First Dahlquist Barrier, III/III Comments For the Newton-Cotes formulas: when n is an even integer, the degree of precision (higher order polynomial for which the formula is exact) is (n+). When n is odd, the degree of precision is only n. For zero-stable s-step LMMs: when s is even, the order is at most (s +2); when s is odd, the order is at most (s +). Coincidence? Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (/3)

13 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule The First Dahlquist Barrier, III/III Comments For the Newton-Cotes formulas: when n is an even integer, the degree of precision (higher order polynomial for which the formula is exact) is (n+). When n is odd, the degree of precision is only n. For zero-stable s-step LMMs: when s is even, the order is at most (s +2); when s is odd, the order is at most (s +). Coincidence? Unlikely! The LMMs get the next y k+ by integrating over the solution history; and the Newton-Cotes formulas give the (numerical) integral over an interval. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (/3)

14 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Simpson s Rule, y n+ y n = h 3 [f n+ +4f n +f n ] For notational convenience, the points have been re-numbered (index lowered by one), and we expand around the center point (t n,y n ): y n+ y n +hy n + h2 2 y n + h3 6 y n + h4 24 y(4) n + h5 2 y(5) n +O(h 6 ) y n y n hy n + h2 2 y n h3 6 y n + h4 24 y(4) n h5 2 y(5) n +O(h 6 ) LHS 2hy n + h3 3 y n + h5 6 y(5) n +O(h 7 ) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

15 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Simpson s Rule, y n+ y n = h 3 [f n+ +4f n +f n ] For notational convenience, the points have been re-numbered (index lowered by one), and we expand around the center point (t n,y n ): y n+ y n +hy n + h2 2 y n + h3 6 y n + h4 24 y(4) n + h5 2 y(5) n +O(h 6 ) y n y n hy n + h2 2 y n h3 6 y n + h4 24 y(4) n h5 2 y(5) n +O(h 6 ) LHS 2hy n + h3 3 y n + h5 6 y(5) n +O(h 7 ) f n f n hf n + h2 2 f n h3 6 f n + h4 24 f (4) n h5 2 f (5) n +O(h 6 ) 4f n 4f n f n+ f n +hf n + h2 n + h3 n + h4 [ ] RHS h 3 6f n +h 2 f n + h4 2 f n (4) +O(h 6 ) 2 f 6 f 24 f n (4) + h5 2 f (5) n +O(h 6 ) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

16 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Simpson s Rule, y n+ y n = h 3 [f n+ +4f n +f n ], II LHS 2hy n + h3 3 y n + h5 6 y(5) n +O(h 7 ) [ ] RHS h 3 6f n +h 2 f n + h4 2 f n (4) +O(h 6 ) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

17 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Simpson s Rule, y n+ y n = h 3 [f n+ +4f n +f n ], II LHS 2hy n + h3 3 y n + h5 6 y(5) n +O(h 7 ) [ ] RHS h 3 6f n +h 2 f n + h4 2 f n (4) +O(h 6 ) Use the equation y (t) = f(t,y) y (k+) (t) = f (k) (t,y): LHS 2hf n + h3 3 f n + h5 6 f n (4) +O(h 7 ) RHS 2hf n + h3 3 f n + h5 24 f n (4) +O(h 7 ) LHS RHS h = h 4[ 6 ] (4) 24 f n +O(h 6 ) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

18 The First Dahlquist Barrier Example: 2-step, Order 4 Simpson s Rule Simpson s Rule, y n+ y n = h 3 [f n+ +4f n +f n ], II LHS 2hy n + h3 3 y n + h5 6 y(5) n +O(h 7 ) [ ] RHS h 3 6f n +h 2 f n + h4 2 f n (4) +O(h 6 ) Use the equation y (t) = f(t,y) y (k+) (t) = f (k) (t,y): LHS 2hf n + h3 3 f n + h5 6 f n (4) +O(h 7 ) RHS 2hf n + h3 3 f n + h5 24 f n (4) +O(h 7 ) LHS RHS h = h 4[ 6 ] (4) 24 f n +O(h 6 ) Simpson s Rule Local Truncation Error LTE Simpson (h) = O ( h 4) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

19 Linear for LMMs As we did for RK-methods we apply our LMMs to the problem y (t) = λy(t), Re(λ) and search for the region ĥ = (hλ) where the LMM does not grow exponentially. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (4/3)

20 Linear for LMMs As we did for RK-methods we apply our LMMs to the problem y (t) = λy(t), Re(λ) and search for the region ĥ = (hλ) where the LMM does not grow exponentially. We get... k k k α j y n+j = h β j f n+j = h β j λy n+j j= j= j= Thus... k [α j hβ j λ]y n+j = j= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (4/3)

21 Linear for LMMs, II We have k [α j hβ j λ]y n+j = j= A general solution of this difference equation is y n = r r n where r is a root of the characteristic polynomial = k [α j hβ j λ]r j = ρ(r) ĥσ(r) = π(r,ĥ) j= π(r,ĥ) is called the stability polynomial. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (5/3)

22 Linear : Absolute Stability Definition (Absolute Stability) A linear multistep method is said to be absolutely stable for a given ĥ, if for that ĥ all the roots of the stability polynomial π(r,ĥ) satisfy r j <, j =,2,...,s, and to be absolutely unstable for that ĥ otherwise. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (6/3)

23 Linear : Absolute Stability Definition (Absolute Stability) A linear multistep method is said to be absolutely stable for a given ĥ, if for that ĥ all the roots of the stability polynomial π(r,ĥ) satisfy r j <, j =,2,...,s, and to be absolutely unstable for that ĥ otherwise. Definition (Region of Absolute Stability) The LMM is said to have the region of absolute stability R A, where R A is a region in the complex ĥ-plane, if it is absolutely stable for all ĥ R A. The intersection of R A with the real axis is called the interval of absolute stability. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (6/3)

24 The Boundary Locus Method The boundary of R A, denoted R A is given by the points where one of the roots of π(r,ĥ) is eiθ. R A is ĥ such that Solving for ĥ gives π(e iθ,ĥ) = ρ(eiθ ) ĥσ(eiθ ) =, θ [,2π) Method: Boundary Locus ĥ(θ) = ρ(eiθ ) σ(e iθ ), θ [,2π) Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (7/3)

25 The Region of Absolute Stability for Simpson s Method Consider Simpson s Rule, and its characteristic polynomials y n+2 y n = h ] [f n+2 +4f n+ +f n 3 The R A is given by ρ(ζ) = ζ 2, σ(ζ) = 3 [ ζ 2 +4ζ + ] e 2iθ ĥ(θ) = 3 e 2iθ +4e iθ + = 3 eiθ e iθ 6i sinθ e iθ = +4+e iθ 4+2cosθ = 3i sinθ 2+cosθ Hence R A is the segment [ i 3,i 3] of the imaginary axis. Simpson s Rule has a zero-area region of absolute stability (Bummer). Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (8/3)

26 Optimal Methods are not so Optimal after all... All optimal methods have regions of absolute stability which are either empty, or essentially useless they do not contain the negative real axis in the neighborhood of the origin. By squeezing out the maximum possible order, subject to zero-stability, the region of absolute stability get squeezed flat. Optimal methods are essentially useless. Peter Blomgren, Linear Multistep Methods (9/3)

27 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

28 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

29 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

30 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

31 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

32 Stability Regions for Adams-Bashforth Methods Adams-Bashforth Methods Stability Regions 2 n= n=2 n=3 n=4 n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

33 Stability Regions for Adams-Bashforth Methods r ν > count 2 Adams Bashforth, order 2 Adams Bashforth, order 2 2 Adams Bashforth, order Adams Bashforth, order Adams Bashforth, order Adams Bashforth, order Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (2/3)

34 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions n= The Exterior of the circle! Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

35 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions n= n= The left half plane! Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

36 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions 4 3 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

37 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions 4 3 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

38 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions 4 3 n= n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

39 Stability Regions for Adams-Moulton Methods 5 Adams-Moulton Methods Stability Regions n= n=2 n=3 n=4 n= Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (22/3)

40 Absolute Stability Matters! So far we have seen (only) two methods which produce bounded solutions to the ODE y (t) = λy(t) for all λ : Re(λ) < : Implicit Euler (Adams-Moulton, n = ) y n+ = y n +hf n+ Trapezoidal Rule (Adams-Moulton, n = 2) y n+ = y n + h ] [f n+ +f n 2 The size of the stability region located in the left half plane tends to shrink as we require higher order accuracy requiring a smaller stepsize h. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (23/3)

41 Can we find high order methods with large stability regions?!? Yes! The class of (BDF) defined by k α j y n+j = hβ k f n+k j= have large regions of absolute stability. Note that the right-hand side is simple, but the left-hand side is more complicated (the opposite of Adams-methods). Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (24/3)

42 Deriving BDF I/IV The kth order BDF is derived by constructing the polynomial interpolant through the points (t n+,y n+ ), (t n,y n ),..., (t n k+,y n k+ ), i.e. (after re-numbering the points:,,...,k) P k (t) = k y n+m L k,m (t), where L k,m (t) = m= k l=,l m t t l t m t l and then computing the derivative of this polynomial at the point corresponding to t n+ and setting it equal to f n+. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (25/3)

43 Deriving BDF II/IV Newton s Backward Difference Formula (Math 54) comes in handy. We can write the interpolating polynomial P k (t n+ +sh) = y n+ + k j= ( ) j ( s j ) j y n+ where Newton s divided differences are ] y n+ = [y n+ y n, 2 y n+ = ] [ y n+ y n,... 2 Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (26/3)

44 Deriving BDF III/IV The binomial coefficient is given by ( ) s s( s ) ( s j +) = j j! In order to compute P k (t n+) we need to compute ( ) d s ds j s= Massive application of the product rule gives us ( ) d s = ( ) j(j )! = ( )j ds j j! j That is hp k (t n+) = s= k ( ) 2j j y n+ = j j= = ( ) js(s +) (s +j ) j! k j= j j y n+ Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (27/3)

45 Deriving BDF IV/IV We now have k j= j j y n+ = hf n+ Making sure that the coefficient for y n+ is : k j= j k j= k j j y n+ = h j j= f n+ Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (28/3)

46 BDFs, k =,2,...,6 k BDF LTE y n+ y n = hf n+ 2 h 2 y n+ 4 3 y n + 3 y n = 2 3 hf n+ 2 9 h2 3 y n+ 8 y n + 9 y n 2 y n 2 = 6 hf n h3 4 y n y n y n 6 25 y n y n 3 = 2 25 hf n h4 5 y n y n y n 2 37 y n y n y 6 n 4 = 37 hf n+ 37 h5 6 y n y n y n 4 47 y n y n y n y 6 n 5 = 47 hf n h6 These are all zero-stable. BDFs for k 7 are not zero-stable. Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (29/3)

47 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=2 The Exterior(s) Peter Blomgren, Linear Multistep Methods (3/3)

48 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=3 The Exterior(s) Peter Blomgren, Linear Multistep Methods (3/3)

49 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=4 The Exterior(s) / Part of Left Half Plane Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

50 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=5 The Exterior(s) / Part of Left Half Plane Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

51 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=6 The Exterior(s) / Part of Left Half Plane Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)

52 Stability Regions for BDF Methods BDF Methods Stability Regions k= k=2 k=3 k=4 k=5 k=6 The Exterior(s) / Part of Left Half Plane Peter Blomgren, blomgren.peter@gmail.com Linear Multistep Methods (3/3)