G.A. Pavliotis. Department of Mathematics. Imperial College London

Transcription

1 EE1 MATHEMATICS NUMERICAL METHODS G.A. Pavliotis Department of Mathematics Imperial College London 1. Numerical solution of nonlinear equations (iterative processes). 2. Numerical evaluation of integrals. 3. Numerical solution of ODEs.

2 Numerical solution of nonlinear equations We want to develop numerical methods for solving nonlinear equations of the form Examples: (i) f(x) = x 2 A = 0, A > 0 ( x = ± A). (ii) f(x) = x 3 3x = 0. (iii) f(x) = 1 x A = 0 ( x = 1 A ). (iv) f(x) = sinx 1 x = 0. f(x) = 0. (1)

3 Numerical solution of nonlinear equations Idea: Approximate Eqn. (1) by an iterative process (recursion) x n+1 = F(x n ), x 1 is given, (2) such that lim x n = X, with f(x) = 0. n Easy to implement (1) on a computer. Goal: find a systematic way for obtaining iterative process of the form (2) for solving equations of the form (1).

4 Numerical solution of nonlinear equations Consider Example (i), f(x) = x 2 A = 0, A > 0. x 2 A = 0 2x 2 x 2 = A 2x 2 = x 2 + A x = 1 ( x + A ). 2 x Use the following iterative scheme: x n+1 = 1 ) (x n + Axn. (3) 2 Choose x 1 (initial guess), use (3) to calculate approximately the square root of A. Let A = 25 X = ±5.

5 Numerical solution of nonlinear equations Choose x 1 = 1: x 1 = 1 x 2 = 13.0 x 3 = x 4 = x 5 = i.e. the iterative process converges to X = 5. Take now x 1 = 1. x 1 = 1 x 2 = 13.0 x 3 = x 4 = x 5 = i.e. the iterative process converges to X = 5. Choose x 1 = 8: x 1 = 8 x 2 = x 3 = x 4 = The iteration process converges faster to the root of the equation.

6 Numerical solution of nonlinear equations When we have an iterative procedure of the form (2) which we use to calculate the roots of (1) we want to address the following issues. 1. Does the iterative process converge, i.e. is there a limit of x n as n? 2. Does the iterative process converge to the right value (i.e. one of the roots of the equation F(x) = 0)? 3. How does its accuracy increase with n (i.e. is it practical)? 4. How does the choice of x 1 affect the convergence of the method and the value to which it converges (in the case when the equation has more than one roots)? A iterative process of the form (2) for solving an equation of the form (1) is called a fixed point method.

7 Numerical solution of nonlinear equations Consider example (ii) x 3 3x = 0: x 3 3x = 0 x 3 = 3x 2 2 x = 3 2 x 2. Use the iterative process x n+1 = 3 2. x 2 n If this method converges, to which of the 3 roots of the equation does it converge?

8 Numerical solution of nonlinear equations If an iterative process is to be convergent, then we have lim x n+1 x n = 0, n i.e. the distance between two subsequent elements of the sequence {x n } n=1 generated by (2) becomes smaller as n increases. We expect that the starting value x 1 affects the eventual convergence of the iterative process to a limit. It might be that the iterative process that we have chosen converges only for certain values of x 1. If we sketch the curves y = x, y = F(x), their intersection satisfies x = F(x). Start with x 1. The iterates generated by (2) should give x = X.

9 Numerical solution of nonlinear equations Calculation of A. Consider the iterative process x n+1 = 1 2 ) (x n + Axn, which approximates A. Assume 0 < x 1 < A. Then (x 1 + Ax1 ) x 2 = 1 2 > 1 ( ) A A + 2 A = A x 2 > A.

10 Numerical solution of nonlinear equations Now: (x 2 + Ax2 ) x 3 = 1 2 < 1 2 (x 2 + x 2 ) Similarly, x n+1 < x n, n = 2,.... = x 2 x 3 < x 2. Assume x 1 > A. Then x n+1 < x n, n = 1,.... This is an indication that the method converges. Furthermore: x n+1 A = 1 ) (x n + Axn A 2 1 ( ) = x 2 n + A 2x n A 2x n 1 ( = x n 2 A). 2x n

11 Numerical solution of nonlinear equations Hence, when x n A the error is essentially squared after each iteration (quadratic rate of convergence). Questions: what happens when x 1 < 0? What happens when A < 0?

12 Numerical solution of nonlinear equations

13 Numerical solution of nonlinear equations Calculation of 1 A. Consider the iterative process x n+1 = x n (2 Ax n ), which approximates 1 A : A 1 x = 0 1 = Ax 1 = 2 Ax x = x(2 Ax). The method can converge to either 0 or 1 A : 0 < x 1 < 2 A lim n x n = 1 A. x 1 = 0 or x 1 = 2 A x n = 0, n = 1, 2,... x 1 < 0 or x 1 > 2 A lim x n =. n

14 Numerical solution of nonlinear equations Summary An iterative process can converge or diverge, depending on the initial choice x 1. The root to which the iterative process converges depends on the initial choice x 1. The convergence can be fast (quadratic).

15 How to choose the iterative process for solving f(x) = 0. A given equation f(x) = 0 may often be written in several different ways as x = F(x). Example: f(x) = x 2 6x + 2 = 0 with roots X 1,2 = 3 ± 7. It can be written as x = F(x) in many different ways: 1. x = x2 2 2x 6 the iterative process is x n+1 = x2 2 2x n 6 : x 2 6x+2 = 0 2x 2 6x+2 = x 2 2x 2 6x = x 2 2 x = x2 2 2x x = 6 2 x the iterative process is x n+1 = 6 2 x n : x 2 6x + 2 = 0 x 2 = 6x 2 x = 6 2 x. 3. x = 1 6 x the iterative process is x n+1 = 1 6 x2 n : 6x = x x = 1 6 x

16 How to choose the iterative process for solving f(x) = 0. The resulting different iterative processes x n+1 = F(x n ) have, in general, different convergence properties. We want to choose the iterative process that has the best convergence properties. This means that not only does it converge to the roots of the equation f(x) = 0, but also that it converges sufficiently fast. We want to develop criteria that enable us to choose the best iterative process for solving a nonlinear equation of the form f(x) = 0.

17 How to choose the iterative process for solving f(x) = 0. Suppose that x n+1 = F(x n ) with lim n x n = X and f(x) = 0 X = F(X). Let us write i.e. X = x n + ǫ n, n = 1, 2,... X = approximation + error. We want to calculate ǫ n+1 as a function of ǫ n. Assume that ǫ n is small so that we can use the Taylor series expansion: x n+1 = F(x n ) F(X) ǫ n F (X) ǫ2 nf (X). But x n+1 = X + ǫ n+1 ǫ n+1 = X x n+1 X F(X) + ǫ n F (X) 1 2 ǫ2 nf (X) = ǫ n F (X) 1 2 ǫ2 nf (X). The size of ǫ n+1, relative to the size of ǫ n, depends on F (X).

18 How to choose the iterative process for solving f(x) = 0. Indeed: ǫ n+1 ǫ n = F (X) + O(ǫ n ). (4) DEFINITION 1 Assume that F (X) 0. Then the iterative process x n+1 = F(x n ) is called a first order process. The convergence of a first order process is guranteed when F (X) < 1. (5) If we combine eqns (4) and (5) we conclude that ǫ n+1 ǫ n < 1, provided that ǫ n is sufficiently small. On the other hand, it is clear that the sequence ǫ n diverges when F (X) > 1. The case F (X) = 1 requires further study.

19 How to choose the iterative process for solving f(x) = 0. We can also define second order processes: DEFINITION 2 Assume that F (X) = 0 and that F (X) 0. Then the iterative process x n+1 = F(x n ) is called a second order process. For a second order process we have that ǫ n+1 = 1 2 ǫ2 nf (X) + O(ǫ 3 n) ǫ n+1 = Cǫ 2 n and the convergence is much faster (quadratic as opposed to linear). It is to our advantage to use a second order iterative process to solve the equation f(x) = 0.

20 How to choose the iterative process for solving f(x) = 0. Examples ( ) 1. The square root formula F(x) = 1 2 x + A x. F (x) = 1 (1 Ax ) 2 2 F ( A) = 1 2 ( 1 A ) A = 0 We have to calculate the second derivative at the root X = A. F (x) = A x 3 F ( A) = 1 A. Second Order Process. The convergence is quadratic.

21 How to choose the iterative process for solving f(x) = 0. Examples 2. The reciprocal formula F(x) = x(2 Ax). F (x) = (2 Ax) + x( A) = 2 2Ax ( ) ( ) 1 1 F = 2 2A = 0 A A We have to calculate the second derivative at the root X = 1 A. ( 1 F (x) = 2A F A Second Order Process. ) = 2A. The convergence is quadratic.

22 How to choose the iterative process for solving f(x) = 0. Examples 3. The quadratic equation f(x) = x 2 6x + 2 = 0. The two roots of this equation are R 1 = 3 + 7, R 2 = 3 7. Consider the following 4 different iterative processes. (a) x n+1 = 6 2 x n. (b) x n+1 = 1 6 x2 n (c) x n+1 = 6x n 2. (d) x n+1 = x2 2 2x n 6.

23 How to choose the iterative process for solving f(x) = 0. The performance of the the four different iterative processes is summarized in the following table. Iterative process R 1 attainable R 2 attainable order of process x n+1 = 6 2 x n. NO YES 1 x n+1 = 1 6 x2 n YES NO 1 x n+1 = 6x n 2. NO YES 1 x n+1 = x2 6 2x n 6. YES YES 2 The fourth iterative process performs better: both roots are attainable and it converges faster. Q: Is there a systematic method for finding the best method for a given nonlinear equation f(x) = 0? A: Yes, the Newton Raphson Method.

24 The Newton Raphson Method Consider the equation f(x) = 0 and let X be the root of this equation: f(x) = 0. Let x 1 be our initial approximation and write X = x 1 + ǫ 1. Use the Taylor series expansion to obtain 0 = f(x) = f(x 1 + ǫ 1 ) = f(x 1 ) + ǫ 1 f (x 1 ) +... Assuming that f (x 1 ) 0, we solve this equation for ǫ 1 to obtain ǫ 1 f(x 1) f (x 1 ). Hence, a better approximation to X than x 1 would appear to be x 2 = x 1 f(x 1) f (x 1 ).

25 The Newton Raphson Method We can repeat this procedure to obtain the iterative process x n+1 = x n f(x n) f (x n ) =: F(x n). (6) This is the Newton Raphson iterative process. The Newton Raphson process converges to a particular root of f(x) = 0 if x 1 is suitably chosen, assuming that all the roots are attainable. The process is usually second order convergent if X is a simple root:

26 The Newton Raphson Method F(x) = x f(x) f (x) F (x) = 1 (f (x)) 2 f(x)f (x) (f (x)) 2 = (f (x)) 2 (f (x)) 2 + f(x)f (x) (f (x)) 2 F (X) = = f(x)f (x) (f (x)) 2 f(x)f (X) (f (X)) 2 = 0, since f(x) = 0. If X is not a simple root then the Newton Raphson process is usually first order, but it still converges.

27 The Newton Raphson Method: Examples 1. The square root formula: f(x) = x 2 A = 0 f (x) = 2x x n+1 = x n f(x n) f (x n ) = x n x2 n A 2x n = x n x n 2 + A 2x n = 1 ) (x n + Axn. 2

28 The Newton Raphson Method: Examples 2. The reciprocal formula: f(x) = 1 x A = 0 f (x) = 1 x 2 x n+1 = x n f(x n) f (x n ) = x n = x n + x 2 n ( ) 1 A x n 1 x n A 1 x 2 n = x n + x n Ax 2 n = x n (2 Ax n ).

29 The Newton Raphson Method: Examples 3. A quadratic equation: f(x) = x 2 6x + 2 = 0 f (x) = 2x 6 x n+1 = x n f(x n) f (x n ) = x n x2 n 6x n + 2 2x n 6 = x n(2x n 6) x 2 n + 6x n 2 2x n 6 = 2x2 n 6x n x 2 n + 6x n 2 2x n 6 = x2 n 2 2x n 6. The Newton Raphson process converges to R 1 or R 2 depending on the initial guess: x 1 < 3 x n R 1, R 1 = 3 7. x 1 > 3 x n R 2, R 2 = x 1 = 3 x 2 = +, the method diverges.

30 The Newton Raphson Method: Examples 4. The equation sinx 1 x = 0. f(x) = sinx 1 x = 0 f (x) = cosx + 1 x 2 x n+1 = x n f(x n) f (x n ) = x n sinx n 1 cosx n + 1 = x n x 2 n sinx n 1 x n x 2 n cos x n + 1 = x n x2 n sinx n x n x 2 n cosx n + 1. The equation sinx 1/x = 0 has infinitely many solutions. Let us find the first positive solution R = Start with x 1 = 0.5. x n x 2 n x 1 = 0.5 x 2 = x 3 = x 4 = x 5 =

31 The Newton Raphson Method: Examples sin(x) 1/x a. f 1 (x) = sin(x), f 2 (x) = 1/x. b. f(x) = sin(x) 1/x.

32 The Newton Raphson Method: Examples sin(x) 1/x f 1 (x) = sin(x), f 2 (x) = 1/x.

33 The Newton Raphson Method: Examples We want to show that a solution to the equation f(x) = sinx 1 x = 0 exists in the interval ( π 4, π 2 ). We have that f (x) = cos x + 1 [ π x 2 > 0 for x 4, π ]. 2 Consequently, the function f(x) is strictly increasing in the interval [ π 4, π 2 ]. Furthermore, f(π/4) = sin(π/4) 1 π/4 = π < 0 and f(π/2) = sin(π/2) 1 π/2 = 1 2 π > 0. Since f(x) is continuous and strictly increasing, there exists an R ( π 4, π 2 ) such that f(r) = 0. This shows that there exists a root of the equation f(x) = 0 in the interval ( π 4, π 2 ).

34 The Newton Raphson Method: Examples We want to show that the equation in the interval ( π 2, 2π 3 ). f(θ) = sin(θ) θ cos(θ) 1 2 π = 0 f (θ) = cosθ cosθ + θ sinθ = θ sinθ > 0 for θ [ π 2, 2π 3 Consequently, the function f(x) is strictly increasing in the interval [ π 2, 3π 2 ]. Furthermore, f( π 2 ) = sin(π 2 ) π 2 cos(π 2 ) π 2 = 1 π 2 < 0. ]. and f( 2π 3 ) = sin(2π 3 ) 2π 3 cos(2π 3 ) 2π 3 = > 0. Since f(x) is continuous and strictly increasing, there exists an R ( π 2, 2π 3 ) such that f(r) = 0. This shows that there exists a root of the equation f(x) = 0 in the interval ( π 2, 2π 3 ).

35 The Newton Raphson Method: Examples Now we find the Newton Raphson process for this equation. We have that The Newton Raphson process is f (θ) = θ sin(θ). θ n+1 = θ n sinθ n θ n cosθ n π 2 θ n sin θ n. The root of the equation f(θ) = sin(θ) θ cos(θ) 1 2π = 0 in the interval ( π 2, 2π 3 ) is R = We calculate it using the Newton-Raphson process with x 1 = π 2 : x 1 = π 2 x 2 = x 3 = x 4 =

36 The Newton Raphson Method: Examples a. θ [ π 2, 2π 3 ]. b. θ [0, 10]. f(θ) = sin(θ) θ cos(θ) 1 2 π.

37 Numerical solution of nonlinear equations Final Remarks One can also define higher order processes: F (X) = 0, F (X) = 0,...F (n) (X) 0. This is an nth order process. There are many other methods for solving nonlinear equations other than the Newton Raphson method: Bisection method, secant method etc. The Newton Raphson methods works also for equations on the complex plane: f(z) = 0, z C.