Lecture Notes for Numerical Methods (MA5625)

Transcription

1 Lecture Notes for Numerical Methods (MA565) Weiwei Sun Chapter One Numerical Solution of Nonlinear Equations 1.1 Introduction This chapter is devoted to the problem of locating roots of equations(or zeros of functions). The problem occurs frequently in scientific work. Problem: Find a root of the equation f(x) = 0. Example 1 : x sin x = 0. Geometrically, we know that there exist three roots, denoted by x, 0 and x. Mathematically, we cannot find the exact roots x and x. Numerically, we need to construct a sequence {x n } which satisfies x n x when n. Such a method is called an iterative method. There are many different ways to construct a sequence, such as for example 1, For a given x 0 x n+1 = sin x n, n = 0, 1,... {x n } We shall consider several standard procedures for constructing a sequence. For an iterative method, the following questions are important: (i) Under what conditions will the sequence {x n } converge to a solution (Convergent or divergent)? (ii) How quickly will the sequence {x n } converge? (Convergent rate.) (iii) How to choose the initial point x 0? 1. Bisection method Theorem 1.1 (in Calculus) : If f(x) is continuous on the interval [a, b] and then f(x) has at least a root (zero) in (a, b). f(a)f(b) < 0 (1.1) 1

2 Now we consider a function f(x) satisfying the above conditions. Let c = (a + b)/ and calculate f(c). If f(c) is small enough, stop and c is the numerical solution (root). Otherwise, if f(a)f(c) < 0, we use the algorithm on the interval [a, c]. If f(a)f(c) > 0 (i.e., f(b)f(c) < 0), we use the algorithm on the interval [c, b]. Procedure of Bisection algorithm: Input a, b, δ( or ɛ) and M u f(a) v f(b) e b a For k = 1, M, do e e/ c (a + b)/ w f(c) if e δ( or w ɛ), then stop if sign(w) sign(u), then b c v w else a c u w end if end Remarks * Since f(a)f(b) < 0, the algorithm is always convergent to a root. However, we do not know which root we will find if there are several roots in [a, b]. * How can we find the initial interval [a, b]? In practice, one often wants to find a root in a special area, i.e., the initial interval can be determined by some physical experience. If not, we also can test some points to find the initial interval. Convergence and error analysis * Convergence Let a 0 = a and b 0 = b. By following the bisection algorithm, we obtain the sequences {a n } and {b n }, which satisfy a 0 a 1... b 1 b 0 i.e., the sequence {a n } is monotonously increasing and bounded above by b 0 and the sequence {b n } is monotonously decreasing and bounded below by a 0. Then both {a n } and {b n } are

3 convergent and b 1 a 1 = (b a)/ b a = (b 1 a 1 )/ = (b a)/4... b n a n = (b n 1 a n 1 )/ = (b a)/ n. Since f(a n )f(b n ) < 0, there is a root x (a n, b n ). Then * Convergence rate If we choose lim (b n a n ) = 0 lim a n = lim b n = x n n n c N = (b N + a N )/ as the numerical solution, i.e., the algorithm stops at n = N, then x c N (b N a N )/ = (b a)/ N+1 By the above formula, we can choose the number of iterations for your δ to satisfy x c N δ. It requires that b a N+1 δ N log ( ) b a 1 δ 1.3 Newton s method We consider the equation f(x) = 0. Let x be a root of f(x), i.e., f( x) = 0. Let x 0 be an initial guess and x = x 0 + h. Then By Taylor expansion, 0 = f( x) = f(x 0 + h) = f(x 0 ) + hf (x 0 ) + h f (x 0 ) +... If x 0 is a good initial approximation to x, then h = x x 0 is small. Approximately, let h satisfy f(x 0 ) + h f (x 0 ) = 0. Then h = f(x 0) f (x 0 ) is an approximation to h and a better approximation to x is given by x 1 = x 0 + h = x 0 f(x 0) f (x 0 ). 3

4 Newton s method (Newton s iteration): For a given x 0, gives a sequence {x n }. Procedure of Newton s iteration: x n+1 = x n f(xn) f (x n), n = 0, 1,,... Input x 0, M, δ( or ɛ) y f(x 0 ) if y < δ then stop Fork = 1, M, do x 1 x 0 y f (x 0 ) y f(x 1 ) if y < δ, ( or x 1 x 0 < ɛ), then stop x 0 x 1 end Example. Use Bisection and Newton methods to find the negative root of the equation f(x) = e x 1.5 tan 1 x = 0. Table 1. The comparison of Bisection and Newton methods Newton s method Bisection method with [-0, 0] No. of Iteration x f(x) c = x f(x) E E E E E E E E E E E E E-7 Graphic interpretation The equation of tangent line of f(x) at x = x n is When y = 0, y f(x n ) = f (x n )(x x n ). x n+1 = x n f(x n) f (x n ). Thus, the Newton method is obtained by local linearization. Convergence 4

5 Let e n = x n x. Then Since by Taylor expansion e n+1 = x n+1 x = x n f(x n) f (x n ) x = e n f(x n) f (x n ) = e nf (x n ) f(x n ) f (x n ) 0 = f( x) = f(x n e n ) = f(x n ) e n f (x n ) + e n f (ξ n ), e n+1 = 1 f (ξ n ) e n f (x n ) (1.) Theorem 1. Let f(x) C (i.e., f (x) is continuous) and x is a simple root of f(x). If the initial guess x 0 is good enough, the Newton s sequence {x n } is convergent and has a quadratic convergence rate, i.e., there exists a constant C such that e n+1 Ce n. Proof. Since x is a simple root, c := f ( x) 0. There exists δ > 0, such that f (x) > c, for x x < δ. Suppose that we start the Newton iteration with a point x 0 satisfying x 0 x δ. Then e 0 = x 0 x δ. Hence, we have where By the equation (1.), 1 f (ξ 0 ) f (x 0 ) M c M = max x x 1 f (x). x 1 x = e 1 e M 0 c e 0 δ M c = e 0 ρ < e 0 δ when δ is small enough, where ρ = δm /c < 1. This shows that the next point, x 1, also lies within δ units of x. Hence, the argument can be repeated, with the results e 1 ρ e 0 e ρ e 1 ρ e 0 e 3 ρ e ρ 3 e 0. 5

6 In general, we have e n ρ n e 0 Since 0 ρ < 1, we have lim n ρ n = 0 and so lim n e n = 0. Thus, lim x n = x. n Combining with equation (1.), the proof is completed. Remarks (i) Newton s method has a quadratic convergence. Therefore, Newton s method is faster than the bisection in general. (ii) A good initial guess is necessary for Newton s method (one can use bisection method to find a good initial guess and then, use Newton s method to get better solution). (iii) At each Newton s iteration, one needs to evaluate both f(x) and f (x). 1.4 Secant method (Quasi-Newton method) One of disadvantages of Newton method is the need of the evaluation of f (x). In many physical and engineering problems, it is difficult (or impossible) to provide the information of derivation of f(x). For example, f(x) is given only by discrete experimental data. We need an iterative method without the evaluation of f (x) and with good convergence rate. The basic idea of the secant method is to approximate the derivative f (x n ) in Newton method by those discrete data. We know that f f(x) f(x 0 ) (x 0 ) = x x0 lim x x 0 Then when x 1 x 0 is small enough, f (x 0 ) f(x 1) f(x 0 ) x x 0 (finite difference). Replacing the first-order derivative by the finite difference We have the algorithm f (x n ) = f(x n) f(x n 1 ) x n x n 1 x n+1 = x n f(x n)(x n x n 1 ) f(x n ) f(x n 1 ). Obviously, the algorithm needs two initial guesses x 0 and x 1. Graphic interpretation: replacing the tangent line with a line which passing through the points (x 0, f(x 0 )) and (x 1, f(x 1 )). 6

7 Procedure Example 3. Input x 0, x 1, M, δ u 0 f(x 0 ) u 1 f(x 1 ) Fork =, M, do x = x 1 u 1 (x 1 x 0 )/(u 1 u 0 ) if x x 1 δ, then stop u 0 u 1 x 0 x 1 u 1 f(x ) x 1 x end Let f(x) = e x 1.5 tan 1 x Table 1. The comparison of numerical results Newton s method Bisection method Secant method with [-0, 0] (x 0 = 7.0, x 1 = 7.) n x f(x) c = x f(x) x f(x) E E E E E E E E E E E E E E E E E E E E-7 Convergence: From the definition of the secant method, we have, with e n = x n x, e n+1 = x n+1 x = f(x n)x n 1 f(x n 1 )x n f(x n ) f(x n 1 ) x = f(x n)e n 1 f(x n 1 )e n f(x n ) f(x n 1 ) and therefore, ( ) ( ) x n x n 1 f(xn )/e n f(x n 1 )/e n 1 e n+1 = e n e n 1 f(x n ) f(x n 1 ) x n x n 1 By Taylor s theorem, f(x n )/e n f(x n 1 )/e n 1 x n x n 1 = 7 ( ) f(x) (c 1 ) f (ξ) x x

8 and Hence, we have x n x n 1 f(x n ) f(x n 1 ) = 1 f (c ). where C is a constant. Let d = max{ e 0, e 1 }. Then and Let It is obvious that e n+1 C e n e n 1 (1.3) e C e 1 e 0 Cd, e 3 C e e 1 Cd 3, e 4 C e 3 e Cd 5, e n Cd sn. s n = s n 1 + s n. We can solve the above finite difference equation to get with s 0 = s 1 = 1, and therefore, s n = C 1 ( e n+1 Cd ( ) n + C ( 1 5 ) 1+ n+1 5 Ce 1+ 5 n. Remarks (i) Secant method has a superlinear convergence. (ii) Convergent only when the initial values are good enough. (iii) The method needs two initial points. (iv) At each iteration, one only needs to evaluate f(x). 1.5 The system of nonlinear equations A simple case is: f 1 (x, y) = 0 f (x, y) = 0 In general, we need to solve the system F (X) = 0 <===> ) n f 1 (x 1, x,..., x m ) = 0 f (x 1, x,..., x m ) = 0... f m (x 1, x,..., x m ) = 0 8

9 where X = (x 1, x,..., x m ) R m and F : R m R m. Question: Can we extend these methods in the last subsection to the system of nonlinear equations? Newton s iteration The Newton s iteration for a single equation is based on the Taylor s expansion of a function of single variable. To design a Newton s iteration for a system. We need the Taylor s expansion F (X) = F (X 0 ) + F (X 0 )(X X 0 ) +... Based on the idea of the Newton s iteration, the Newton s iteration solution satisfies i.e., where F (X 0 ) + F (X 0 )(X 1 X 0 ) = 0 X 1 = X 0 ( F (X 0 )) 1 F (X 0 ) F (X 0 ) = ( f i x j ) m i,j=1 is an m m matrix. The General Quasi-Newton iteration is defined by X 1 = X 0 BF (X 0 ) where B can be chosen in some different ways. 9