h = 1 compute f (1) using the sequence of approximation for the derivative: with h k = 10 k, k 1

Transcription

1 Numerical Differentiation First Derivative We assume tat we can compute a function f, but tat we ave no information about ow to compute f. We want ways of estimating f (x), given wat we know about f. Reminder: definition of differentiation: df dx = lim f(x + x) f(x) x 0 x For second derivatives, we ave te definition: d 2 f dx 2 = lim f (x + x) f (x) x 0 x We can use tis formula, by taking x equal to some small value, to get te following approximation, known as te Forward Difference (D + ()): f (x) D + () = f(x + ) f(x) Alternatively we could use te interval on te oter side of x, to get te Backward Difference (D ()) : f f(x) f(x ) (x) D () = A more symmetric form, te Central Difference (D 0 ()), uses intervals on eiter side of x: f f(x + ) f(x ) (x) D 0 () = All of tese give (different) approximations to f (x). Second Derivative Error Estimation in Differentiation I We sall see tat te error involved in using tese differences is a form of truncation error (R T ): = D + () f (x) R T = 1 (f(x + ) f(x)) f (x) Te simplest way is to get a symmetrical equation about x by using bot te forward and backward differences to estimate f (x + x) and f (x) respectively: f (x) D +() D () = f(x + ) 2f(x) + f(x ) 2 Using Taylor s Teorem: f(x + ) = f(x) + f (x) + f (x) 2 /2! + f (3) (x) 3 / + : R T = 1 (f (x) + f (x) 2 /2! + f (x) 3 / + ) f (x) = 1 f (x) + 1(f (x) 2 /2! + f (x) 3 / + )) f (x) = f (x)/2! + f (x) 2 / + Using te Mean Value Teorem, for some ξ witin of x: R T = f (ξ) 2 Error Estimation in Differentiation II Exercise: differentiation I We don t know te value of eiter f or ξ, but we can say tat te error is order : R T for D + () is O() so te error is proportional to te step size as one migt naively expect. For D () we get a similar result for te truncation error also O(). Limit of te Difference Quotient. Consider te function f(x) = e x. 1 compute f (1) using te sequence of approximation for te derivative: wit k = 10 k, k 1 D k = f(x + k) f(x) k 2 for wic value k do you ave te best precision (knowing e 1 = ). Wy?

2 Exercise: differentiation II Central Difference 1 xls/lect13.xls 2 Best precision at k = 8. Wen k is too small, f(1) and f(1 + k ) are very close togeter. Te difference f(1 + k ) f(1) can exibit te problem of loss of significance due to te substraction of quantities tat are nearly equal. we ave looked at approximating f (x) wit te backward D () and forward difference D + (). Now we just ceck out te approximation wit te central difference: f (x) D 0 () = f(x + ) f(x ) Ricardson extrapolation Error analysis of Central Difference I Error analysis of Central Difference II We consider te error in te Central Difference estimate (D 0 ()) of f (x): We apply Taylor s Teorem, D 0 () = f(x + ) = f(x) + f (x) + f (x) 2 2! f(x ) = f(x) f (x) + f (x) 2 f(x + ) f(x ) 2! + f (x) 3 f (x) 3 (A) (B) = 2f (x) + 2 f (x) 3 (A) (B) = f (x) + f (x) 2 + f (4) (x) 4 4! + f (4) (x) 4 4! + 2 f (5) (x) 5 5! + f (5) (x) 4 5! + (A) + (B) + + We see tat te difference can be written as or alternatively, as D 0 () = f (x) + f (x) 6 were be know ow to compute b 1, b 2, etc. 2 + f (4) (x) + 24 D 0 () = f (x) + b b We see tat te error R T = D 0 () f (x) is O( 2 ). Remark. Remember: for D and D +, te error is O(). Error analysis of Central Difference III Rounding Error in Difference Equations I Example. Let try again te example: f(x) = e x We evaluate f (1) = e wit for = 10 k, k 1. D 0 () = Numerical values: xls/lect13.xls f (x) = e x f(1 + ) f(1 ) Wen presenting te iterative tecniques for root-finding, we ignored rounding errors, and paid no attention to te potential error problems wit performing subtraction. Tis did not matter for suc tecniques because: 1 te tecniques are self-correcting, and tend to cancel out te accumulation of rounding errors 2 te iterative equation x n+1 = x n c n were c n is some form of correction factor as a subtraction wic is safe because we are subtracting a small quantity (c n) from a large one (e.g. for Newton-Rapson, c n = f(x) f (x) ).

3 Rounding Error in Difference Equations II Rounding Error in Difference Equations III However, wen using a difference equation like D 0 () = f(x + ) f(x ) we seek a situation were is small compared to everyting else, in order to get a good approximation to te derivative. Tis means tat x + and x are very similar in magnitude, and tis means tat for most f (well-beaved) tat f(x + ) will be very close to f(x ). So we ave te worst possible case for subtraction: te difference between two large quantities wose values are very similar. We cannot re-arrange te equation to get rid of te subtraction, as tis difference is inerent in wat it means to compute an approximation to a derivative (differentiation uses te concept of difference in a deeply intrinsic way). We see now tat te total error in using D 0 () to estimate f (x) as two components 1 te truncation error R T wic we ave already calculated, 2 and a function calculation error R XF wic we now examine. Wen calculating D 0 (), we are not using totally accurate computations of f, but instead we actually compute an approximation f, to get D 0 () = f(x + ) f(x ) We sall assume tat te error in computing f near to x is bounded in magnitude by ǫ: f(x) f(x) ǫ Rounding Error in Difference Equations IV Te calculation error is ten given as R XF = D 0 () D 0 () = f(x + ) f(x ) f(x + ) f(x ) = f(x + ) f(x ) (f(x + ) f(x )) = f(x + ) f(x + ) ( f(x ) f(x )) R XF f(x + ) f(x + ) + f(x ) f(x ) ǫ + ǫ ǫ So we see tat R XF is proportional to 1/, so as srinks, tis error grows, unlike R T wic srinks quadratically as does. Rounding Error in Difference Equations V We see tat te total error R is bounded by R T + R XF, wic expands out to R f (ξ) 2 + ǫ 6 So we see tat to minimise te overall error we need to find te value of = opt wic minimises te following expression: Unfortunately, we do not know f or ξ! f (ξ) 2 + ǫ 6 Many tecniques exist to get a good estimate of opt, most of wic estimate f numerically someow. Tese are complex and not discussed ere. Ricardson Extrapolation I Ricardson Extrapolation II Te trick is to compute D 0 () for 2 different values of, and combine te results in some appropriate manner, as guided by our knowledge of te error beaviour. In tis case we ave already establised tat D 0 () = f(x + ) f(x ) We now consider using twice te value of : D 0 () = We can subtract tese to get: f(x + ) f(x ) 4 = f (x) + b O( 4 ) = f (x) + b O( 4 ) D 0 () D 0 () = 3b O( 4 ) Te rigtand side of tis equation is simply D 0 () f (x), so we can substitute to get D 0 () D 0 () 3 Tis re-arranges (carefully) to obtain f (x) = D 0 () f (x) + O( 4 ) = D 0 () + D0() D0() 3 + O( 4 ) = 4D0() D0() 3 + O( 4 ) We divide across by 3 to get: D 0 () D 0 () 3 = b O( 4 )

4 Ricardson Extrapolation III Summary It is an estimate for f (x) wose truncation error is O( 4 ), and so is an improvement over D 0 used alone. Tis tecnique of using calculations wit different values to get a better estimate is known as Ricardson Extrapolation. Ricardson s Extrapolation. Suppose tat we ave te two approximations D 0 () and D 0 () for f (x), ten an improved approximation as te form: f (x) = 4D 0() D 0 () + O( 4 ) 3 Approximation for numerical differentiation: Approximation for f (x) Error Forward/backward difference D +, D O() Central difference D 0 O( 2 ) Ricardson Extrapolation O( 4 ) Considering te total error (approximation error + calculation error): R f (ξ) 2 + ǫ 6 remember tat sould not be cosen too small. Solving Differential Equations Numerically Working Example Definition. Te Initial value Problem deals wit finding te solution y(x) of y = f(x, y) wit te initial condition y(x 0 ) = y 0 It is a 1st order differential equations (D.E.s). Alternative ways of writing y = f(x, y) are: y (x) = f(x, y) dy(x) dx = f(x, y) We sall take te following D.E. as an example: f(x, y) = y or y = y (or y (x) = y(x)). Tis as an infinite number of solutions: y(x) = C e x C R We can single out one solution by supplying an initial condition y(x 0 ) = y 0. So, in our example, if we say tat y(0) = 1, ten we find tat C = 1 and out solution is y = e x Working Example Te Lipscitz Condition I y - Initial Condition We can give a condition tat determines wen te initial condition is sufficient to ensure a unique solution, known as te Lipscitz Condition Lipscitz Condition: For a x b, for all < y, y <, if tere is an L suc tat 10 f(x, y) f(x, y ) L y y 5 x Te dased lines sow te many solutions for different values of C. Te solid line sows te solution singled out by te initial condition tat y(0) = 1. Ten te solution to y = f(x, y) is unique, given an initial condition. L is often referred to as te Lipscitz Constant. A useful estimate for L is to take f y L, for x in (a, b).

5 Te Lipscitz Condition II Numerically solving y = f(x, y) Example. given our example of y = y = f(x, y), ten we can see do we get a suitable L. So we sall try L = 1 f y = (y) (y) = 1 f(x, y) f(x, y ) = y y 1 y y So we see tat we satisfy te Lipscitz Condition wit a Constant L = 1. We assume we are trying to find values of y for x ranging over te interval [a, b]. We start wit te one point were we ave te exact answer, namely te initial condition y 0 = y(x 0 ). We generate a series of x-points from a = x 0 to b, separated by a small step-interval : x 0 = a x i = a + i = b a N x N = b we want to compute {y i }, te approximations to {y(x i )}, te true values. Euler s Metod Euler s Metod Te tecnique works by using applying f at te current point (x n, y n ) to get an estimate of y at tat point. Euler s Metod. Tis is ten used to compute y n+1 as follows: y n+1 = y n + f(x n, y n ) Tis tecnique for solving D.E. s is known as Euler s Metod. Example. In our example, we ave y = y f(x, y) = y y n+1 = y n + y n At eac point after x 0, we accumulate an error, because we are using te slope at x n to estimate y n+1, wic assumes tat te slope doesn t cange over interval [x n, x n+1 ]. It is simple, slow and inaccurate, wit experimentation sowing tat te error is O(). Truncation Errors I Definitions. Te error introduced at eac step is called te Local Truncation Error. Te error introduced at any given point, as a result of accumulating all te local truncation errors up to tat point, is called te Global Truncation Error. y(x ) n+1 y n y n+1 Truncation Errors II We can estimate te local truncation error y(x n+1 ) y n+1, by assuming te value y n for x n is exact as follows: as follows: Using Taylor Expansion about x = x n y(x n+1 ) = y(x n + ) y(x n+1 ) = y(x n ) + y (x n ) y (ξ) Assuming y n is exact (y n = y(x n )), so y (x n ) = f(x n, y n ) y(x n+1 ) = y n + f(x n, y n ) y (ξ) Now looking at y n+1 by definition of te Euler metod: x n x n+1 In te diagram above, te local truncation error is y(x n+1 ) y n+1. We subtract te two results: y n+1 = y n + f(x n, y n ) y(x n+1 ) y n+1 = 2 2 y (ξ)

6 Truncation Errors III Introduction So y(x n+1 ) y n+1 O( 2 ) We saw tat te local truncation error for Euler s Metod is O( 2 ). By integration (accumulation of error wen starting from x 0 ), we see tat global error is O(). As a general principle, we find tat if te Local Truncation Error is O( p+1 ), ten te Global Truncation Error is O( p ). Considering te problem of solving differential equations wit one initial condition, we learnt about: Lipscitz Condition (unicity of te solution) finding numerically te solution : Euler metod Today is about ow to improve te Euler s algoritm: Heun s metod and more generally Runge-Kutta s tecniques. Improved Differentiation Tecniques I Improved Differentiation Tecniques II We can improve on Euler s tecnique to get better estimates for y n+1. Te idea is to use te equation y = f(x, y) to estimate te slope at x n+1 as well, and ten average tese two slopes to get a better result. k1 (e) y n y n+1 0.5(k1+k2) x n y n+1 x n+1 k2 Using te slope y (x n, y n ) = f(x n, y n ) at x n, te Euler approximation is: (A) f(x n, y n ) Considering te slope y (x n+1, y n+1 ) = f(x n+1, y n+1 ) at x n+1, we can propose tis new approximation: (B) f(x n+1, y n+1 ) Te trouble is: we dont know y n+1 in f (because tis is wat we are looking for!). So instead we use y (e) n+1 te Euler s approximation of y n+1: (B) f(x n+1, y (e) n+1 ) Improved Differentiation Tecniques III Runge-Kutta Tecniques I So considering te two approximations of yn+1 yn wit expressions (A) and (B), we get a better approximation by averaging (ie. by computing A+B/2): Heun s Metod. Te approximation: y n+1 = y n + 2 is known as Heun s Metod. 1 ( 2 f(x n, y n ) + f(x n+1, y (e) ) n+1 ( f(x n, y n ) + f(x n+1, y (e) ) n+1 = y n + 2 (f(x n, y n ) + f(x n+1, y n + f(x n, y n )) It can be sown to ave a global truncation error tat is O( 2 ). Te cost of tis improvement in error beaviour is tat we evaluate f twice on eac -step. We can repeat te Heun s approac by considering te approximations of slopes in te interval [x n ; x n+1 ]. Tis leads to a large class of improved differentiation tecniques wic evaluate f many times at eac -step, in order to get better error performance. Tis class of tecniques is referred to collectively as Runge-Kutta tecniques, of wic Heun s Metod is te simplest example. Te classical Runge-Kutta tecnique evaluates f four times to get a metod wit global truncation error of O( 4 ).

7 Runge-Kutta Tecniques II Runge-Kutta s tecnique using 4 approximations. It is computed using approximations of te slope at x n, x n+1 and also two approximations at mid interval x n + 2 : wit = 1 6 (f f f 3 + f 4 ) f 1 = f (x n, y n) ( ) f 2 = f x n + 2, y n + 2 f 1 ( ) f 3 = f x n + 2, y n + 2 f 2 f 4 = f (x n+1, y n + f 3 ) It can be sown tat te global truncation error is O( 4 ).