CHAPTER 11 Numericl Differentition nd Integrtion Differentition nd integrtion re bsic mthemticl opertions with wide rnge of pplictions in mny res of science. It is therefore importnt to hve good methods to compute nd mnipulte derivtives nd integrls. You probbly lernt the bsic rules of differentition nd integrtion in school symbolic methods suitble for pencil-nd-pper clcultions. These re importnt, nd most derivtives cn be computed this wy. Integrtion however, is different, nd most integrls cnnot be determined with symbolic methods like the ones you lernt in school. Another compliction is the fct tht in prcticl pplictions function is only known t few points. For exmple, we my mesure the position of cr every minute vi GPS (Globl Positioning System) unit, nd we wnt to compute its speed. If the position is known s continuous function of time, we cn find the speed by differentiting this function. But when the position is only known t isolted times, this is not possible. The sme pplies to integrls. The solution, both when it comes to integrls tht cnnot be determined by the usul methods, nd functions tht re only known t isolted points, is to use pproximte methods of differentition nd integrtion. In our context, these re going to be numericl methods. We re going to present number of methods for doing numericl integrtion nd differentition, but more importntly, we re going to present generl strtegy for deriving such methods. In this wy you will not only hve number of methods vilble to you, but you will lso be ble to develop new methods, tilored to specil situtions tht you my encounter. 7
We use the sme generl strtegy for deriving both numericl integrtion nd numericl differentition methods. The bsic ide is to evlute function t few points, find the polynomil tht interpoltes the function t these points, nd use the derivtive or integrl of the polynomil s n pproximtion to the function. This technique lso llows us to keep trck of the so-clled trunction error, the mthemticl error committed by integrting or differentiting the polynomil insted of the function itself. However, when it comes to roundoff error, we hve to tret differentition nd integrtion differently: Numericl integrtion is very insensitive to round-off errors, while numericl differentition behves in the opposite wy; it is very sensitive to round-off errors. 11.1 A simple method for numericl differentition We strt by studying numericl differentition. We first introduce the simplest method, derive its error, nd its sensitivity to round-off errors. The procedure used here for deriving the method nd nlysing the error is used over gin in lter sections to derive nd nlyse dditionl methods. Let us first mke it cler wht numericl differentition is. Problem 11.1 (Numericl differentition). Let f be given function tht is only known t number of isolted points. The problem of numericl differentition is to compute n pproximtion to the derivtive f of f by suitble combintions of the known vlues of f. A typicl exmple is tht f is given by computer progrm (more specificlly function, procedure or method, depending on you choice of progrmming lnguge), nd you cn cll the progrm with floting-point rgument x nd receive bck floting-point pproximtion of f (x). The chllenge is to compute n pproximtion to f () for some rel number when the only id we hve t our disposl is the progrm to compute vlues of f. 11.1.1 The bsic ide Since we re going to compute derivtives, we must be cler bout they re defined. Recll tht f () is defined by f f ( + h) f () () = lim. (11.1) h 0 h In the following we will ssume tht this limit exists; i.e., tht f is differentible. From (11.1) we immeditely hve nturl pproximtion to f (); we simply 8
pick positive h nd use the pproximtion f () f ( + h) f (). (11.) h Note tht this corresponds to pproximting f by the stright line p 1 tht interpoltes f t nd h, nd then using p 1 () s n pproximtion to f (). Observtion 11.. The derivtive of f t cn be pproximted by f () f ( + h) f (). h In prcticl sitution, the number would be given, nd we would hve to locte the two nerest vlues 1 nd to the left nd right of such tht f ( 1 ) nd f ( ) cn be found. Then we would use the pproximtion f () f ( ) f ( 1 ) 1. In lter sections, we will derive severl formuls like (11.). Which formul to use for specific exmple, nd exctly how to use it, will hve to be decided in ech cse. Exmple 11.3. Let us test the pproximtion (11.) for the function f (x) = sin x t = 0.5 (using 64-bit floting-point numbers). In this cse we hve f (x) = cos x so f () = 0.8775856. This mkes it is esy to check the ccurcy. We try with few vlues of h nd find h ( f ( + h) f () )/ h E1 (f ;,h) 10 1 0.851693479.5 10 10 0.875170879.4 10 3 10 3 0.87734709.4 10 4 10 4 0.877558589.4 10 5 10 5 0.8775801647.4 10 6 10 6 0.877583.4 10 7 where E 1 (f ;,h) = f () ( f (+h) f () )/ h. In other words, the pproximtion seems to improve with decresing h, s expected. More precisely, when h is reduced by fctor of 10, the error is reduced by the sme fctor. 9
11.1. The trunction error Whenever we use pproximtions, it is importnt to try nd keep trck of the error, if t ll possible. To nlyse the error in numericl differentition, Tylor polynomils with reminders re useful. To nlyse the error in the pproximtion bove, we do Tylor expnsion of f ( + h). We hve f ( + h) = f () + h f () + h f (ξ h ), where ξ h lies in the intervl (, + h). If we rerrnge this formul, we obtin f f ( + h) f () () = h h f (ξ h ). (11.3) This is often referred to s the trunction error of the pproximtion, nd is resonble error formul, but it would be nice to get rid of ξ h. We first tke bsolute vlues in (11.3), f f ( + h) f () () h = h f (ξ h ). Recll from the Extreme vlue theorem tht if function is continuous, then its mximum lwys exists on ny closed nd bounded intervl. In our setting here, it is nturl to let the closed nd bounded intervl be [, +h]. This leds to the following lemm. Lemm 11.4. Suppose tht f hs continuous derivtives up to order two ner. If the derivtive f () is pproximted by f ( + h) f (), h then the trunction error is bounded by E(f ;,h) = f f ( + h) f () () h h mx x [,+h] f (x). (11.4) Let us check tht the error formul (11.3) confirms the numericl vlues in exmple 11.3. We hve f (x) = sin x, so the right-hnd side in (11.4) becomes E(sin;0.5,h) = h sinξ h, 30
where ξ h (0.5,0.5 + h). For h = 0.1 we therefore hve tht the error must lie in the intervl [0.05sin0.5, 0.05sin0.6] = [.397 10,.83 10 ], nd the right end of the intervl is the mximum vlue of the right-hnd side in (11.4). When h is reduced by fctor of 10, the fctor h/ is reduced by the sme fctor. This mens tht ξ h will pproch 0.5 so sinξ h will pproch the lower vlue sin0.5 0.47946. For h = 10 n, the error will therefore tend to 10 n sin0.5/ 10 n 0.397, which is in complete greement with exmple 11.3. This is true in generl. If f is continuous, then ξ h will pproch when h goes to zero. But even when h > 0, the error in using the pproximtion f (ξ h ) f () is smll. This is the cse since it is usully only necessry to know the mgnitude of the error, i.e., it is sufficient to know the error with one or two correct digits. Observtion 11.5. The trunction error is pproximtely given by f f ( + h) f () () h h f (). 11.1.3 The round-off error So fr, we hve just considered the mthemticl error committed when f () is pproximted by ( f ( + h) f () )/ h. But wht bout the round-off error? In fct, when we compute this pproximtion we hve to perform the one criticl opertion f ( + h) f () subtrction of two lmost equl numbers which we know from chpter 5 my led to lrge round-off errors. Let us continue exmple 11.3 nd see wht hppens if we use smller vlues of h. Exmple 11.6. Recll tht we estimted the derivtive of f (x) = sin x t = 0.5 nd tht the correct vlue with ten digits is f (0.5) 0.877585619. If we check vlues of h from 10 7 nd smller we find ( )/ h f ( + h) f () h E(f ;,h) 10 7 0. 87758537.5 10 8 10 8 0.8775856.9 10 10 10 9 0.8775856.9 10 10 10 11 0.8775813409 1. 10 6 10 14 0.8770761895 5.1 10 4 10 15 0.8881784197 1.1 10 10 16 1.110305.3 10 1 10 17 0.000000000 8.8 10 1 31
This shows very clerly tht something quite drmtic hppens, nd when we come to h = 10 17, the derivtive is computed s zero. If f () is the floting-point number closest to f (), we know from lemm 5.6 tht the reltive error will be bounded by 5 53 since floting-point numbers re represented in binry (β = ) with 53 bits for the significnd (m = 53). We therefore hve ɛ 5 53 6 10 16. In prctice, the rel upper bound on ɛ is usully smller, nd in the following we will denote this upper bound by ɛ. This mens tht definite upper bound on ɛ is 6 10 16. Nottion 11.7. The mximum reltive error when rel number is represented by floting-point number is denoted by ɛ. There is hndy wy to express the reltive error in f (). If we denote the computed vlue of f () by f (), we will hve f () = f ()(1 + ɛ) which corresponds to the reltive error being ɛ. Observtion 11.8. Suppose tht f () is computed with 64-bit floting-point numbers nd tht no underflow or overflow occurs. Then the computed vlue f () stisfies f () = f ()(1 + ɛ) (11.5) where ɛ ɛ, nd ɛ depends on both nd f. The computtion of f ( + h) is of course lso ffected by round-off error, so we hve f () = f ()(1 + ɛ 1 ), f ( + h) = f ( + h)(1 + ɛ ) (11.6) where ɛ i ɛ for i = 1,. Here we should relly write ɛ = ɛ (h), becuse the exct round-off error in f ( + h) will inevitbly depend on h in rther rndom wy. The next step is to see how these errors ffect the computed pproximtion of f (). Recll from exmple 5.11 tht the min source of round-off in subtrction is the replcement of the numbers to be subtrcted by the nerest flotingpoint numbers. We therefore consider the computed pproximtion to be f ( + h) f (). h 3
If we insert the expressions (11.6), nd lso mke use of lemm 11.4, we obtin f ( + h) f () f ( + h) f () = h h + f ( + h)ɛ f ()ɛ 1 h = f () + h f (ξ h ) + f ( + h)ɛ f ()ɛ 1 h where ξ h (, + h). This leds to the following importnt observtion. (11.7) Theorem 11.9. Suppose tht f nd its first two derivtives re continuous ner. When the derivtive of f t is pproximted by f ( + h) f (), h the error in the computed pproximtion is given by f f ( + h) f () () h h M 1 + ɛ h M, (11.8) where M 1 = mx x [,+h] f (), M = mx x [,+h] f (). Proof. To get to (11.8) we hve rerrnged (11.7) nd used the tringle inequlity. We hve lso replced f (ξ h ) by its mximum on the intervl [, + h], s in (11.4). Similrly, we hve replced f (ξ h ) nd f ( + h) by their common mximum on [, +h]. The lst term then follows by pplying the tringle inequlity to the lst term in (11.7) nd replcing ɛ 1 nd ɛ (h) by the upper bound ɛ. The inequlity (11.8) cn be replced by n pproximte equlity by mking the pproximtions M 1 f () nd M f (), just like in observtion 11.8 nd using the mximum of ɛ 1 nd ɛ in (11.7), which we denote ɛ(h). Observtion 11.10. The inequlity (11.8) is pproxmtely equivlent to f f ( + h) f () () h h f () ɛ(h) + f (). (11.9) h Let us check how well observtion 11.10 grees with the computtions in exmples 11.3 nd 11.6. 33
Exmple 11.11. For lrge vlues of h the first term on the right in (11.9) will dominte the error nd we hve lredy seen tht this grees very well with the computed vlues in exmple 11.3. The question is how well the numbers in exmple 11.6 cn be modelled when h becomes smller. To estimte the size of ɛ(h), we consider the cse when h = 10 16. Then the observed error is.3 10 1 so we should hve 10 16 We solve this eqution nd find sin0.5 ɛ( 10 16) 10 16 =.3 10 1. ɛ ( 10 16) = 10 16 (.3 10 1 + 10 16 ) sin 0.5. = 1. 10 17. If we try some other vlues of h we find ɛ ( 10 11) = 6.1 10 18, ɛ ( 10 13) =.4 10 18, ɛ ( 10 15) = 5.3 10 18. We observe tht ll these vlues re considerbly smller thn the upper limit 6 10 16 which we mentioned bove. Figure 11.1 shows plots of the error. The numericl pproximtion hs been computed for the vlues n = 0.01i, i = 0,..., 00 nd plotted in log-log plot. The errors re shown s isolted dots, nd the function g (h) = h sin0.5 + ɛ h sin0.5 (11.10) with ɛ = 7 10 17 is shown s solid grph. It seems like this choice of ɛ mkes g (h) resonble upper bound on the error. 11.1.4 Optiml choice of h Figure 11.1 indictes tht there is n optiml vlue of h which minimises the totl error. We cn find this mthemticlly by minimising the upper bound in (11.9), with ɛ(h) replced by the upper bound ɛ. This gives g (h) = h f () ɛ + f (). (11.11) h To find the vlue of h which minimises this expression, we differentite with respect to h nd set the derivtive to zero. We find f g () (h) = ɛ f (). h If we solve the eqution g (h) = 0, we obtin the pproximte optiml vlue. 34
0 15 10 5 4 6 8 Figure 11.1. Numericl pproximtion of the derivtive of f (x) = sin x t x = 0.5 using the pproximtion in lemm 11.4. The plot is log 10 -log 10 plot which shows the logrithm to bse 10 of the bsolute vlue of the totl error s function of the logrithm to bse 10 of h, bsed on 00 vlues of h. The point 10 on the horizontl xis therefore corresponds h = 10 10, nd the point 6 on the verticl xis corresponds to n error of 10 6. The plot lso includes the function given by (11.10). 10 Lemm 11.1. Let f be function with continuous derivtives up to order. If the derivtive of f t is pproximted s in lemm 11.4, then the vlue of h which minimises the totl error (trunction error + round-off error) is pproximtely ɛ f () h f (). It is esy to see tht the optiml vlue of h is the vlue tht blnces the two terms in (11.11)l, i.e., the trunction error nd the round-off error re equl. In the exmple with f (x) = sin x nd = 0.5 we cn use ɛ = 7 10 17 which gives h = ɛ = 7 10 17 1.7 10 8. 11. Summry of the generl strtegy Before we continue, let us sum up the derivton nd nlysis of the numericl differentition method in section 11.1, since we will use this over nd over gin. The first step ws to derive the numericl method. In section 11.1 this ws very simple since the method cme stright out of the definition of the derivtive. Just before observtion 11. we indicted tht the method cn lso be de- 35
rived by pproximting f by polynomil p nd using p () s n pproximtion to f (). This is the generl pproch tht we will use below. Once the numericl method is known, we estimte the mthemticl error in the pproximtion, the trunction error. This we do by performing Tylor expnsions with reminders. For numericl differentition methods which provide estimtes of derivtive t point, we replce ll function vlues t points other thn by Tylor polynomils with reminders. There my be chllenge to choose the degree of the Tylor polynomil. The next tsk is to estimte the totl error, including round-off error. We consider the difference between the derivtive to be computed nd the computed pproximtion, nd replce the computed function evlutions by expressions like the ones in observtion 11.8. This will result in n expression involving the mthemticl pproximtion to the derivtive. This cn be simplified in the sme wy s when the trunction error ws estimted, with the ddition of n expression involving the reltive round-off errors in the function evlutions. These expressions cn then be simplified to something like (11.8) or (11.9). As finl step, the optiml vlue of h cn be found by minimising the totl error. Algorithm 11.13. To derive nd nlyse numericl differentition method, the following steps re necessry: 1. Derive the method using polynomil interpoltion.. Estimte the trunction error using Tylor series with reminders. 3. Estimte the totl error (trunction error + round-off error) by ssuming ll function evlutions re replced by the nerest floting-point numbers. 4. Estimte the optiml vlue of h. 11.3 A simple, symmetric method The numericl differentition method in section 11.1 is not symmetric bout, so let us try nd derive symmetric method. 11.3.1 Construction of the method We wnt to find n pproximtion to f () using vlues of f ner. To obtin symmetric method, we ssume tht f ( h), f (), nd f ( + h) re known 36
vlues, nd we wnt to find n pproximtion to f () using these vlues. The strtegy is to determine the qudrtic polynomil p tht interpoltes f t h, nd + h, nd then we use p () s n pproximtion to f (). We write p in Newton form, p (x) = f [ h] + f [ h, ](x ( h)) We differentite nd find Setting x = yields + f [ h,, + h](x ( h))(x ). (11.1) p (x) = f [ h, ] + f [ h,, + h](x + h). p () = f [ h, ] + f [ h,, + h]h. To get prcticlly useful formul we must express the divided differences in terms of function vlues. If we expnd the second expression we obtin p f [, + h] f [ h, ] f [, + h] + f [ h, ] () = f [ h, ]+ h = h The two first order differences re (11.13) f [ h, ] = f () f ( h), f [, + h] = h f ( + h) f (), h nd if we insert this in (11.13) we end up with p f ( + h) f ( h) () =. h Lemm 11.14. Let f be given function, nd let nd h be given numbers. If f ( h), f (), f ( + h) re known vlues, then f () cn be pproximted by p () where p is the qudrtic polynomil tht interpoltes f t h,, nd + h. The pproximtion is given by f () p f ( + h) f ( h) () =. (11.14) h Let us test this pproximtion on the function f (x) = sin x t = 0.5 so we cn compre with the method in section 11.1. 37
Exmple 11.15. We test the pproximtion (11.14) with the sme vlues of h s in exmples 11.3 nd 11.6. Recll tht f (0.5) 0.877585619 with ten correct decimls. The results re ( )/ h f ( + h) f ( h) (h) E(f ;,h) 10 1 0.876106554 1.5 10-3 10 0.8775679356 1.5 10-5 10 3 0.877584156 1.5 10-7 10 4 0.877585604 1.5 10-9 10 5 0.877585619 1.8 10-11 10 6 0.877585619 7.5 10-1 10 7 0.877585616.7 10-10 10 8 0.8775856.9 10-10 10 11 0.8775813409 1. 10-6 10 13 0.8776313010 4.9 10-5 10 15 0.8881784197 1.1 10-10 17 0.0000000000 8.8 10-1 If we compre with exmples 11.3 nd 11.6, the errors re generlly smller. In prticulr we note tht when h is reduced by fctor of 10, the error is reduced by fctor of 100, t lest s long s h is not too smll. However, when h becomes smller thn bout 10 6, the error becomes lrger. It therefore seems like the trunction error is smller thn in the first method, but the round-off error mkes it impossible to get ccurte results for smll vlues of h. The optiml vlue of h seems to be h 10 6, which is lrger thn for the first method, but the error is then bout 10 1, which is smller thn the best we could do with the first method. 11.3. Trunction error Let us ttempt to estimte the trunction error for the method in lemm 11.14. The ide is to do replce f ( h) nd f ( + h) with Tylor expnsions bout. We use the Tylor expnsions f ( + h) = f () + h f () + h f () + h3 6 f (ξ 1 ), f ( h) = f () h f () + h f () h3 6 f (ξ ), where ξ 1 (, + h) nd ξ ( h, ). If we subtrct the second formul from the first nd divide by h, we obtin f ( + h) f ( h) h = f () + h ( f (ξ 1 ) + f (ξ ) ). (11.15) 1 38
This leds to the following lemm. Lemm 11.16. Suppose tht f nd its first three derivtives re continuous ner, nd suppose we pproximte f () by f ( + h) f ( h). (11.16) h The trunction error in this pproximtion is bounded by E (f ;,h) = f () f ( + h) f ( h) h h 6 mx x [ h,+h] f (x). (11.17) Proof. Wht remins is to simplify the lst term in (11.15) to the term on the right in (11.17). This follows from f (ξ 1 ) + f (ξ ) mx f (x) + mx f (x) x [,+h] x [ h,] mx f (x) + mx f (x) x [ h,+h] x [ h,+h] = mx f (x). x [ h,+h] The lst inequlity is true becuse the width of the intervls over which we tke the mximums re incresed, so the mximum vlues my lso increse. The error formul (11.17) confirms the numericl behviour we sw in exmple 11.15 for smll vlues of h since the error is proportionl to h : When h is reduced by fctor of 10, the error is reduced by fctor 10. 11.3.3 Round-off error The round-off error my be estimted just like for the first method. When the pproximtion (11.16) is computed, the vlues f ( h) nd f ( + h) re replced by the nerest floting point numbers f ( h) nd f ( + h) which cn be expressed s f ( + h) = f ( + h)(1 + ɛ 1 ), f ( h) = f ( h)(1 + ɛ ), where both ɛ 1 nd ɛ depend on h nd stisfy ɛ i ɛ for i = 1,. Using these expressions we obtin f ( + h) f ( h) = h f ( + h) f ( h) h + f ( + h)ɛ 1 f ( h)ɛ. h 39
We insert (11.15) nd get the reltion f ( + h) f ( h) h = f () + h ( f (ξ 1 ) + f (ξ ) ) + f ( + h)ɛ 1 f ( h)ɛ. 1 h This leds to n estimte of the totl error if we use the sme technique s in the proof of lemm 11.8. Theorem 11.17. Let f be given function with continuous derivtives up to order three, nd let nd h be given numbers. Then the error in the pproximtion f f ( + h) f ( h) (), h including round-off error nd trunction error, is bounded by f f ( + h) f ( h) () h h 6 M 1 + ɛ h M (11.18) where M 1 = mx x [ h,+h] f (x), M = mx x [ h,+h] f (x). (11.19) In prctice, the interesting vlues of h will usully be so smll tht there is very little error in mking the pproximtions M 1 = mx x [ h,+h] f (x) f (), M = mx x [ h,+h] f (x) f (). If we mke this simplifiction in (11.18) we obtin slightly simpler error estimte. Observtion 11.18. The error (11.18) is pproximtely bounded by f f ( + h) f ( h) () h h f () ɛ f () +. (11.0) 6 h A plot of how the error behves in this pproximtion, together with the estimte of the error on the right in (11.0), is shown in figure 11.. 40
15 10 5 4 6 8 10 Figure 11.. Log-log plot of the error in the pproximtion to the derivtive of f (x) = sin x t x = 1/ for vlues of h in the intervl [0,10 17 ], using the method in theorem 11.17. The function plotted is the right-hnd side of (11.0) with ɛ = 7 10 17, s function of h. 1 11.3.4 Optiml choice of h As for the first numericl differentition method, we cn find n optiml vlue of h which minimises the error. The error is minimised when the trunction error nd the round-off error hve the sme mgnitude. We cn find this vlue of h if we differentite the right-hnd side of (11.18) with respect to h nd set the derivtive to 0. This leds to the eqution h 3 M 1 ɛ h M = 0 which hs the solution h = 3 3ɛ M 3 M1 3 3ɛ f () f (). 3 At the end of section 11.1.4 we sw tht resonble vlue for ɛ ws ɛ = 7 10 17. The optiml vlue of h in exmple 11.15, where f (x) = sin x nd = 1/, then becomes h = 4.6 10 6. For this vlue of h the pproximtion is f (0.5) 0.87758561887 with error 3.1 10 1. 11.4 A four-point method for differentition In wy, the two methods for numericl differentition tht we hve considered so fr re the sme. If we use step length of h in the first method, the 41
pproximtion becomes f f ( + h) f () (). h The nlysis of the symmetric method shows tht the pproximtion is considerbly better if we ssocite the pproximtion with the midpoint between nd + h, f f ( + h) f () ( + h). h At the point +h the pproximtion is proportionl to h rther thn h, nd this mkes big difference s to how quickly the error goes to zero, s is evident if we compre exmples 11.3 nd 11.15. In this section we derive nother method for which the trunction error is proportionl to h 4. The computtions below my seem overwhelming, nd hve in fct been done with the help of computer to sve time nd reduce the risk of misclcultions. The method is included here just to illustrte tht the principle for deriving both the method nd the error terms is just the sme s for the simple symmetric method in the previous section. To sve spce we hve only included one highlight, of the pproximtion method nd the totl error. 11.4.1 Derivtion of the method We wnt better ccurcy thn the symmetric method which ws bsed on interpoltion with qudrtic polynomil. It is therefore nturl to bse the pproximtion on cubic polynomil, which cn interpolte four points. We hve seen the dvntge of symmetry, so we choose the interpoltion points x 0 = h, x 1 = h, x = + h, nd x 3 = + h. The cubic polynomil tht interpoltes f t these points is p 3 (x) = f (x 0 ) + f [x 0, x 1 ](x x 0 ) + f [x 0, x 1, x ](x x 0 )(x x 1 ) nd its derivtive is p 3 (x) = f [x 0, x 1 ] + f [x 0, x 1, x ](x x 0 x 1 ) + f [x 0, x 1, x, x 3 ](x x 0 )(x x 1 )(x x ). + f [x 0, x 1, x, x 3 ] ( (x x 1 )(x x ) + (x x 0 )(x x ) + (x x 0 )(x x 1 ) ). If we evlute this expression t x = nd simplify (this is quite bit of work), we find tht the resulting pproximtion of f () is f () p 3 f ( h) 8f ( h) + 8f ( + h) f ( + h) () =. (11.1) 1h 4
11.4. Trunction error To estimte the error, we expnd the four terms in the numertor in (11.1) in Tylor series, f ( h) = f () h f () + h f () 4h3 3 f () + h4 3 f (i v) () 4h5 15 f (v) (ξ 1 ), f ( h) = f () h f () + h f () h3 6 f () + h4 4 f (i v) () h5 10 f (v) (ξ ), f ( + h) = f () + h f () + h f () + h3 6 f () + h4 4 f (i v) () + h5 10 f (v) (ξ 3 ), f ( + h) = f () + h f () + h f () + 4h3 3 f () + h4 3 f (i v) () + 4h5 15 f (v) (ξ 4 ), where ξ 1 ( h, ), ξ ( h, ), ξ 3 (, + h), nd ξ 4 (, + h). If we insert this into the formul for p 3 () we obtin f ( h) 8f ( h) + 8f ( + h) f ( + h) = 1h f () h4 45 f (v) (ξ 1 ) + h4 180 f (v) (ξ ) + h4 180 f (v) (ξ 3 ) h4 45 f (v) (ξ 4 ). If we use the sme trick s for the symmetric method, we cn combine ll lst four terms in nd obtin n upper bound on the trunction error. The result is f f ( h) 8f ( h) + 8f ( + h) f ( + h) () 1h h4 18 M (11.) where M = mx x [ h,+h] f (v) (x). 11.4.3 Round-off error The trunction error is derived in the sme wy s before. The quntities we ctully compute re f ( h) = f ( h)(1 + ɛ 1 ), f ( + h) = f ( + h)(1 + ɛ 3 ), f ( h) = f ( h)(1 + ɛ ), f ( + h) = f ( + h)(1 + ɛ 4 ). We estimte the difference between f () nd the computed pproximtion, mke use of the estimte (11.), combine the function vlues tht re multiplied by ɛs, nd pproximte the mximum vlues by function vlues t. We sum up the result. 43
15 10 5 5 10 15 Figure 11.3. Log-log plot of the error in the pproximtion to the derivtive of f (x) = sin x t x = 1/, using the method in observtion 11.19, with h in the intervl [0,10 17 ]. The function plotted is the right-hnd side of (11.3) with ɛ = 7 10 17. Observtion 11.19. Suppose tht f nd its first five derivtives re continuous. If f () is pproximted by f () f ( h) 8f ( h) + 8f ( + h) f ( + h), 1h the totl error is pproximtely bounded by f f ( h) 8f ( h) + 8f ( + h) f ( + h) () 1h h 4 18 f (v) () 3ɛ + f (). (11.3) h A plot of the error in the pproximtion for the sin x exmple is shown in figure 11.3. 11.4.4 Optiml vlue of h From observtion 11.19 we cn compute the optiml vlue of h by differentiting the right-hnd side with respect to h nd setting it to zero, h 3 9 f (v) () 3ɛ h f () = 0 44
which hs the solution h = 5 7ɛ f () f (v) (). 5 For the cse bove with f (x) = sin x nd = 0.5 the solution is h 8.8 10 4. For this vlue of h the ctul error is 10 14. 11.5 Numericl pproximtion of the second derivtive We consider one more method for numericl pproximtion of derivtives, this time of the second derivtive. The pproch is the sme: We pproximte f by polynomil nd pproximte the second derivtive of f by the second derivtive of the polynomil. As in the other cses, the error nlysis is bsed on expnsion in Tylor series. 11.5.1 Derivtion of the method Since we re going to find n pproximtion to the second derivtive, we hve to pproximte f by polynomil of degree t lest two, otherwise the second derivtive is identiclly 0. The simplest is therefore to use qudrtic polynomil, nd for symmetry we wnt it to interpolte f t h,, nd + h. The resulting polynomil p is the one we used in section 11.3 nd it is given in eqution (11.1). The second derivtive of p is constnt, nd the pproximtion of f () is f () p () = f [ h,, + h]. The divided difference is esy to expnd. Lemm 11.0. The second derivtive of function f t cn be pproximted by f f ( + h) f () + f ( h) () h. (11.4) 11.5. The trunction error Estimtion of the error goes s in the other cses. The Tylor series of f ( h) nd f ( + h) re f ( h) = f () h f () + h f () h3 6 f () + h4 4 f (i v) (ξ 1 ), f ( + h) = f () + h f () + h f () + h3 6 f () + h4 4 f (i v) (ξ ), 45
where ξ 1 ( h, ) nd ξ (, + h). If we insert these Tylor series in (11.4) we obtin f ( + h) f () + f ( h) h = f () + h ( f (i v) (ξ 1 ) + f (i v) (ξ ) ). 4 From this obtin n expression for the trunction error. Lemm 11.1. Suppose f nd its first three derivtives re continuous ner. If the second derivtive f () is pproximted by f () f ( + h) f () + f ( h) h, the error is bounded by f f ( + h) f () + f ( h) () h h 1 mx x [ h,+h] f (x). (11.5) 11.5.3 Round-off error The round-off error cn lso be estimted s before. Insted of computing the exct vlues, we compute f ( h), f (), nd f ( + h), which re linked to the exct vlues by f ( h) = f ( h)(1 + ɛ 1 ), f () = f ()(1 + ɛ ), f ( + h) = f ( + h)(1 + ɛ 3 ), where ɛ i ɛ for i = 1,, 3. The difference between f () nd the computed pproximtion is therefore f f ( + h) f () + f ( h) () h = h ( f (ξ 1 ) + f (ξ ) ) ɛ 1 f ( h) ɛ f () + ɛ 3 f ( + h) 4 h. If we combine terms on the right s before, we end up with the following theorem. Theorem 11.. Suppose f nd its first three derivtives re continuous ner, nd tht f () is pproximted by f () f ( + h) f () + f ( h) h. 46
8 6 4 4 6 8 Figure 11.4. Log-log plot of the error in the pproximtion to the derivtive of f (x) = sin x t x = 1/ for h in the intervl [0,10 8 ], using the method in theorem 11.. The function plotted is the right-hnd side of (11.3) with ɛ = 7 10 17. Then the totl error (trunction error + round-off error) in the computed pproximtion is bounded by f f ( + h) f () + f ( h) () h h 1 M 1 + 3ɛ h M. (11.6) where M 1 = mx x [ h,+h] f (i v) (x), M = mx x [ h,+h] f (x). As before, we cn simplify the right-hnd side to h f (i v) () + 3ɛ f () 1 h (11.7) if we cn tolerte slightly pproximte upper bound. Figure 11.4 shows the errors in the pproximtion to the second derivtive given in theorem 11. when f (x) = sin x nd = 0.5 nd for h in the rnge [0,10 8 ]. The solid grph gives the function in (11.7) which describes the upper limit on the error s function of h, with ɛ = 7 10 17. For h smller thn 10 8, the pproximtion becomes 0, nd the error constnt. Recll tht for the pproximtions to the first derivtive, this did not hppen until h ws bout 10 17. This illustrtes the fct tht the higher the derivtive, the more problemtic is the round-off error, nd the more difficult it is to pproximte the derivtive with numericl methods like the ones we study here. 47
Figure 11.5. The re under the grph of function. 11.5.4 Optiml vlue of h Agin, we find the optiml vlue of h by minimising the right-hnd side of (11.6). To do this we find the derivtive with respect to h nd set it to 0, h 6 M 1 6ɛ h 3 M = 0. As usul it does not mke much difference if we use the pproximtions M 1 f () nd M = f (). Observtion 11.3. The upper bound on the totl error (11.6) is minimised when h hs the vlue 4 36ɛ f () h = f (i v) (). 4 When f (x) = sin x nd = 0.5 this gives h =. 10 4 if we use the vlue ɛ = 7 10 17. Then the pproximtion to f () = sin is 0.47945535 with n ctul error of 3.4 10 9. 11.6 Generl bckground on integrtion Our next tsk is to develop methods for numericl integrtion. Before we turn to this, it is necessry to briefly review the definition of the integrl. 48
() (b) (c) (d) Figure 11.6. The definition of the integrl vi inscribed nd circumsribed step functions. Recll tht if f (x) is function, then the integrl of f from x = to x = b is written f (x)dx. This integrl gives the re under the grph of f, with the re under the positive prt counting s positive re, nd the re under the negtive prt of f counting s negtive re, see figure 11.5. Before we continue, we need to define term which we will use repetedly in our description of integrtion. Definition 11.4. Let nd b be two rel numbers with < b. A prtition of [,b] is finite sequence {x i } n i=0 of incresing numbers in [,b] with x 0 = nd x n = b, = x 0 < x 1 < x < x n 1 < x n = b. The prtition is sid to be uniform if there is fixed number h, clled the step length, such tht x i x i 1 = h = (b )/n for i = 1,..., n. 49
The trditionl definition of the integrl is bsed on numericl pproximtion to the re. We pick prtition {x i } of [,b], nd in ech subintervl [x i 1, x i ] we determine the mximum nd minimum of f (for convenience we ssume tht these vlues exist), m i = min f (x), M i = mx f (x), x [x i 1,x i ] x [x i 1,x i ] for i = 1,,..., n. We use these vlues to compute the two sums n n I = m i (x i x i 1 ), I = M i (x i x i 1 ). i=1 To define the integrl, we consider lrger prtitions nd consider the limits of I nd I s the distnce between neighbouring x i s goes to zero. If those limits re the sme, we sy tht f is integrble, nd the integrl is given by this limit. More precisely, I = f (x)dx = sup I = inf I, where the sup nd inf re tken over ll prtitions of the intervl [,b]. This process is illustrted in figure 11.6 where we see how the piecewise constnt pproximtions become better when the rectngles become nrrower. The bove definition cn be used s numericl method for computing pproximtions to the integrl. We choose to work with either mxim or minim, select prtition of [,b] s in figure 11.6, nd dd together the res of the rectngles. The problem with this technique is tht it cn be both difficult nd time consuming to determine the mxim or minim, even on computer. However, it cn be shown tht the integrl hs very convenient property: If we choose point t i in ech intervl [x i 1, x i ], then the sum Ĩ = i=1 n f (t i )(x i x i 1 ) i=1 will lso converge to the integrl when the distnce between neighbouring x i s goes to zero. If we choose t i equl to x i 1 or x i, we hve simple numericl method for computing the integrl. An even better choice is the more symmetric t i = (x i + x i 1 )/ which leds to the pproximtion I n f ( (x i + x i 1 )/ ) )(x i x i 1 ). (11.8) i=1 This is the so-clled midpoint method which we will study in the next section. 50
x 1 () x 1 x 3 x 5 x 7 x 9 (b) Figure 11.7. The midpoint rule with one subintervl () nd five subintervls (b). In generl, we cn derive numericl integrtion methods by splitting the intervl [, b] into smll subintervls, pproximte f by polynomil on ech subintervl, integrte this polynomil rther thn f, nd then dd together the contributions from ech subintervl. This is the strtegy we will follow, nd this works s long s f cn be pproximted well by polynomils on ech subintervl. 11.7 The midpoint method for numericl integrtion We hve lredy introduced the midpoint rule (11.8) for numericl integrtion. In our stndrd frmework for numericl methods bsed on polynomil pproximtion, we cn consider this s using constnt pproximtion to the function f on ech subintervl. Note tht in the following we will lwys ssume the prtition to be uniform. Algorithm 11.5. Let f function which is integrble on the intervl [, b] nd let {x i } n be uniform prtition of [,b]. In the midpoint rule, the integrl of f is pproximted i=0 by f (x)dx I mid (h) = h n f (x i 1/ ), (11.9) i=1 where x i 1/ = (x i + x i 1 )/ = + (i 1/)h. This my seem like strngely formulted lgorithm, but ll there is to it is to compute the sum on the right in (11.9). The method is illustrted in figure 11.7 in the cses where we hve one nd five subintervls. 51
Exmple 11.6. Let us try the midpoint method on n exmple. As usul, it is wise to test on n exmple where we know the nswer, so we cn esily check the qulity of the method. We choose the integrl 1 0 cos x dx = sin1 0.8414709848 where the exct nswer is esy to compute by trditionl, symbolic methods. To test the method, we split the intervl into k subintervls, for k = 1,,..., 10, i.e., we hlve the step length ech time. The result is By error, we here men h I mid (h) Error 0.500000 0.85030065 8.8 10-3 0.50000 0.8436663. 10-3 0.15000 0.8401907 5.5 10-4 0.06500 0.84160796 1.4 10-4 0.03150 0.8415053 3.4 10-5 0.01565 0.84147954 8.6 10-6 0.007813 0.8414731.1 10-6 0.003906 0.8414715 5.3 10-7 0.001953 0.8414711 1.3 10-7 0.000977 0.8414710 3.3 10-8 1 0 f (x)dx I mid (h). Note tht ech time the step length is hlved, the error seems to be reduced by fctor of 4. 11.7.1 Locl error nlysis As usul, we should try nd keep trck of the error. We first focus on wht hppens on one subintervl. In other words we wnt to study the error f (x)dx f ( 1/ ) (b ), 1/ = ( + b)/. (11.30) Once gin, Tylor polynomils with reminders help us out. We expnd both f (x) nd f ( 1/ ) bout the the left endpoint f (x) = f () + (x )f (x ) () + f (ξ 1 ), f ( 1/ ) = f () + ( 1/ )f () + ( 1/ ) f (ξ ), 5
where ξ 1 is number in the intervl (, x) tht depends on x, nd ξ is in the intervl (, 1/ ). If we multiply the second Tylor expnsion by (b ), we obtin (b ) f ( 1/ )(b ) = f ()(b ) + f (b )3 () + f (ξ ). (11.31) 8 Next, we integrte the Tylor expnsion nd obtin f (x)dx = (f () + (x )f (x ) ) () + f (ξ 1 ) dx = f ()(b ) + 1 [ (x ) ] b f () + 1 (x ) f (ξ 1 )dx (b ) = f ()(b ) + f () + 1 (x ) f (ξ 1 )dx. (11.3) We then see tht the error cn be written f (x)dx f ( 1/ )(b ) = 1 b (x ) f (b )3 (ξ 1 )dx f (ξ ) 8 1 (x ) f (ξ 1 )dx (b )3 + f (ξ ). 8 (11.33) For the lst term, we use our stndrd trick, f (ξ ) M = mx x [,b] f (x). (11.34) Note tht since ξ (, 1/ ), we could just hve tken the mximum over the intervl [, 1/ ], but we will see lter tht it is more convenient to mximise over the whole intervl [,b]. The first term in (11.33) needs some mssging. Let us do the work first, nd explin fterwords, 1 b (x ) f (ξ 1 )dx 1 (x ) f (ξ 1 ) dx = 1 M = M 1 3 (x ) f (ξ 1 ) dx (x ) dx [ (x ) 3 ] b = M 6 (b )3. (11.35) 53
The first inequlity is vlid becuse when we move the bsolute vlue sign inside the integrl sign, the function tht we integrte becomes nonnegtive everywhere. This mens tht in the res where the integrnd in the originl expression is negtive, everything is now positive, nd hence the second integrl is lrger thn the first. Next there is n equlity which is vlid becuse (x ) is never negtive. The next inequlity follows becuse we replce f (ξ 1 ) with its mximum on the intervl [,b]. The lst step is just the evlution of the integrl of (x ). We hve now simplified both terms on the right in (11.33), so we hve The result is the following lemm. f (x)dx f ( 1/ ) (b ) M 6 (b )3 + M 8 (b )3. Lemm 11.7. Let f be continuous function whose first two derivtives re continuous on the intervl [.b]. The the error in the midpoint method, with only one intervl, is bounded by f (x)dx f ( 1/ ) (b ) 7M 4 (b )3, where M = mx x [,b] f (x) nd 1/ = ( + b)/. The importnce of this lemm lies in the fctor (b ) 3. This mens tht if we reduce the size of the intervl to hlf its width, the error in the midpoint method will be reduced by fctor of 8. Perhps you feel completely lost in the work tht led up to lemm 11.7. The wise wy to red something like this is to first focus on the generl ide tht ws used: Consider the error (11.30) nd replce both f (x) nd f ( 1/ ) by its qudrtic Tylor polynomils with reminders. If we do this, number of terms cncel out nd we re left with (11.33). At this point we use some stndrd techniques tht give us the finl inequlity. Once you hve n overview of the derivtion, you should check tht the detils re correct nd mke sure you understnd ech step. 11.7. Globl error nlysis Above, we nlysed the error on one subintervl. Now we wnt to see wht hppens when we dd together the contributions from mny subintervls; it should not surprise us tht this my ffect the error. 54
We consider the generl cse where we hve prtition tht divides [,b] into n subintervls, ech of width h. On ech subintervl we use the simple midpoint rule tht we nlysed in the previous section, I = The totl error is then f (x)dx = I I mid = n i=1 n i=1 ( xi xi x i 1 f (x)dx n f (x i 1/ )h. i=1 x i 1 f (x)dx f (x i 1/ )h But the expression inside the prenthesis is just the locl error on the intervl [x i 1, x i ]. We therefore hve n ( xi ) I I mid = f (x)dx f (x i 1/ )h i=1 x i 1 n xi f (x)dx f (x i 1/ )h x i 1 i=1 n 7h 3 i=1 4 M i (11.36) where M i is the mximum of f (x) on the intervl [xi 1, x i ]. To simiplify the expression (11.36), we extend the mximum on [x i 1, x i ] to ll of [,b]. This will usully mke the mximum lrger, so for ll i we hve M i = mx f (x) mx f (x) = M. x [x i 1,x i ] Now we cn simplify (11.36) further, i=1 x [,b] n 7h 3 n 4 M 7h 3 i 4 M = 7h3 nm. (11.37) 4 i=1 Here, we need one finl little observtion. Recll tht h = (b )/n, so hn = b. If we insert this in (11.37), we obtin our min result. ). Theorem 11.8. Suppose tht f nd its first two derivtives re continuous on the intervl [,b], nd tht the integrl of f on [,b] is pproximted by the midpoint rule with n subintervls of equl width, I = f (x)dx I mid = n f (x i 1/ )h. i=1 55
Then the error is bounded by where x i 1/ = + (i 1/)h. I I mid (b ) 7h 4 mx f (x) (11.38) x [,b] This confirms the error behviour tht we sw in exmple 11.6: If h is reduced by fctor of, the error is reduced by fctor of = 4. One notble omission in our discussion of the midpoint method is round-off error, which ws mjor concern in our study of numericl differentition. The good news is tht round-off error is not usully problem in numericl integrtion. The only sitution where round-off my cuse problems is when the vlue of the integrl is 0. In such sitution we my potentilly dd mny numbers tht sum to 0, nd this my led to cncelltion effects. However, this is so rre tht we will not discuss it here. You should be wre of the fct tht the error estimte (11.38) is not the best possible in tht the constnt 7/4 cn be reduced to 1/4, but then the derivtion becomes much more complicted. 11.7.3 Estimting the step length The error estimte (11.38) lets us ply stndrd gme: If someone demnds tht we compute n integrl with error smller thn ɛ, we cn find step length h tht gurntees tht we meet this demnd. To mke sure tht the error is smller thn ɛ, we enforce the inequlity which we cn esily solve for h, (b ) 7h 4 mx f (x) ɛ x [,b] 4ɛ h 7(b )M, M = mx f (x). x [,b] This is not quite s simple s it my look since we will hve to estimte M, the mximum vlue of the second derivtive. This cn be difficult, but in some cses it is certinly possible, see exercise 4. 11.7.4 A detiled lgorithm Algorithm 11.5 describes the midpoint method, but lcks lot of detil. In this section we give more detiled lgorithm. 56
Whenever we compute quntity numericlly, we should try nd estimte the error, otherwise we hve no ide of the qulity of our computtion. A stndrd wy to do this for numericl integrtion is to compute the integrl for decresing step lengths, nd stop the computtions when difference between two successive pproximtions is less thn the tolernce. More precisely, we choose n initil step length h 0 nd compute the pproximtions I mid (h 0 ), I mid (h 1 ),..., I mid (h k ),..., where h k = h 0 / k. Suppose I mid (h k ) is our ltest pproximtion. Then we estimte the reltive error by the number I mid (h k ) I mid (h k 1 I mid (h k ) nd stop the computtions if this is smller thn ɛ. To void potentil division by zero, we use the test I mid (h k ) I mid (h k 1 ) ɛ I mid (h k ). As lwys, we should lso limit the number of pproximtions tht re computed. Algorithm 11.9. Suppose the function f, the intervl [,b], the length n 0 of the intitil prtition, positive tolernce ɛ < 1, nd the mximum number of itertions M re given. The following lgorithm will compute sequence of pproximtions to f (x)dx by the midpoint rule, until the estimted reltive error is smller thn ɛ, or the mximum number of computed pproximtions rech M. The finl pproximtion is stored in I. n := n 0 ; h := (b )/n; I := 0; x := + h/; for k := 1,,..., n I := I + f (x); x := x + h; j := 1; I := h I ; bser r := I ; while j < M nd bser r > ɛ I j := j + 1; I p := I ; 57
x 0 x 1 () x 0 x 1 x x 3 x 4 x 5 (b) Figure 11.8. The trpezoid rule with one subintervl () nd five subintervls (b). n := n; h := (b )/n; I := 0; x := + h/; for k := 1,,..., n I := I + f (x); x := x + h; I := h I ; bser r := I I p ; Note tht we compute the first pproximtion outside the min loop. This is necessry in order to hve meningful estimtes of the reltive error (the first time we rech the top of the while loop we will lwys get pst the condition). We store the previous pproximtion in I p so tht we cn estimte the error. In the coming sections we will describe two other methods for numericl integrtion. These cn be implemented in lgorithms similr to Algorithm 11.9. In fct, the only difference will be how the ctul pproximtion to the integrl is computed. 11.8 The trpezoid rule The midpoint method is bsed on very simple polynomil pproximtion to the function f to be integrted on ech subintervl; we simply use constnt pproximtion by interpolting the function vlue t the middle point. We re now going to consider nturl lterntive; we pproximte f on ech subintervl with the secnt tht interpoltes f t both ends of the subintervl. The sitution is shown in figure 11.8. The pproximtion to the integrl is 58
the re of the trpezoidl figure under the secnt so we hve f (x)dx f () + f (b) (b ). (11.39) To get good ccurcy, we will hve to split [,b] into subintervls with prtition nd use this pproximtion on ech subintervl, see figure 11.8b. If we hve uniform prtition {x i } n with step length h, we get the pproximtion i=0 f (x)dx = n i=1 xi x i 1 f (x)dx n f (x i 1 ) + f (x i ) h. (11.40) We should lwys im to mke our computtionl methods s efficient s possible, nd in this cse n improvement is possible. Note tht on the intervl [x i 1, x i ] we use the function vlues f (x i 1 ) nd f (x i ), nd on the next intervl we use the vlues f (x i ) nd f (x i+1 ). All function vlues, except the first nd lst, therefore occur twice in the sum on the right in (11.40). This mens tht if we implement this formul directly we do lot of unnecessry work. From the explntion bove the following observtion follows. i=1 Observtion 11.30 (Trpezoid rule). Suppose we hve function f defined on n intervl [,b] nd prtition {x i } n of [.b]. If we pproximte f by its i=0 secnt on ech subintervl nd pproximte the integrl of f by the integrl of the resulting piecewise liner pproximtion, we obtin the pproximtion ( f () + f (b) f (x)dx h n 1 + i=1 ) f (x i ). (11.41) In the formul (11.41) there re no redundnt function evlutions. 11.8.1 Locl error nlysis Our next step is to nlyse the error in the trpezoid method. We follow the sme recipe s for the midpoint method nd use Tylor series. Becuse of the similrities with the midpoint method, we will skip some of the detils. We first study the error in the pproximtion (11.39) where we only hve one secnt. In this cse the error is given by f () + f (b) f (x)dx (b ), (11.4) 59
nd the first step is to expnd the function vlues f (x) nd f (b) in Tylor series bout, f (x) = f () + (x )f (x ) () + f (ξ 1 ), f (b) = f () + (b )f (b ) () + f (ξ ), where ξ 1 (, x) nd ξ (,b). The integrtion of the Tylor series for f (x) we did in (11.3) so we just quote the result here, (b ) f (x)dx = f ()(b ) + f () + 1 If we insert the Tylor series for f (b) we obtin (x ) f (ξ 1 )dx. f () + f (b) (b ) (b ) = f ()(b ) + f (b )3 () + f (ξ ). 4 If we insert these expressions into the error (11.4), the first two terms cncel ginst ech other, nd we obtin f () + f (b) f (x)dx (b ) 1 (x ) f (b )3 (ξ 1 )dx f (ξ ) 4 These expressions cn be simplified just like in (11.34) nd (11.35), nd this yields f () + f (b) f (x)dx (b ) M 6 (b )3 + M 4 (b )3. Let us sum this up in lemm. Lemm 11.31. Let f be continuous function whose first two derivtives re continuous on the intervl [.b]. The the error in the trpezoid rule, with only one line segment on [,b], is bounded by where M = mx x [,b] f (x). f () + f (b) f (x)dx (b ) 5M 1 (b )3, This lemm is completely nlogous to lemm 11.7 which describes the locl error in the midpoint method. We prticulrly notice tht even though the trpezoid rule uses two vlues of f, the error estimte is slightly lrger thn the 60
estimte for the midpoint method. The most importnt feture is the exponent on (b ), which tells us how quickly the error goes to 0 when the intervl width is reduced, nd from this point of view the two methods re the sme. In other words, we hve gined nothing by pproximting f by liner functions insted of constnt. This does not men tht the trpezoid rule is bd, it rther mens tht the midpoint rule is unusully good. 11.8. Globl error We cn find n expression for the globl error in the trpezoid rule in exctly the sme wy s we did for the midpoint rule, so we skip the proof. We sum everything up in theorem bout the trpezoid rule. Theorem 11.3. Suppose tht f nd its first two derivtives re continuous on the intervl [,b], nd tht the integrl of f on [,b] is pproximted by the trpezoid rule with n subintervls of equl width h, ( f () + f (b) I = f (x)dx I tr p = h Then the error is bounded by n 1 + i=1 ) f (x i ). I Itr p 5h (b ) 1 mx f (x). (11.43) x [,b] As we mentioned when we commented on the midpoint rule, the error estimtes tht we obtin re not best possible in the sense tht it is possible to derive better error estimtes (using other techniques) with smller constnts. In the cse of the trpezoid rule, the constnt cn be reduced from 5/1 to 1/1. However, the fct remins tht the trpezoid rule is disppointing method compred to the midpoint rule. 11.9 Simpson s rule The finl method for numericl integrtion tht we consider is Simpson s rule. This method is bsed on pproximting f by prbol on ech subintervl, which mkes the derivtion bit more involved. The error nlysis is essentilly the sme s before, but becuse the expressions re more complicted, it pys off to pln the nlysis better. You my therefore find the mteril in this section more chllenging thn the tretment of the other two methods, nd should 61
mke sure tht you hve good understnding of the error nlysis for these methods before you strt studying section 11.9.. 11.9.1 Deriving Simpson s rule As for the other methods, we derive Simpson s rule in the simplest cse where we use one prbol on ll of [,b]. We find the polynomil p tht interpoltes f t, 1/ = ( + b)/ nd b, nd pproximte the integrl of f by the integrl of p. We could find p vi the Newton form, but in this cse it is esier to use the Lgrnge form. Another simiplifiction is to first contstruct Simpson s rule in the cse where = 1, 1/ = 0, nd b = 1, nd then use this to generlise the method. The Lgrnge form of the polyomil tht interpoltes f t 1, 0, 1, is given by x(x 1) (x + 1)x p (x) = f ( 1) f (0)(x + 1)(x 1) + f (1), nd it is esy to check tht the interpoltion conditions hold. To integrte p, we must integrte ech of the three polynomils in this expression. For the first one we hve 1 1 Similrly, we find 1 x(x 1)dx = 1 1 1 1 1(x x)dx = 1 (x + 1)(x 1)dx = 4 3, 1 1 1 [ 1 3 x3 1 x] 1 (x + 1)x dx = 1 3. 1 = 1 3. On the intervl [ 1,1], Simpson s rule therefore gives the pproximtion 1 1 f (x)dx 1 3( f ( 1) + 4f (0) + f (1) ). (11.44) To obtin n pproximtion on the intervl [,b], we use stndrd technique. Suppose tht x nd y re relted by x = (b ) y + 1 +. (11.45) We see tht if y vries in the intervl [ 1,1], then x will vry in the intervl [,b]. We re going to use the reltion (11.45) s substitution in n integrl, so we note tht d x = (b )d y/. We therefore hve 1 ( ) b f (x)dx = f (y + 1) + d y = b 1 f (y)d y, (11.46) 1 1 6
x 0 x 1 x () x 0 x 1 x x 3 x 4 x 5 x 6 (b) Figure 11.9. Simpson s rule with one subintervl () nd three subintervls (b). where ( ) f b (y) = f (y + 1) +. To determine n pproximiton to the intgerl of f on the intervl [ 1,1], we cn use Simpson s rule (11.44). The result is 1 1 f (y)d y 1 3 ( f ( 1) + 4f (0) + f (1)) = 1 ( f () + 4f 3 ( + b ) ) + f (b), since the reltion in (11.45) mps 1 to, the midpoint 0 to ( + b)/, nd the right endpoint b to 1. If we insert this in (11.46), we obtin Simpson s rule for the generl intervl [,b], see figure 11.9. In prctice, we will usully divide the intervl [, b] into smller ntervls nd use Simpson s rule on ech subintervl, see figure 11.9b. Observtion 11.33. Let f be n integrble function on the intervl [,b]. If f is interpolted by qudrtic polynomil p t the points, ( + b)/ nd b, then the integrl of f cn be pproximted by the integrl of p, f (x)dx p (x)dx = b ( f () + 4f 6 ( + b ) ) + f (b). (11.47) We could just s well hve derived this formul by doing the interpoltion directly on the intervl [,b], but then the lgebr becomes quite messy. 63
11.9. Locl error nlysis The next step is to nlyse the error. We follow the usul recipe nd perform Tylor expnsions of f (x), f ( ( + b)/ ) nd f (b) round the left endpoint. However, those Tylor expnsions become long nd tedious, so we re gong to see how we cn predict wht hppens. For this, we define the error function, E(f ) = f (x)dx b ( f () + 4f 6 ( + b ) ) + f (b). (11.48) Note tht if f (x) is polynomil of degree, then the interpolnt p will be exctly the sme s f. Since the lst term in (11.48) is the integrl of p, we see tht the error E(f ) will be 0 for ny qudrtic polynomil. We cn check this by clculting the error for the three functions 1, x, nd x, E(1) = (b ) b (1 + 4 + 1) = 0, 6 E(x) = 1 [ x ] b b ( + 4 + b + b 6 E(x ) = 1 3 [ x 3 ] b b 6 ) ( ( + b) + 4 + b 4 = 1 3 (b3 3 ) b 3 ( + b + b ) = 1 ( b 3 3 ( b + b + b 3 3 b b ) ) 3 = 0. ) = 1 (b (b )(b + ) ) = 0, Let us lso check wht hppens if f (x) = x 3, E(x 3 ) = 1 [ x 4 ] b 4 b ) ( 3 ( + b)3 + 4 + b 3 6 8 = 1 ( b 4 4 ) b ( 3 + 3 + 3 3 b + 3b + b 3 ) + b 3 4 6 = 1 4 (b4 4 ) b 3 ( 3 + 3 b + 3b + b 3) 6 = 0. The fct tht the error is zero when f (x) = x 3 comes s plesnt surprise; by the construction of the method we cn only expect the error to be 0 for qudrtic polynomils. The bove computtions men tht E ( c 0 + c 1 x + c x + c 3 x 3) = c 0 E(1) + c 1 E(x) + c E(x ) + c 3 E(x 3 ) = 0 64
for ny rel numbers {c i } 3, i.e., the error is 0 whenever f is cubic polynomil. i=0 Lemm 11.34. Simpson s rule is exct for cubic polynomils. To obtin n error estimte, suppose tht f is generl function which cn be expnded in Tylor polynomil of degree 3 bout with reminder Then we see tht f (x) = T 3 (f ; x) + R 3 (f ; x). E(f ) = E ( T 3 (f ) + R 3 (f ) ) = E ( T 3 (f ) ) + E ( R 3 (f ) ) = E ( R 3 (f ) ). The second equlity follows from simple properties of the integrl nd function evlutions, while the lst equlity follows becuse the error in Simpson s rule is 0 for cubic polynomils. The Lgrnge form of the error term is given by where ξ x (, x). We then find E ( R 3 (f ; x) ) = 1 4 = 1 4 (x )4 R 3 (f ; x) = f (i v) (ξ x ), 4 (x ) 4 f (i v) (ξ x )dx b ) (b )4 (0 + 4 6 4 16 f (i v) (b )4 (ξ (+b)/ ) + f (i v) (ξ b ) 4 (x ) 4 f (i v) (b )5 ( (ξ x )dx f (i v) (ξ (+b)/ ) + 4f (i v) (ξ b ) ), 576 where ξ 1 = ξ (+b)/ nd ξ = ξ b (the error is 0 t x = ). If we tke bsolute vlues, use the tringle inequlity, the stndrd trick of replcing the function vlues by mxim over the whole intervl [,b], nd evlute the integrl, we obtin (b )5 (b )5 E(f ) M + (M + 4M). 5 4 576 This gives the following estimte of the error. Lemm 11.35. If f is continuous nd hs continuous derivtives up to order 4 on the intervl [,b], the error in Simpson s rule is bounded by E(f ) 49 880 (b )5 mx f (i v) (x). x [,b] 65
x 0 x 1 x x 3 x 4 x 5 x 6 Figure 11.10. Simpson s rule with three subintervls. We note tht the error in Simpson s rule depends on (b ) 5, while the error in the midpoint rule nd trpezoid rule depend on (b ) 3. This mens tht the error in Simpson s rule goes to zero much more quickly thn for the other two methods when the width of the intervl [,b] is reduced. More precisely, reduction of h by fctor of will reduce the error by fctor of 3. As for the other two methods the constnt 49/880 is not best possible; it cn be reduced to 1/880 by using other techniques. 11.9.3 Composite Simpson s rule Simpson s rule is used just like the other numericl integrtion techniques we hve studied: The intervl over which f is to be integrted is split into subintervls, nd Simpson s rule is pplied on neighbouring pirs of intervls, see figure 11.10. In other words, ech prbol is defined over two subintervls which mens tht the totl number of subintervls must be even nd the number of given vlues of f must be odd. If the prtition is (x i ) n i=0 with x i = + ih, Simpson s rule on the intervl [x i, x i ] is xi x i f (x)dx h 3 ( f (xi + 4f (x i 1 ) + f (x i ) ). The pproximtion to the totl integrl is therefore f (x)dx h 3 n ( (f (xi + 4f (x i 1 ) + f (x i ) ). i=1 66
In this sum we observe tht the right endpoint of one subintervl becomes the left endpoint of the following subintervl to the right. Therefore, if this is implemented directly, the function vlues t the points with n even subscript will be evluted twice, except for the extreme endpoints nd b which only occur once in the sum. We cn therefore rewrite the sum in wy tht voids these redundnt evlutions. Observtion 11.36. Suppose f is function defined on the intervl [, b], nd let {x i } n be uniform prtition of [,b] with step length h. The composite i=0 Simpson s rule pproximtes the integrl of f by f (x)dx h ( n 1 n f () + f (b) + f (x i ) + 4 3 i=1 i=1 ) f (x i 1 ). With the midpoint rule, we computed sequence of pproximtions to the integrl by successively hlving the width of the subintervls. The sme is often done with Simpson s rule, but then cre should be tken to void unnecessry function evlutions since ll the function vlues computed t one step will lso be used t the next step, see exercise 7. 11.9.4 The error in the composite Simpson s rule The pproch we used to deduce the globl error for the midpoint rule, cn lso be used for Simpson s rule, see theorem 11.8. The following theorem sums this up. Theorem 11.37. Suppose tht f nd its first four derivtives re continuous on the intervl [, b], nd tht the integrl of f on [, b] is pproximted by Simpson s rule with n subintervls of equl width h. Then the error is bounded by E(f ) 49h 4 (b ) 880 mx x [,b] f (i v) (x). (11.49) 11.10 Summry In this chpter we hve derived three methods for numericl differentition nd three methods for numericl integrtion. All these methods nd their error nlyses my seem rther overwhelming, but they ll follow common thred: 67
Procedure 11.38. The following is generl procedure for deriving numericl methods for differentition nd integrtion: 1. Interpolte the function f by polynomil p t suitble points.. Approximte the derivtive or integrl of f by the derivtive or integrl of p. This mkes it possible to express the pproximtion in terms of function vlues of f. 3. Derive n estimte for the error by expnding the function vlues (other thn the one t ) in Tylor series with reminders. 4D. For numericl differentition, derive n estimte of the round-off error by ssuming tht the reltive errors in the function vlues re bounded by ɛ. By minimising the totl error, n optiml step length h cn be determined. 4I. For numericl integrtion, the globl error cn esily be derived from the locl error using the technique leding up to theorem 11.8. Perhps the most delicte prt of the bove procedure is to choose the degree of the Tylor polynomils. This is discussed in exercise 6. It is procedure 11.38 tht is the min content of this chpter. The individul methods re importnt in prctice, but lso serve s exmples of how this procedure is implemented, nd should show you how to derive other methods more suitble for your specific needs. Exercises 11.1 ) Write progrm tht implements the numericl differentition method f f ( + h) f ( h) (), h nd test the method on the function f (x) = e x t = 1. b) Determine the optiml vlue of h given in section 11.3.4 which minimises the totl error. Use ɛ = 7 10 17. c) Use your progrm to determine the optiml vlue h of experimentlly. d) Use the optiml vlue of h tht you found in (c) to determine better vlue for ɛ in this specific exmple. 68
11. Repet exercise 1, but compute the second derivtive using the pproximtion f () f ( + h) f () + f ( h) h. In (b) you should use the vlue of h given in observtion 11.3. 11.3 ) Suppose tht we wnt to derive method for pproximting the derivtive of f t which hs the form f () c 1 f ( h) + c f ( + h), c 1,c R. We wnt the method to be exct when f (x) = 1 nd f (x) = x. Use these conditions to determine c 1 nd c. b) Show tht the method in () is exct for ll polynomils of degree 1, nd compre it to the methods we hve discussed in this chpter. c) Use the procedure in () nd (b) to derive method for pproximting the second derivtive of f, f () c 1 f ( h) + c f () + c 3 f ( + h), c 1,c,c 3 R, by requiring tht the method should be exct when f (x) = 1, x nd x. d) Show tht the method in (c) is exct for ll qudrtic polynomils. 11.4 ) Write progrm tht implements the midpoint method s in lgorithm 11.9 nd test it on the integrl 1 e x dx = e 1. 0 b) Determine vlue of h tht gurntees tht the bsolute error is smller thn 10 10. Run your progrm nd check wht the ctul error is for this vlue of h. (You my hve to djust lgorithm 11.9 slightly nd print the bsolute error.) 11.5 Repet exercise 4, but use Simpson s rule insted of the midpoint method. 11.6 It my sometimes be difficult to judge how mny terms to include in the Tylor series used in the nlysis of numericl methods. In this exercise we re going to see how this cn be done. We use the numericl pproximtion in section 11.3 for our experiments. f f ( + h) f ( h) () h ) Do the sme derivtion s section 11.3., but include only two terms in the Tylor series (plus reminder). Wht hppens? b) Do the sme derivtion s section 11.3., but include four terms in the Tylor series (plus reminder). Wht hppens now? 11.7 When h is hlved in the trpezoid method, some of the function vlues used with step length h/ re the sme s those used for step length h. Derive formul for the trpezoid method with step length h/ tht mkes use of the function vlues tht were computed for step length h. 69