Numerical Differentiation and Integration

Transcription

1 Numerical Differentiation and Integration Numerical Differentiation Finite Differences Interpolating Polynomials Taylor Series Expansion Richardson Extrapolation Numerical Integration Basic Numerical Integration Improved Numerical Integration Trapezoidal, Simpson s Rules Rhomberg Integration ITCS 4133/5133: Numerical Comp. Methods 1 Numerical Differentiation and Integration

2 Numerical Differentiation and Integration Many engineering applications require numerical estimates of derivatives of functions Especially true, when analytical solutions are not possible Differentiation: Use finite differences Integration (definite integrals): Weighted sum of function values at specified points (area under the curve). ITCS 4133/5133: Numerical Comp. Methods 2 Numerical Differentiation and Integration

3 Application:Integral of a Normal Distribution Represented as a Gaussian, a scaled form of f(x) = e x2, very important function in statistics Not easy to determine indefinite integral - use numerical techniques A = b e x2 a ITCS 4133/5133: Numerical Comp. Methods 3 Numerical Differentiation and Integration

4 Application:Integral of a Sinc f(x) = sin(x) x ITCS 4133/5133: Numerical Comp. Methods 4 Numerical Differentiation and Integration

5 Numerical Differentiation:Approach Evaluate the function at two consecutive points, separated by x, of the independent variable, and taking their difference: df(x) dx f(x + x) f(x) x Fit a function to a set of points that define the relationship between the independent and dependent variables, such as an nth order polynomial; then differentiate the polynomial ITCS 4133/5133: Numerical Comp. Methods 5 Numerical Differentiation and Integration

6 Finite Differences Forward Difference Backward Difference df(x) dx df(x) dx f(x + x) f(x) = x f(x) f(x x) = x Two Step Method, Central Difference df(x) dx f(x + x) f(x x) = 2 x ITCS 4133/5133: Numerical Comp. Methods 6 Numerical Differentiation and Integration

7 Finite Differences: Example ITCS 4133/5133: Numerical Comp. Methods 7 Numerical Differentiation and Integration

8 Forward Differences: Derivation Finite difference formulas can be derived from the Taylor series. For h = x i+1 x i, where x i < η < x i+1. f(x + h) = f(x) + hf (x) + h2 2 f (η), f (x i ) = f(x i+1) f(x i ) h h 2 f (η), ITCS 4133/5133: Numerical Comp. Methods 8 Numerical Differentiation and Integration

9 Backward Differences: Derivation For h = x i 1 x i, f(x + h) = f(x) + hf (x) + h2 2 f f(η), f(x i + h) = f(x i + x i 1 x i ) = f(x i 1 ) = f(x i ) + hf (x i ) + h2 2 f f(η) f (x i ) = f(x i 1) f(x i ) h h 2 f (η) where x i 1 < η < x i. ITCS 4133/5133: Numerical Comp. Methods 9 Numerical Differentiation and Integration

10 Central Differences: Derivation Use the next higher order Taylor polynomial, f(x i+1 ) = f(x i ) + hf (x i ) + h2 2 f (x i ) + h3 6 f (η 1 ), f(x i 1 ) = f(x i ) hf (x i ) + h2 2 f (x i ) h3 6 f (η 2 ) with x < η 1 < x + h, x h < η 2 < x Thus, or, f(x i+1 ) f(x i 1 ) = 2hf (x i ) + h3 6 [f (η 1 ) + f (η 2 )] f (x i ) = f(x i+1) f(x i 1 ) 2h h2 6 f (η), x i 1 < η < xi + 1 ITCS 4133/5133: Numerical Comp. Methods 10 Numerical Differentiation and Integration

11 Second Derivatives Thus f(x + h) = f(x) + hf (x) + h2 2 f (x) + h3 3! f (x) + h4 4! f (4) (x)(η 1 ) f(x h) = f(x) hf (x) + h2 2 f (x) h3 3! f (x) + h4 4! f (4) (x)(η 2 ) f(x + h) + f(x h) = 2f(x) + h 2 f (x) + h4 4! [f (4) (x)(η 1 ) + f (4) (x)(η 2 )] f (x) 1 h2[f(x + h) 2f(x) + f(x h)] with truncation error of the O(h 4 ) ITCS 4133/5133: Numerical Comp. Methods 11 Numerical Differentiation and Integration

12 Partial Derivatives Generally, interested in partial derivatives of functions of 2 variables, (x i, y j ), based on a mesh of points. Subscripts denote partial derivatives. ITCS 4133/5133: Numerical Comp. Methods 12 Numerical Differentiation and Integration

13 Partial Derivatives (contd) Laplacian operator: 2 u = u xx + u yy Biharmonic Operator: 4 u = u xxxx + u xxyy + u yyyy ITCS 4133/5133: Numerical Comp. Methods 13 Numerical Differentiation and Integration

14 Using Interpolating Polynomials Given a function in discrete form, the data can be interpolated to fit an nth order polynomial For those functions that are in analytical form, but difficult to differentiate, the function can be discretized and fitted by a polynomial. whose derivative is f(x) = b n x n + b n 1 x n b 1 x + b 0 df(x) dx = nb nx n 1 + (n 1)b n 2 x n b 1 ITCS 4133/5133: Numerical Comp. Methods 14 Numerical Differentiation and Integration

15 Power Series Type Interpolating Polynomial Consider the nth order polynomial passing through (n+1) points; the (n+ 1) coefficients can be uniquely determined. Example f(x) = a 0 + a 1 x + a 2 x 2 Consider f(x i ), f(x i + h), f(x i + 2h) f(x i ) = a 0 + a 1 x i + a 2 x 2 i f(x i + h) = a 0 + a 1 (x i + h) + a 2 (x i + h) 2 f(x i + 2h) = a 0 + a 1 (x i + 2h) + a 2 (x i + 2h) 2 ITCS 4133/5133: Numerical Comp. Methods 15 Numerical Differentiation and Integration

16 Example (contd) For the 3 points, x i = 0, h, 2h, which has the solution, f i = a 0 f i+1 = a 0 + a 1 h + a 2 h 2 f i+2 = a 0 + 2a 1 h + 4a 2 h 2 a 0 a 1 a 2 = f i = f i+2 + 4f i+1 3f i 2h = f i+2 2f i+1 + f i 2h 2 ITCS 4133/5133: Numerical Comp. Methods 16 Numerical Differentiation and Integration

17 Example (contd) f(x) = a 0 + a 1 x + a 2 x 2 f (x i ) = f i = f (x i = 0) = a 1 + 2a 2 x i = a 1 = f i+2 + 4f i+1 3f i 2h Similarly, second derivatives can also be obtained. ITCS 4133/5133: Numerical Comp. Methods 17 Numerical Differentiation and Integration

18 Numerical Integration Integration can be thought of as considering some continuous function f(x) and the area A subtended by it; for instance, within a particular interval A = b a f(x)dx Numerical Integration is needed when f(x) does not have a known analytical solution, or, if f(x) is only defined at discrete points. There are two approaches to a numerical solution: Fitting polynomials to f(x) and integrating using analytical calculus, Area under f(x) can be approximated using geometric shapes defined by adjacent points. ITCS 4133/5133: Numerical Comp. Methods 18 Numerical Differentiation and Integration

19 Interpolation Approach We can derive an nth order polynomial (for instance, Gregory-Newton method), which has the general form as f(x) = b 1 x n + b 2 x n b n x + b n+1 After the b i s are determined, this can be integrated as f(x)dx = b 1x n+1 n b 2x n n + + b nx 2 + b n+1 x 2 which can be solved for, within the limits of the integration ITCS 4133/5133: Numerical Comp. Methods 19 Numerical Differentiation and Integration

20 Basic Numerical Integration (Newton-Cotes Closed Formulas) Trapezoid Rule: Approximates function by a straightline to compute area under curve. b a f(x)dx h 2 [f(x 0) + f(x 1 )] Solution is exact, for polynomials of degree 1, i.e. linear functions. ITCS 4133/5133: Numerical Comp. Methods 20 Numerical Differentiation and Integration

21 Basic Numerical Integration (Newton-Cotes Closed Formulas) Simpson s Rule: Instead of a linear approximation, use a quadratic polynomial: h = b a 2, x 0 = a, x 1 = x 0 + h = b + a 2, x 2 = b Approximate integral is given by b a f(x)dx h 3 [f(x 0) + 4f(x 1 ) + f(x 2 ) ITCS 4133/5133: Numerical Comp. Methods 21 Numerical Differentiation and Integration

22 Basic Simpson s Rule: Examp1e 1 ITCS 4133/5133: Numerical Comp. Methods 22 Numerical Differentiation and Integration

23 Basic Simpson s Rule: Examp1e 2 ITCS 4133/5133: Numerical Comp. Methods 23 Numerical Differentiation and Integration

24 Improved Numerical Integration Improve the accuracy by applying lower order methods repeatedly on several subintervals. Known as composite integration Simpson, Trapezoid rules use equal subintervals; using unequal (adaptive) intervals leads to Gaussian Quadrature methods. ITCS 4133/5133: Numerical Comp. Methods 24 Numerical Differentiation and Integration

25 Composite Trapezoid Rule Approximates the area under the function f(x) by a set of discrete trapezoids fitted between each pair of points of the dependent variable (and the X axis) xn x 1 f(x)dx n 1 i=1 (x i+1 x i ) f(x i+1 + f(x i ) 2 f(x) f(x) x1 x2 x3 x4 If the intervals are the same, h = (b a)/n, we get xn x 1 f(x)dx b a 2n [f(a) + 2f(x 1) f(b)] ITCS 4133/5133: Numerical Comp. Methods 25 Numerical Differentiation and Integration x

26 Composite Trapezoid Rule:Example ITCS 4133/5133: Numerical Comp. Methods 26 Numerical Differentiation and Integration

27 Composite Trapezoid Rule:Algorithm ITCS 4133/5133: Numerical Comp. Methods 27 Numerical Differentiation and Integration

28 Composite Trapezoid Rule: Notes f(x i+1+f(x i ) 2 represents the average height of the trapezoid. Linear approximation between pairs of successive points used; error is proportional to the distance between sample points. Error Error in the trapezoidal can be approximated by Taylor s series x i+1 x i f (x) 2 where f (x) is evaluated at the value of x that maximizes the second derivative between the two sample points. ITCS 4133/5133: Numerical Comp. Methods 28 Numerical Differentiation and Integration

29 Composite Simpson s Rule Uses the same idea as the composite Trapezoid rule, by using multiple sub-intervals to improve the Simpson rule approximation. For 2 subintervals, Simpson s rule has the following form: b a f(x)dx = x2 a f(x)dx + b x 2 f(x)dx h 3 [f(a) + 4f(x 1) + f(x 2 )] + h 3 [f(x 2) + 4f(x 3 ) + f(b)] h 3 [f(a) + 4f(x 1) + 2f(x 2 ) + 4f(x 3 ) + f(b)]) Trapezoidal Rule f(x) f(x) Simpson s Rule ITCS 4133/5133: Numerical Comp. Methods 29 Numerical x a b Differentiation and Integration (a+b)/2

30 Composite Simpson s Rule (contd) For n (n must be even) subintervals, h = (b a)/n we get b a f(x)dx = h 3 [f(a) + 4f(x 1) + 2f(x 2 ) + 4f(x 3 ) + n must be even number of subintervals. 2f(x 4 ) f(x n 2 + 4f(x n 1 ) + f(b) Error Bound: Based on the 4th derivative: Error = (x i+1 x i ) 5 f (4) (x i ) 90 where f (4) (x i ) is evaluated at the value of x that maximizes the 4th derivative between the two sample points. ITCS 4133/5133: Numerical Comp. Methods 30 Numerical Differentiation and Integration

31 Composite Simpson s Rule:Example ITCS 4133/5133: Numerical Comp. Methods 31 Numerical Differentiation and Integration

32 Composite Simpson s Rule:Algorithm ITCS 4133/5133: Numerical Comp. Methods 32 Numerical Differentiation and Integration

33 Romberg Integration Simpson s rule improves on Trapezoid rule with additional evaluations of f(x), required to fit a more accurate polynomial More evaluations will further improve the integral, leading to the Romberg Integration I 01 = b a (f(a) + f(b)) 2 A second estimate can be obtained with subdividing the interval ab into 2 equal intervals is given by I 11 = b a ( f(a) + f(m) + 1 ) 2 f(b) = 1 [ ( I 01 + (b a)f a + b a )] 2 2 ITCS 4133/5133: Numerical Comp. Methods 33 Numerical Differentiation and Integration

34 Romberg Integration(contd.) A third estimate, I 21, using 3 intermediate points m 1, m 2, m 3 is given by I 21 = b a [ f(a) f(m 1) f(m 2) f(m 3) + 1 ] 4 f(b) [ = 1 I 11 + b a 3 ( f a + b a ) ] k k=1,k 2 This leads to the recursive relationship [ I i1 = 1 I i 1,1 + b a 2 i 1 ( f a + b a ) ] k, for i = 1, 2, i 1 2 i k=1,3,5 ITCS 4133/5133: Numerical Comp. Methods 34 Numerical Differentiation and Integration

35 I i1 = 1 2 [ Romberg Integration(contd.) I i 1,1 + b a 2 i 1 2 i 1 k=1,3,5 f ( a + b a ) ] k, for i = 1, 2,... 2 i We can combine these estimates using Richardson s formula, I ij = 4j 1 I i+1,j 1 I i,j 1 4 j 1 1 for i = 0, 1,..., N j + 1, j = 1, 2,... N. The estimates form an upper triangular matrix: I 01 I 02 I 03 I 04 I 0,N 1 I 0,N I 0,N+1 I 11 I 12 I 13. I 1,N 1 I 1,N I 21 I 22.. I 2,N 1 I I N 2,3. I N 1,2 I N1 ITCS 4133/5133: Numerical Comp. Methods 35 Numerical Differentiation and Integration

36 Romberg Integration(contd.) Algorithm (To solve for I 0N Determine I 01 (Trapezoid formula) Determine I i1, for i = 1, 2,... Determine I i2, I i3, etc., using Richardson s formula. ITCS 4133/5133: Numerical Comp. Methods 36 Numerical Differentiation and Integration

37 ITCS 4133/5133: Numerical Comp. Methods 37 Numerical Differentiation and Integration