Numerical Methods for Differential Equations

Numerical Methods for Differential Equations Course objectives and preliminaries Gustaf Söderlind and Carmen Arévalo Numerical Analysis, Lund University Textbooks: A First Course in the Numerical Analysis of Differential Equations, by Arieh Iserles and Introduction to Mathematical Modelling with Differential Equations, by Lennart Edsberg c Gustaf Söderlind, Numerical Analysis, Mathematical Sciences, Lund University, 2010-11 Numerical Methods for Differential Equations p.1/26

Chapter 0: Objectives and Preliminaries Course structure and objectives What is Numerical Analysis? The four principles of Numerical Analysis Numerical Methods for Differential Equations p.2/26

1. Course structure and objectives To solve a differential equation analytically we look for a differentiable function that satisfies the equation Large, complex and nonlinear systems cannot be solved analytically Numerical Methods for Differential Equations p.3/26

1. Course structure and objectives To solve a differential equation analytically we look for a differentiable function that satisfies the equation Large, complex and nonlinear systems cannot be solved analytically Instead, we compute numerical solutions with standard methods and software Numerical Methods for Differential Equations p.3/26

Course objectives Understand correspondence between physics and mathematical equations Learn basic numerical techniques for solving differential equations Understand mathematics numerics interaction, and how to match numerical method to mathematical properties Construct elementary MATLAB programs for differential equations Numerical Methods for Differential Equations p.4/26

Course topics IVP Initial value problems first-order equations; higher-order equations; systems of differential equations Numerical Methods for Differential Equations p.5/26

Course topics IVP Initial value problems first-order equations; higher-order equations; systems of differential equations BVP Boundary value problems two-point boundary value problems; Sturm Liouville eigenvalue problems Numerical Methods for Differential Equations p.5/26

Course topics IVP Initial value problems first-order equations; higher-order equations; systems of differential equations BVP Boundary value problems two-point boundary value problems; Sturm Liouville eigenvalue problems PDE Partial differential equations advection equation; wave equation diffusion equation; Numerical Methods for Differential Equations p.5/26

Course topics IVP Initial value problems first-order equations; higher-order equations; systems of differential equations BVP Boundary value problems two-point boundary value problems; Sturm Liouville eigenvalue problems PDE Partial differential equations advection equation; wave equation diffusion equation; Applications in all three areas Numerical Methods for Differential Equations p.5/26

Initial value problems. Examples A first-order equation A simple equation without elementary analytical solution dy dt = y e t2, y(0) = y 0 Numerical Methods for Differential Equations p.6/26

Initial value problems. Examples A first-order equation A simple equation without elementary analytical solution A second-order equation Motion of a pendulum θ + g L sin θ = 0, θ(0) = θ 0, θ(0) = θ0 where θ(t) is the angle, g is the gravitational constant and L is the pendulum length Numerical Methods for Differential Equations p.6/26

Initial value problems. Examples A first-order equation A simple equation without elementary analytical solution A second-order equation Pendulum equation A system of equations Predator prey model ẏ 1 = k 1 y 1 k 2 y 1 y 2 ẏ 2 = k 3 y 1 y 2 k 4 y 2 where y 1 (t) is the population of the prey species and y 2 (t) is the population of the predator species at time t Numerical Methods for Differential Equations p.6/26

Boundary value problems. Examples Second-order two-point BVP The electrostatic potential u(r) between two concentric metal spheres satisfies d 2 u dr 2 + 2 r du dr = 0, u(r 1) = V 1, u(r 2 ) = 0 at the distance r from the center; R 1 and R 2 are the radii of the two spheres Numerical Methods for Differential Equations p.7/26

Boundary value problems. Examples Second-order two-point BVP The electrostatic potential u(r) between two concentric metal spheres satisfies d 2 u dr 2 + 2 r du dr = 0, u(r 1) = V 1, u(r 2 ) = 0 at the distance r from the center; R 1 and R 2 are the radii of the two spheres A Sturm Liouville eigenproblem Euler buckling of column y + λy = 0, y (0) = 0, y(1) = 0 Find eigenvalues λ and eigenfunctions y Numerical Methods for Differential Equations p.7/26

Some application areas in ODEs Initial value problems mechanics M q = F(q) electrical circuits C v = I(v) chemical reactions ċ = f(c) Boundary value problems materials u = M(u)/EI microphysics 2m ψ = E eigenmodes u + λu = 0 Numerical Methods for Differential Equations p.8/26

Partial differential equations. Examples The Poisson equation u xx + u yy = f(x,y) subject to u(x,y) = g(x,y) for x = 0 x = 1 y = 0 y = 1 is an elliptic PDE modelling e.g. the displacement an elastic membrane under load f u(x, y) of Numerical Methods for Differential Equations p.9/26

Partial differential equations. Examples The Poisson equation u xx + u yy = f(x,y) The Diffusion equation u t = u xx subject to u(x, 0) = f(x), x [0,a] u(0,t) = c 1, u(a,t) = c 2, t [0,b] is a parabolic PDE modelling e.g. the time evolution of temperature u(x, t) in an insulated rod with constant temperatures c 1 and c 2 at its ends, and initial temperature distribution f(x) Numerical Methods for Differential Equations p.9/26

Partial differential equations. Examples The Poisson equation u xx + u yy = f(x,y) The Diffusion equation u t = u xx The Wave equation u tt = u xx subject to u(0,t) = 0, u(a,t) = 0 for t [0,b] u(x, 0) = f(x) for x [0,a] u t (x, 0) = g(x) for x (0,a) is a hyperbolic PDE e.g. modelling the displacement u(x, t) of a vibrating elastic string fixed at x = 0 and x = a Numerical Methods for Differential Equations p.9/26

2. What is Numerical Analysis? Categories of mathematical problems Category Algebra Analysis linear computable not computable nonlinear not computable not computable Numerical Methods for Differential Equations p.10/26

2. What is Numerical Analysis? Categories of mathematical problems Category Algebra Analysis linear computable not computable nonlinear not computable not computable Algebra Only finite constructions Numerical Methods for Differential Equations p.10/26

2. What is Numerical Analysis? Categories of mathematical problems Category Algebra Analysis linear computable not computable nonlinear not computable not computable Algebra Analysis Only finite constructions Limits, derivatives, integrals etc. (transfinite) Numerical Methods for Differential Equations p.10/26

2. What is Numerical Analysis? Categories of mathematical problems Category Algebra Analysis linear computable not computable nonlinear not computable not computable Algebra Analysis Only finite constructions Limits, derivatives, integrals etc. (transfinite) Computable number of operations The exact solution can be obtained in a finite Numerical Methods for Differential Equations p.10/26

What is Numerical Analysis? Category Algebra Analysis linear computable not computable nonlinear not computable not computable With numerical methods, problems from all four categories can be solved: Numerical analysis aims to construct and analyze quantitative methods for the automatic computation of approximate solutions to mathematical problems. Goal: Construction of mathematical software Numerical Methods for Differential Equations p.11/26

3. The four principles of numerical analysis There are four basic principles in numerical analysis. Every computational method is constructed from these principles Discretization To replace a continuous problem by a discrete one Linear algebra tools, including polynomials All techniques that are computable Iteration To apply a computation repeatedly (convergence) Linearization To approximate a nonlinear problem by a linear one Numerical Methods for Differential Equations p.12/26

Discretization. From function to vector 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 A continuous function is sampled at discrete points Numerical Methods for Differential Equations p.13/26

Discretization... Reduces the amount of information in a problem to a finite set Select a subset Ω N = {x 0,x 1,...,x N } of distinct points from the interval Ω = [0, 1] Continuous function f(x) is replaced by vector F = {f(x k )} N 0 The subset Ω N is called the grid The vector F is called a grid function or discrete function Discretization yields a computable problem. Approximations are computed only at a finite set of grid points Numerical Methods for Differential Equations p.14/26

Discretization. Approximation of derivatives Definition of derivative f (x) = lim x 0 f(x + x) f(x) x Take x = x k+1 x k finite and approximate f (x k ) f(x k + x) f(x k ) x = F k+1 F k x k+1 x k How accurate is this approximation? Numerical Methods for Differential Equations p.15/26

Error and accuracy Expand in Taylor series f(x k + x) f(x k ) x f(x k) + xf (x k ) + O( x 2 ) f(x k ) x The error is proportional to x = f (x k ) + O( x) Try this formula for f(x) = e x at x k = 1 for various x Numerical Methods for Differential Equations p.16/26

Numerical differentiation Error in numerically calculated derivative of e x at x = 1 10 0 Error in numerical differentiation 10 2 10 4 log error 10 6 10 8 10 10 10 12 10 15 10 10 10 5 10 0 log step size Log error vs. log step size log x Numerical Methods for Differential Equations p.17/26

Numerical differentiation Error in numerically calculated derivative of e x at x = 1 10 0 Error in numerical differentiation 10 2 10 4 log error 10 6 10 8 10 10 10 12 10 15 10 10 10 5 10 0 log step size Log error vs. log step size log x Numerical Methods for Differential Equations p.18/26

Another approach Use a symmetric approximation f(x k + x) f(x k x) 2 x = F k+1 F k 1 x k+1 x k 1 = f (x k ) + O( x 2 ) The approximation is 2nd order accurate because error is x 2 Whenever you have a power law, plot in log log diagram r C x p log r log C + p log x This is a straight line with slope p, easy to identify Numerical Methods for Differential Equations p.19/26

Numerical 2nd order differentiation Error in numerically calculated derivative of e x at x = 1 10 0 Error in numerical differentiation 10 2 10 4 log error 10 6 10 8 10 10 10 12 10 15 10 10 10 5 10 0 log step size Log error vs. log step size log x Numerical Methods for Differential Equations p.20/26

Numerical 2nd order differentiation Error in numerically calculated derivative of e x at x = 1 10 0 Error in numerical differentiation 10 2 10 4 log error 10 6 10 8 10 10 10 12 10 15 10 10 10 5 10 0 log step size Log error vs. log step size log x Numerical Methods for Differential Equations p.21/26

1st and 2nd order comparison Error in numerically calculated derivative of e x at x = 1 10 0 Error in numerical differentiation 10 2 10 4 log error 10 6 10 8 10 10 10 12 10 15 10 10 10 5 10 0 log step size 2nd order approximation has much higher accuracy! Numerical Methods for Differential Equations p.22/26

Discretization of a differential equation Problem Solve ẏ = qy numerically on [0,T], with y(0) = y 0 The analytical solution is a function The numerical solution is a vector y k y(t k ), where t k = kt/n y(t) = y 0 e qt y = {y k } N k=0 with Method Approximate derivative by difference quotient, using step size t = t k+1 t k = T/N Numerical Methods for Differential Equations p.23/26

Discretization of a differential equation... Discretization method Approximate derivative by finite difference quotient y k+1 y k t k+1 t k ẏ(t k ) to get the linear algebraic equation system (k = 0, 1,...N) y k+1 y k t k+1 t k = qy k or y k+1 = ( 1 + qt N ) y k y N = ( 1 + qt N ) N y 0 Numerical Methods for Differential Equations p.24/26

Discretization of a differential equation... NOTE The numerical approximation at time T is y N = ( 1 + qt N ) N y 0 But ( lim 1 + qt N N ) N = e qt The numerical approximation converges to the exact solution as N Numerical Methods for Differential Equations p.25/26

Main course objective The course centers on discretization methods for differential equations of all types We will also encounter the other three principles of numerical analysis as we go linear algebra methods and polynomial techniques; iterative methods; and linearization Numerical Methods for Differential Equations p.26/26