Solving partial differential equations (PDEs)

Solving partial differential equations (PDEs) Hans Fangor Engineering and te Environment University of Soutampton United Kingdom fangor@soton.ac.uk May 3, 2012 1 / 47

Outline I 1 Introduction: wat are PDEs? 2 Computing derivatives using finite differences 3 Diffusion equation 4 Recipe to solve 1d diffusion equation 5 Boundary conditions, numerics, performance 6 Finite elements 7 Summary 2 / 47

Tis lecture tries to compress several years of material into 45 minutes as lecture notes and code available for download at ttp://www.soton.ac.uk/ fangor/teacing/comp6024 3 / 47

Wat are partial differential equations (PDEs) Ordinary Differential Equations (ODEs) one independent variable, for example t in d 2 x dt 2 = k m x often te indepent variable t is te time solution is function x(t) important for dynamical systems, population growt, control, moving particles Partial Differential Equations (ODEs) multiple independent variables, for example t, x and y in u t = D ( 2 u x 2 + 2 u y 2 solution is function u(t, x, y) important for fluid dynamics, cemistry, electromagnetism,..., generally problems wit spatial resolution ) 4 / 47

2d Diffusion equation ( ) u 2 t = D u x + 2 u 2 y 2 u(t, x, y) is te concentration [mol/m 3 ] t is te time [s] x is te x-coordinate [m] y is te y-coordinate [m] D is te diffusion coefficient [m 2 /s] Also known as Fick s second law. Te eat equation as te same structure (and u represents te temperature). Example: ttp://www.youtube.com/watc?v=wc6kj5yswkq 5 / 47

Examples of PDEs Can Hilliard Equation (pase separation) Fluid dynamics (including ocean and atmosperic models, plasma pysics, gas turbine and aircraft modelling) Structural mecanics and vibrations, superconductivity, micromagnetics,... 6 / 47

Computing derivatives using finite differences 7 / 47

Overview Motivation: We need derivatives of functions for example for optimisation and root finding algoritms Not always is te function analytically known (but we are usually able to compute te function numerically) Te material presented ere forms te basis of te finite-difference tecnique tat is commonly used to solve ordinary and partial differential equations. Te following slides sow te forward difference tecnique te backward difference tecnique and te central difference tecnique to approximate te derivative of a function. We also derive te accuracy of eac of tese metods. 8 / 47

Te 1st derivative (Possible) Definition of te derivative (or differential operator d dx ) f (x) df f(x + ) f(x) (x) = lim dx 0 Use difference operator to approximate differential operator f (x) = df f(x + ) f(x) (x) = lim dx 0 f(x + ) f(x) can now compute an approximation of f (x) simply by evaluating f (twice). Tis is called te forward difference because we use f(x) and f(x + ). Important questions: How accurate is tis approximation? 9 / 47

Accuracy of te forward difference Formal derivation using te Taylor series of f around x f(x + ) = n f (n) (x) n! n=0 Rearranging for f (x) = f(x) + f (x) + 2 f (x) 2! f (x) = f(x + ) f(x) 2 f (x) 2! f (x) = 1 = = + 3 f (x) 3! 3 f (x) 3! +...... ( f(x + ) f(x) 2 f (x) 3 f (x)... 2! 3! f(x + ) f(x) f(x + ) f(x) 2 f (x) 2! 3 f (x) 3! f (x) 2! 2 f (x) 3!...... ) 10 / 47

Accuracy of te forward difference (2) f (x) = f (x) = f(x + ) f(x) f(x + ) f(x) f (x) 2 f (x)... } 2! {{ 3! } E forw () + E forw () Terefore, te error term E forw () is E forw () = f (x) 2! Can also be expressed as f (x) = 2 f (x) 3! f(x + ) f(x)... + O() 11 / 47

Te 1st derivative using te backward difference Anoter definition of te derivative (or differential operator d ) dx df f(x) f(x ) (x) = lim dx 0 Use difference operator to approximate differential operator df f(x) f(x ) (x) = lim dx 0 f(x) f(x ) Tis is called te backward difference because we use f(x) and f(x ). How accurate is te backward difference? 12 / 47

Accuracy of te backward difference Formal derivation using te Taylor Series of f around x f(x ) = f(x) f (x) + 2 f (x) 2! Rearranging for f (x) f (x) = f(x) f(x ) + 2 f (x) 2! f (x) = 1 = = 3 f (x) 3! 3 f (x) 3!... +... ( f(x) f(x ) + 2 f (x) 3 f ) (x)... 2! 3! f(x) f(x ) f(x) f(x ) + 2 f (x) 2! 3 f (x) 3! + f (x) 2! 2 f (x) 3!...... 13 / 47

Accuracy of te backward difference (2) f (x) = f (x) = f(x) f(x ) f(x) f(x ) Terefore, te error term E back () is E back () = f (x) 2! Can also be expressed as f (x) = + f (x) 2 f (x)... } 2! {{ 3! } E back () + E back () (1) 2 f (x) 3! f(x) f(x )... + O() 14 / 47

Combining backward and forward differences (1) Te approximations are forward: backward f (x) = f (x) = E forw () = f (x) 2! E back () = f (x) 2! f(x + ) f(x) f(x) f(x ) 2 f (x) 3! 2 f (x) 3! 3 f (x) 4! + 3 f (x) 4! + E forw () (2) + E back () (3) 4 f (x) 5! 4 f (x) 5!... +... Add equations (2) and (3) togeter, ten te error cancels partly! 15 / 47

Combining backward and forward differences (2) Add tese lines togeter f (x) = f (x) = f(x + ) f(x) + E forw () f(x) f(x ) + E back () 2f f(x + ) f(x ) (x) = + E forw () + E back () Adding te error terms: E forw () + E back () = 2 2 f (x) 2 4 f (x)... 3! 5! Te combined (central) difference operator is wit f (x) = f(x + ) f(x ) 2 E cent () = 2 f (x) 3! 4 f (x) 5! + E cent ()... 16 / 47

Central difference Can be derived (as on previous slides) by adding forward and backward difference Can also be interpreted geometrically by defining te differential operator as df f(x + ) f(x ) (x) = lim dx 0 2 and taking te finite difference form df f(x + ) f(x ) (x) dx 2 Error of te central difference is only O( 2 ), i.e. better tan forward or backward difference It is generally te case tat symmetric differences are more accurate tan asymmetric expressions. 17 / 47

Example (1) Using forward difference to estimate te derivative of f(x) = exp(x) f (x) f forw = Numerical example: f(x + ) f(x) = exp(x + ) exp(x) = 0.1, x = 1 f (1) f forw exp(1.1) exp(1) (1.0) = = 2.8588 0.1 Exact answers is f (1.0) = exp(1) = 2.71828 (Central diff: f cent(1.0) = exp(1+0.1) exp(1 0.1) 0.2 = 2.72281) 18 / 47

Example (2) Comparison: forward difference, central difference and exact derivative of f(x) = exp(x) df/dx(x) 140 120 100 80 60 40 20 Approximations of df/dx for f(x)=exp(x) forward =1 central =1 forward =0.0001 exact 0 0 1 2 3 4 5 x 19 / 47

Summary Can approximate derivatives of f numerically using only function evaluations of f size of step very important central differences as smallest error term name formula error forward f (x) = f(x+) f(x) O() backward f (x) = f(x) f(x ) O() central f (x) = f(x+) f(x ) 2 O( 2 ) 20 / 47

Appendix: source to compute figure on page 19 I EPS =1 # very large EPS to provoke inaccuracy def forwarddiff (f,x, = EPS ): # df / dx = ( f(x+)-f(x) )/ + O() return ( f(x+)-f(x) )/ def backwarddiff (f,x, = EPS ): # df / dx = ( f(x)-f(x -) )/ + O() return ( f(x)-f(x-) )/ def centraldiff (f,x, = EPS ): # df / dx = (f(x+) - f(x - ))/ + O( ^2) return (f(x+) - f(x- ))/(2* ) if name == " main ": # create example plot import pylab import numpy as np a=0 # left and b=5 # rigt limits for x N =11 # steps 21 / 47

Appendix: source to compute figure on page 19 II def f(x): """ Our test funtion wit convenient property tat df / dx = f """ return np.exp (x) xs=np. linspace (a,b, N) forward = [] forward_small_ = [] central = [] for x in xs: forward. append ( forwarddiff (f, x) ) central. append ( centraldiff (f, x) ) forward_small_. append ( forwarddiff (f,x, =1e -4)) pylab. figure ( figsize =(6,4)) pylab. axis ([a,b,0, np.exp (b )]) pylab. plot (xs, forward, ^,label = forward =%g % EPS ) pylab. plot (xs, central, x,label = central =%g % EPS ) pylab. plot (xs, forward_small_, o, label = forward =% g % 1e -4) xsfine = np. linspace (a,b, N *100) 22 / 47

Appendix: source to compute figure on page 19 III pylab. plot ( xsfine,f( xsfine ), -,label = exact ) pylab. grid () pylab. legend ( loc = upper left ) pylab. xlabel ("x") pylab. ylabel ("df/dx(x)") pylab. title (" Approximations of df/dx for f(x)= exp (x)") pylab. plot () pylab. savefig ( central -and - forward - difference. pdf ) pylab. sow () 23 / 47

Note: Euler s (integration) metod derivation using finite difference operator Use forward difference operator to approximate differential operator dy y(x + ) y(x) (x) = lim dx 0 y(x + ) y(x) Cange differential to difference operator in dy dx f(x, y) = dy y(x + ) y(x) dx f(x, y) y(x + ) y(x) = y i+1 = y i + f(x i, y i ) = f(x, y) Euler s metod (for ODEs) can be derived from te forward difference operator. 24 / 47

Note: Newton s (root finding) metod derivation from Taylor series We are looking for a root, i.e. we are looking for a x so tat f(x) = 0. We ave an initial guess x 0 wic we refine in subsequent iterations:. x i+1 = x i i were i = f(x i) f (x i ). (4) Tis equation can be derived from te Taylor series of f around x. Suppose we guess te root to be at x and x + is te actual location of te root (so is unknown and f(x + ) = 0): f(x + ) = f(x) + f (x) +... 0 = f(x) + f (x) +... = 0 f(x) + f (x) f(x) f (x). (5) 25 / 47

Te diffusion equation 26 / 47

Diffusion equation Te 2d operator 2 + 2 is called te Laplace x 2 y 2 operator, so tat we can also write ( ) u 2 t = D x + 2 = D u 2 y 2 Te diffusion equation (wit constant diffusion coefficient D) reads u = D u were te Laplace t operator depends on te number d of spatial dimensions d = 1: = 2 x 2 d = 2: = 2 + 2 x 2 y 2 d = 3: = 2 + 2 + 2 x 2 y 2 z 2 27 / 47

1d Diffusion equation u t = D 2 u x 2 In one spatial dimension, te diffusion equation reads u t = D 2 u x 2 Tis is te equation we will use as an example. Let s assume an initial ( concentration ) u(x, t 0 ) = 1 σ exp (x xmean)2 wit x 2π σ 2 mean = 0 and widt σ = 0.5. u(x,t_0) 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 (x,t 0 ) 0.0 6 4 2 0 2 4 6 x 28 / 47

1d Diffusion eqn u t = D 2 u x 2, time integration I Let us assume tat we ave some way of computing at time t 0 and let s call tis g(x, t 0 ), i.e. D 2 u x 2 We like to solve g(x, t 0 ) D 2 u(x, t 0 ) x 2 u(x, t) t = g(x, t 0 ) to compute u(x, t 1 ) at some later time t 1. Use finite difference time integration sceme: Introduce a time step size so tat t 1 = t 0 +. 29 / 47

1d Diffusion eqn u t = D 2 u x 2, time integration II Cange differential operator to forward difference operator g(x, t 0 ) = u(x, t) t u(x, t 0 + ) u(x, t 0 ) = lim (6) 0 u(x, t 0 + ) u(x, t 0 ) (7) Rearrange to find u(x, t 1 ) u(x, t 0 + ) gives u(x, t 1 ) u(x, t 0 ) + g(x, t 0 ) We can generalise tis using t i = t 0 + i to read u(x, t i+1 ) u(x, t i ) + g(x, t i ) (8) If we can find g(x, t i ), we can compute u(x, t i+1 ) 30 / 47

1d Diffusion eqn u t = D 2 u x 2, spatial part I u t = D 2 u = g(x, t) x2 Need to compute g(x, t) = D 2 u(x,t) for a given u(x, t). x 2 Can ignore te time dependence ere, and obtain Recall tat g(x) = D 2 u(x) x 2. 2 u x = 2 x u x and we tat know ow to compute u using central x differences. 31 / 47

Second order derivatives from finite differences I Recall central difference equation for first order derivative df f(x + ) f(x ) (x) dx 2 will be more convenient to replace by 1 2 : df dx (x) f(x + 1) f(x 1) 2 2 32 / 47

Second order derivatives from finite differences II Apply te central difference equation twice to obtain d2 f dx 2 : d 2 f dx (x) = d df 2 dx dx (x) d ( f(x + 1 ) f(x 1) ) 2 2 dx = 1 ( d (x dx f + 12 ) d (x dx f 12 )) 1 ( ) f(x + ) f(x) f(x) f(x ) f(x + ) 2f(x) + f(x ) = (9) 2 33 / 47

Recipe to solve u t = D 2 u x 2 1 Discretise solution u(x, t) into discrete values 2 u i j u(x j, t i ) were x j x 0 + j x and t i t 0 + i t. 3 Start wit time iteration i = 0 4 Need to know configuration u(x, t i ). 5 Ten compute g(x, t i ) = D 2 u using finite differences x 2 (9). 6 Ten compute u(x, t i+1 ) based on g(x, t i ) using (8) 7 increase i to i + 1, ten go back to 5. 34 / 47

A sample solution u t = D 2 u x 2, I import numpy as np import matplotlib. pyplot as plt import matplotlib. animation as animation a, b= -5,5 # size of box N = 51 # number of subdivisions x=np. linspace (a,b, N) # positions of subdivisions =x[1] - x [0] # discretisation stepsize in x - direction def def total (u): """ Computes total number of moles in u. """ return ((b-a)/ float (N)* np.sum (u)) gaussdistr (mean, sigma,x): """ Return gauss distribution for given numpy array x """ return 1./( sigma *np. sqrt (2* np.pi ))* np.exp ( -0.5*( x- mean )**2/ sigma **2) # starting configuration for u(x, t0 ) u = gaussdistr ( mean =0., sigma =0.5, x=x) 35 / 47

A sample solution u t = D 2 u x 2, II def compute_g ( u, D, ): """ given a u(x,t) in array, compute g(x,t )= D*d ^2 u/ dx ^2 using central differences wit spacing, and return g(x,t ). """ d2u_dx2 = np. zeros (u. sape,np. float ) for i in range (1, len (u ) -1): d2u_dx2 [i] = (u[i +1] - 2*u[i]+u[i -1])/ **2 # special cases at boundary : assume Neuman boundary # conditions, i. e. no cange of u over boundary # so tat u [0] - u [ -1]=0 and tus u [ -1]= u [0] i=0 d2u_dx2 [i] = (u[i +1] - 2*u[i]+u[i ])/ **2 # same at oter end so tat u[n -1] - u[ N ]=0 # and tus u[n ]= u[n -1] i= len (u)-1 d2u_dx2 [i] = (u[i] - 2*u[i]+u[i -1])/ **2 return D* d2u_dx2 def advance_time ( u, g, dt ): """ Given te array u, te rate of cange array g, and a timestep dt, compute te solution for u after t, using simple Euler metod. """ u = u +dt* g 36 / 47

A sample solution u t = D 2 u x 2, III return u # sow example, quick and dirtly, lots of global variables dt = 0.01 # step size or time stepsbeforeupdatinggrap = 20 # plotting is slow D = 1. # Diffusion coefficient stepsdone = 0 # keep track of iterations def do_steps (j, nsteps = stepsbeforeupdatinggrap ): """ Function called by FuncAnimation class. Computes nsteps iterations, i.e. carries forward solution from u(x, t_i ) to u(x, t_ {i+ nsteps }). """ global u, stepsdone for i in range ( nsteps ): g = compute_g ( u, D, ) u = advance_time ( u, g, dt) stepsdone += 1 time_passed = stepsdone * dt print (" stepsdone =%5d, time =%8 gs, total (u )=%8 g" % ( stepsdone, time_passed, total (u ))) l. set_ydata ( u) # update data in plot fig1. canvas. draw () # redraw te canvas 37 / 47

A sample solution u t = D 2 u x 2, IV return l, fig1 = plt. figure () # setup animation l,= plt. plot (x,u, b-o ) # plot initial u(x,t) # ten compute solution and animate line_ani = animation. FuncAnimation ( fig1, do_steps, range (10000)) plt. sow () 38 / 47

Boundary conditions I For ordinary differential equations (ODEs), we need to know te initial value(s) to be able to compute a solution. For partial differential equations (PDEs), we need to know te initial values and extra information about te beaviour of te solution u(x, t) at te boundary of te spatial domain (i.e. at x = a and x = b in tis example). Commonly used boundary conditions are Diriclet boundary conditions: fix u(a) = c to some constant. Would correspond ere to some mecanism tat keeps te concentration u at position x = a constant. 39 / 47

Boundary conditions II Neuman boundary conditions: fix te cange of u across te boundary, i.e. u (a) = c. x For positive/negative c tis corresponds to an imposed concentration gradient. For c = 0, tis corresponds to conservation of te atoms in te solution: as te gradient across te boundary cannot cange, no atoms can move out of te box. (Used in our program on slide 35) 40 / 47

Numerical issues Te time integration sceme we use is explicit because we ave an explicit equation tat tells us ow to compute u(x, t i+1 ) based on u(x, t i ) (equation (8) on slide 30) An implicit sceme would compute u(x, t i+1 ) based on u(x, t i ) and on u(x, t i+1 ). Te implicit sceme is more complicated as it requires solving an additional equation system just to find u(x, t i+1 ) but allows larger step sizes t for te time. Te explicit integration sceme becomes quickly unstable if t is too large. t depends on te cose spatial discretisation x. 41 / 47

Performance issues Our sample code is (nearly) as slow as possible interpreted language explicit for loops enforced small step size from explicit sceme Solutions: Refactor for-loops into matrix operations and use (compiled) matrix library (numpy for small systems, use scipy.sparse for larger systems) Use library function to carry out time integration (will use implicit metod if required), for example scipy.integrate.odeint. 42 / 47

Finite Elements Anoter widely spread way of solving PDEs is using so-called finite elements. Matematically, te solution u(x) for a problem like 2 u x 2 = f(x) is written as u(x) = N u i φ i (x) (10) i=1 were eac u i is a number (a coefficient), and eac φ i (x) a known function of space. Te φ i are called basis or sape functions. Eac φ i is normally cosen to be zero for nearly all x, and to be non-zero close to a particular node in te finite element mes. By substitution (10) into te PDE, a matrix system can be obtained, wic if solved provides te coefficients u i, and tus te solution. 43 / 47

Finite Elements vs Finite differences Finite differences are matematically muc simpler and for simple geometries (suc as cuboids) easier to program Finite elements ave greater flexibility in te sape of te domain, te specification and implementation of boundary conditions is easier but te basic matematics and code is more complicated. 44 / 47

Practical observation on time integration Usually, we solve te spatial part of a PDE using some discretisation sceme suc as finite differences and finite elements). Tis results in a set of coupled ordinary differential equations (were time is te independent variable). Can tink of tis as one ODE for every cube from our discretisation. Tis temporal part is ten solved using time integration scemes for (systems of) ordinary differential equations. 45 / 47

Summary Partial differential equations important in many contexts If no analytical solution known, use numerics. Discretise te problem troug finite differences (replace differential wit difference operator, corresponds to copping space and time in little cuboids) finite elements (project solution on localised basis functions, often used wit tetraedral meses) related metods (finite volumes, mesless metods). Finite elements and finite difference calculations are standard tools in many areas of engineering, pysics, cemistry, but increasingly in oter fields. 46 / 47

cangeset: 53:a22b7f13329e tag: tip user: Hans Fangor [MBP13] <fangor@soton.ac.uk> date: Fri Dec 16 10:57:15 2011 +0000 47 / 47