# Scientific Computing: An Introductory Survey

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Scientific Computing: An Introductory Survey Chapter 10 Boundary Value Problems for Ordinary Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c Reproduction permitted for noncommercial, educational use only. Michael T. Heath Scientific Computing 1 / 45

2 Outline 1 Boundary Value Problems 2 Michael T. Heath Scientific Computing 2 / 45

3 Boundary Value Problems Boundary Values Existence and Uniqueness Conditioning and Stability Side conditions prescribing solution or derivative values at specified points are required to make solution of ODE unique For initial value problem, all side conditions are specified at single point, say t 0 For boundary value problem (BVP), side conditions are specified at more than one point kth order ODE, or equivalent first-order system, requires k side conditions For ODEs, side conditions are typically specified at endpoints of interval [a, b], so we have two-point boundary value problem with boundary conditions (BC) at a and b. Michael T. Heath Scientific Computing 3 / 45

4 Boundary Values Existence and Uniqueness Conditioning and Stability Boundary Value Problems, continued General first-order two-point BVP has form with BC y = f(t, y), g(y(a), y(b)) = 0 a < t < b where f : R n+1 R n and g : R 2n R n Boundary conditions are separated if any given component of g involves solution values only at a or at b, but not both Boundary conditions are linear if they are of form B a y(a) + B b y(b) = c where B a, B b R n n and c R n BVP is linear if ODE and BC are both linear Michael T. Heath Scientific Computing 4 / 45

5 Boundary Values Existence and Uniqueness Conditioning and Stability Example: Separated Linear Boundary Conditions Two-point BVP for second-order scalar ODE u = f(t, u, u ), a < t < b with BC u(a) = α, u(b) = β is equivalent to first-order system of ODEs [ ] [ ] y 1 y = 2, a < t < b f(t, y 1, y 2 ) y 2 with separated linear BC [ ] [ ] 1 0 y1 (a) y 2 (a) [ ] [ ] y1 (b) = y 2 (b) [ ] α β Michael T. Heath Scientific Computing 5 / 45

6 Existence and Uniqueness Boundary Values Existence and Uniqueness Conditioning and Stability Unlike IVP, with BVP we cannot begin at initial point and continue solution step by step to nearby points Instead, solution is determined everywhere simultaneously, so existence and/or uniqueness may not hold For example, with BC u = u, u(0) = 0, 0 < t < b u(b) = β with b integer multiple of π, has infinitely many solutions if β = 0, but no solution if β 0 Michael T. Heath Scientific Computing 6 / 45

7 Boundary Values Existence and Uniqueness Conditioning and Stability Existence and Uniqueness, continued In general, solvability of BVP y = f(t, y), a < t < b with BC g(y(a), y(b)) = 0 depends on solvability of algebraic equation g(x, y(b; x)) = 0 where y(t; x) denotes solution to ODE with initial condition y(a) = x for x R n Solvability of latter system is difficult to establish if g is nonlinear Michael T. Heath Scientific Computing 7 / 45

8 Boundary Values Existence and Uniqueness Conditioning and Stability Existence and Uniqueness, continued For linear BVP, existence and uniqueness are more tractable Consider linear BVP y = A(t) y + b(t), a < t < b where A(t) and b(t) are continuous, with BC B a y(a) + B b y(b) = c Let Y (t) denote matrix whose ith column, y i (t), called ith mode, is solution to y = A(t)y with initial condition y(a) = e i, ith column of identity matrix Then BVP has unique solution if, and only if, matrix is nonsingular Q B a Y (a) + B b Y (b) Michael T. Heath Scientific Computing 8 / 45

9 Boundary Values Existence and Uniqueness Conditioning and Stability Existence and Uniqueness, continued Assuming Q is nonsingular, define Φ(t) = Y (t) Q 1 and Green s function { Φ(t)Ba Φ(a)Φ G(t, s) = 1 (s), a s t Φ(t)B b Φ(b)Φ 1 (s), t < s b Then solution to BVP given by y(t) = Φ(t) c + b a G(t, s) b(s) ds This result also gives absolute condition number for BVP κ = max{ Φ, G } Michael T. Heath Scientific Computing 9 / 45

10 Boundary Values Existence and Uniqueness Conditioning and Stability Conditioning and Stability Conditioning or stability of BVP depends on interplay between growth of solution modes and boundary conditions For IVP, instability is associated with modes that grow exponentially as time increases For BVP, solution is determined everywhere simultaneously, so there is no notion of direction of integration in interval [a, b] Growth of modes increasing with time is limited by boundary conditions at b, and growth (in reverse) of decaying modes is limited by boundary conditions at a For BVP to be well-conditioned, growing and decaying modes must be controlled appropriately by boundary conditions imposed Michael T. Heath Scientific Computing 10 / 45

11 For IVP, initial data supply all information necessary to begin numerical solution method at initial point and step forward from there For BVP, we have insufficient information to begin step-by-step numerical method, so numerical methods for solving BVPs are more complicated than those for solving IVPs We will consider four types of numerical methods for two-point BVPs Shooting Finite difference Collocation Galerkin Michael T. Heath Scientific Computing 11 / 45

12 Boundary Value Problems In statement of two-point BVP, we are given value of u(a) If we also knew value of u (a), then we would have IVP that we could solve by methods discussed previously Lacking that information, we try sequence of increasingly accurate guesses until we find value for u (a) such that when we solve resulting IVP, approximate solution value at t = b matches desired boundary value, u(b) = β Michael T. Heath Scientific Computing 12 / 45

13 , continued For given γ, value at b of solution u(b) to IVP with initial conditions u = f(t, u, u ) u(a) = α, u (a) = γ can be considered as function of γ, say g(γ) Then BVP becomes problem of solving equation g(γ) = β One-dimensional zero finder can be used to solve this scalar equation Michael T. Heath Scientific Computing 13 / 45

14 Example: Consider two-point BVP for second-order ODE with BC u = 6t, 0 < t < 1 u(0) = 0, u(1) = 1 For each guess for u (0), we will integrate resulting IVP using classical fourth-order Runge-Kutta method to determine how close we come to hitting desired solution value at t = 1 For simplicity of illustration, we will use step size h = 0.5 to integrate IVP from t = 0 to t = 1 in only two steps First, we transform second-order ODE into system of two [ ] [ ] first-order ODEs y y (t) = 1 (t) y2 y 2 (t) = 6t Michael T. Heath Scientific Computing 14 / 45

15 Example, continued We first try guess for initial slope of y 2 (0) = 1 y (1) = y (0) + h 6 (k 1 + 2k 2 + 2k 3 + k 4 ) [ ] 0 = ([ ] [ ] [ ] [ ]) [ ] = [ ] y (2) = ([ ] [ ] [ ] [ ]) [ ] = So we have hit y 1 (1) = 2 instead of desired value y 1 (1) = 1 Michael T. Heath Scientific Computing 15 / 45

16 Example, continued We try again, this time with initial slope y 2 (0) = 1 [ ] y (1) 0 = ([ ] [ ] [ ] [ ] = [ ] y (2) = ([ ] [ ] [ ] 0 = 2 [ ] [ ]) [ ]) 2 6 So we have hit y 1 (1) = 0 instead of desired value y 1 (1) = 1, but we now have initial slope bracketed between 1 and 1 Michael T. Heath Scientific Computing 16 / 45

17 Example, continued We omit further iterations necessary to identify correct initial slope, which turns out to be y 2 (0) = 0 [ ] y (1) 0 = ([ ] [ ] [ ] [ ]) [ ] = [ ] y (2) = ([ ] [ ] [ ] [ ]) [ ] 1 = 3 So we have indeed hit target solution value y 1 (1) = 1 Michael T. Heath Scientific Computing 17 / 45

18 Example, continued < interactive example > Michael T. Heath Scientific Computing 18 / 45

19 Multiple Shooting Boundary Value Problems Simple shooting method inherits stability (or instability) of associated IVP, which may be unstable even when BVP is stable Such ill-conditioning may make it difficult to achieve convergence of iterative method for solving nonlinear equation Potential remedy is multiple shooting, in which interval [a, b] is divided into subintervals, and shooting is carried out on each Requiring continuity at internal mesh points provides BC for individual subproblems Multiple shooting results in larger system of nonlinear equations to solve Michael T. Heath Scientific Computing 19 / 45

20 Finite difference method converts BVP into system of algebraic equations by replacing all derivatives with finite difference approximations For example, to solve two-point BVP with BC u = f(t, u, u ), u(a) = α, a < t < b u(b) = β we introduce mesh points t i = a + ih, i = 0, 1,..., n + 1, where h = (b a)/(n + 1) We already have y 0 = u(a) = α and y n+1 = u(b) = β from BC, and we seek approximate solution value y i u(t i ) at each interior mesh point t i, i = 1,..., n Michael T. Heath Scientific Computing 20 / 45

21 , continued We replace derivatives by finite difference approximations such as u (t i ) y i+1 y i 1 2h u (t i ) y i+1 2y i + y i 1 h 2 This yields system of equations y i+1 2y i + y i 1 h 2 = f ( t i, y i, y ) i+1 y i 1 2h to be solved for unknowns y i, i = 1,..., n System of equations may be linear or nonlinear, depending on whether f is linear or nonlinear Michael T. Heath Scientific Computing 21 / 45

22 , continued For these particular finite difference formulas, system to be solved is tridiagonal, which saves on both work and storage compared to general system of equations This is generally true of finite difference methods: they yield sparse systems because each equation involves few variables Michael T. Heath Scientific Computing 22 / 45

23 Example: Consider again two-point BVP with BC u = 6t, 0 < t < 1 u(0) = 0, u(1) = 1 To keep computation to minimum, we compute approximate solution at one interior mesh point, t = 0.5, in interval [0, 1] Including boundary points, we have three mesh points, t 0 = 0, t 1 = 0.5, and t 2 = 1 From BC, we know that y 0 = u(t 0 ) = 0 and y 2 = u(t 2 ) = 1, and we seek approximate solution y 1 u(t 1 ) Michael T. Heath Scientific Computing 23 / 45

24 Example, continued Replacing derivatives by standard finite difference approximations at t 1 gives equation ( y 2 2y 1 + y 0 h 2 = f t 1, y 1, y ) 2 y 0 2h Substituting boundary data, mesh size, and right hand side for this example we obtain or so that 1 2y (0.5) 2 = 6t 1 4 8y 1 = 6(0.5) = 3 y(0.5) y 1 = 1/8 = Michael T. Heath Scientific Computing 24 / 45

25 Example, continued In a practical problem, much smaller step size and many more mesh points would be required to achieve acceptable accuracy We would therefore obtain system of equations to solve for approximate solution values at mesh points, rather than single equation as in this example < interactive example > Michael T. Heath Scientific Computing 25 / 45

26 Collocation method approximates solution to BVP by finite linear combination of basis functions For two-point BVP with BC u = f(t, u, u ), u(a) = α, a < t < b u(b) = β we seek approximate solution of form n u(t) v(t, x) = x i φ i (t) i=1 where φ i are basis functions defined on [a, b] and x is n-vector of parameters to be determined Michael T. Heath Scientific Computing 26 / 45

27 Popular choices of basis functions include polynomials, B-splines, and trigonometric functions Basis functions with global support, such as polynomials or trigonometric functions, yield spectral method Basis functions with highly localized support, such as B-splines, yield finite element method Michael T. Heath Scientific Computing 27 / 45

28 , continued To determine vector of parameters x, define set of n collocation points, a = t 1 < < t n = b, at which approximate solution v(t, x) is forced to satisfy ODE and boundary conditions Common choices of collocation points include equally-spaced points or Chebyshev points Suitably smooth basis functions can be differentiated analytically, so that approximate solution and its derivatives can be substituted into ODE and BC to obtain system of algebraic equations for unknown parameters x Michael T. Heath Scientific Computing 28 / 45

29 Example: Consider again two-point BVP with BC u = 6t, 0 < t < 1, u(0) = 0, u(1) = 1 To keep computation to minimum, we use one interior collocation point, t = 0.5 Including boundary points, we have three collocation points, t 0 = 0, t 1 = 0.5, and t 2 = 1, so we will be able to determine three parameters As basis functions we use first three monomials, so approximate solution has form v(t, x) = x 1 + x 2 t + x 3 t 2 Michael T. Heath Scientific Computing 29 / 45

30 Example, continued Derivatives of approximate solution function with respect to t are given by v (t, x) = x 2 + 2x 3 t, v (t, x) = 2x 3 Requiring ODE to be satisfied at interior collocation point t 2 = 0.5 gives equation or v (t 2, x) = f(t 2, v(t 2, x), v (t 2, x)) 2x 3 = 6t 2 = 6(0.5) = 3 Boundary condition at t 1 = 0 gives equation x 1 + x 2 t 1 + x 3 t 2 1 = x 1 = 0 Boundary condition at t 3 = 1 gives equation x 1 + x 2 t 3 + x 3 t 2 3 = x 1 + x 2 + x 3 = 1 Michael T. Heath Scientific Computing 30 / 45

31 Example, continued Solving this system of three equations in three unknowns gives x 1 = 0, x 2 = 0.5, x 3 = 1.5 so approximate solution function is quadratic polynomial u(t) v(t, x) = 0.5t + 1.5t 2 At interior collocation point, t 2 = 0.5, we have approximate solution value u(0.5) v(0.5, x) = Michael T. Heath Scientific Computing 31 / 45

32 Example, continued < interactive example > Michael T. Heath Scientific Computing 32 / 45

33 Boundary Value Problems Rather than forcing residual to be zero at finite number of points, as in collocation, we could instead minimize residual over entire interval of integration For example, for scalar Poisson equation in one dimension, u = f(t), a < t < b with homogeneous BC u(a) = 0, u(b) = 0 subsitute approximate solution u(t) v(t, x) = into ODE and define residual r(t, x) = v (t, x) f(t) = n x i φ i (t) i=1 n xi φ (t) f(t) Michael T. Heath Scientific Computing 33 / 45

34 , continued Using least squares method, we can minimize F (x) = 1 2 b a r(t, x) 2 dt by setting each component of its gradient to zero This yields symmetric system of linear algebraic equations Ax = b, where a ij = b a φ j (t)φ i (t) dt, b i = b whose solution gives vector of parameters x a f(t)φ i (t) dt Michael T. Heath Scientific Computing 34 / 45

35 , continued More generally, weighted residual method forces residual to be orthogonal to each of set of weight functions or test functions w i, b a r(t, x)w i (t) dt = 0, i = 1,..., n This yields linear system Ax = b, where now a ij = b a φ j (t)w i (t) dt, b i = b whose solution gives vector of parameters x a f(t)w i (t) dt Michael T. Heath Scientific Computing 35 / 45

36 , continued Matrix resulting from weighted residual method is generally not symmetric, and its entries involve second derivatives of basis functions Both drawbacks are overcome by Galerkin method, in which weight functions are chosen to be same as basis functions, i.e., w i = φ i, i = 1,..., n Orthogonality condition then becomes or b a b a r(t, x)φ i (t) dt = 0, v (t, x)φ i (t) dt = b a f(t)φ i (t) dt, i = 1,..., n i = 1,..., n Michael T. Heath Scientific Computing 36 / 45

37 , continued b a Degree of differentiability required can be reduced using integration by parts, which gives v (t, x)φ i (t) dt = v (t)φ i (t) b a b a v (t)φ i(t) dt = v (b)φ i (b) v (a)φ i (a) b a v (t)φ i(t) dt Assuming basis functions φ i satisfy homogeneous boundary conditions, so φ i (0) = φ i (1) = 0, orthogonality condition then becomes b a v (t)φ i(t) dt = b a f(t)φ i (t) dt, i = 1,..., n Michael T. Heath Scientific Computing 37 / 45

38 , continued This yields system of linear equations Ax = b, with a ij = b a φ j(t)φ i(t) dt, b i = b whose solution gives vector of parameters x a f(t)φ i (t) dt A is symmetric and involves only first derivatives of basis functions Michael T. Heath Scientific Computing 38 / 45

39 To keep computation to minimum, we again use same three mesh points, but now they become knots in piecewise linear Michael polynomial T. Heath approximation Scientific Computing 39 / 45 Boundary Value Problems Example: Consider again two-point BVP with BC u = 6t, 0 < t < 1, u(0) = 0, u(1) = 1 We will approximate solution by piecewise linear polynomial, for which B-splines of degree 1 ( hat functions) form suitable set of basis functions

40 Example, continued Thus, we seek approximate solution of form u(t) v(t, x) = x 1 φ 1 (t) + x 2 φ 2 (t) + x 3 φ 3 (t) From BC, we must have x 1 = 0 and x 3 = 1 To determine remaining parameter x 2, we impose Galerkin orthogonality condition on interior basis function φ 2 and obtain equation 3 ( 1 ) 1 φ j(t)φ 2(t) dt x j = 6tφ 2 (t) dt j=1 0 or, upon evaluating these simple integrals analytically 2x 1 4x 2 + 2x 3 = 3/2 0 Michael T. Heath Scientific Computing 40 / 45

41 Example, continued Substituting known values for x 1 and x 3 then gives x 2 = 1/8 for remaining unknown parameter, so piecewise linear approximate solution is u(t) v(t, x) = 0.125φ 2 (t) + φ 3 (t) We note that v(0.5, x) = Michael T. Heath Scientific Computing 41 / 45

42 Example, continued More realistic problem would have many more interior mesh points and basis functions, and correspondingly many parameters to be determined Resulting system of equations would be much larger but still sparse, and therefore relatively easy to solve, provided local basis functions, such as hat functions, are used Resulting approximate solution function is less smooth than true solution, but nevertheless becomes more accurate as more mesh points are used < interactive example > Michael T. Heath Scientific Computing 42 / 45

43 Eigenvalue Problems Standard eigenvalue problem for second-order ODE has form u = λf(t, u, u ), a < t < b with BC u(a) = α, u(b) = β where we seek not only solution u but also parameter λ Scalar λ (possibly complex) is eigenvalue and solution u is corresponding eigenfunction for this two-point BVP Discretization of eigenvalue problem for ODE results in algebraic eigenvalue problem whose solution approximates that of original problem Michael T. Heath Scientific Computing 43 / 45

44 Example: Eigenvalue Problem Consider linear two-point BVP with BC u = λg(t)u, a < t < b u(a) = 0, u(b) = 0 Introduce discrete mesh points t i in interval [a, b], with mesh spacing h and use standard finite difference approximation for second derivative to obtain algebraic system y i+1 2y i + y i 1 h 2 = λg i y i, i = 1,..., n where y i = u(t i ) and g i = g(t i ), and from BC y 0 = u(a) = 0 and y n+1 = u(b) = 0 Michael T. Heath Scientific Computing 44 / 45

45 Example, continued Assuming g i 0, divide equation i by g i for i = 1,..., n, to obtain linear system Ay = λy where n n matrix A has tridiagonal form 2/g 1 1/g A = 1 1/g 2 2/g 2 1/g.. 2. h /gn 1 2/g n 1 1/g n /g n 2/g n This standard algebraic eigenvalue problem can be solved by methods discussed previously Michael T. Heath Scientific Computing 45 / 45

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

### Numerical Methods for Differential Equations

Numerical Methods for Differential Equations Course objectives and preliminaries Gustaf Söderlind and Carmen Arévalo Numerical Analysis, Lund University Textbooks: A First Course in the Numerical Analysis

### g(y(a), y(b)) = o, B a y(a)+b b y(b)=c, Boundary Value Problems Lecture Notes to Accompany

Lecture Notes to Accompny Scientific Computing An Introductory Survey Second Edition by Michel T Heth Boundry Vlue Problems Side conditions prescribing solution or derivtive vlues t specified points required

### AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS

AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS Revised Edition James Epperson Mathematical Reviews BICENTENNIAL 0, 1 8 0 7 z ewiley wu 2007 r71 BICENTENNIAL WILEY-INTERSCIENCE A John Wiley & Sons, Inc.,

### Introduction to the Finite Element Method

Introduction to the Finite Element Method 09.06.2009 Outline Motivation Partial Differential Equations (PDEs) Finite Difference Method (FDM) Finite Element Method (FEM) References Motivation Figure: cross

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 3 Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### 3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field

3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field 77 3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field Overview: The antiderivative in one variable calculus is an important

### Numerical Methods for Solving Systems of Nonlinear Equations

Numerical Methods for Solving Systems of Nonlinear Equations by Courtney Remani A project submitted to the Department of Mathematical Sciences in conformity with the requirements for Math 4301 Honour s

### Eigenvalues and eigenvectors of a matrix

Eigenvalues and eigenvectors of a matrix Definition: If A is an n n matrix and there exists a real number λ and a non-zero column vector V such that AV = λv then λ is called an eigenvalue of A and V is

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### 3.2 Sources, Sinks, Saddles, and Spirals

3.2. Sources, Sinks, Saddles, and Spirals 6 3.2 Sources, Sinks, Saddles, and Spirals The pictures in this section show solutions to Ay 00 C By 0 C Cy D 0. These are linear equations with constant coefficients

### Numerical Methods for Differential Equations

Numerical Methods for Differential Equations Chapter 1: Initial value problems in ODEs Gustaf Söderlind and Carmen Arévalo Numerical Analysis, Lund University Textbooks: A First Course in the Numerical

### Math 2280 - Assignment 6

Math 2280 - Assignment 6 Dylan Zwick Spring 2014 Section 3.8-1, 3, 5, 8, 13 Section 4.1-1, 2, 13, 15, 22 Section 4.2-1, 10, 19, 28 1 Section 3.8 - Endpoint Problems and Eigenvalues 3.8.1 For the eigenvalue

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Numerical Analysis An Introduction

Walter Gautschi Numerical Analysis An Introduction 1997 Birkhauser Boston Basel Berlin CONTENTS PREFACE xi CHAPTER 0. PROLOGUE 1 0.1. Overview 1 0.2. Numerical analysis software 3 0.3. Textbooks and monographs

### Nonlinear Algebraic Equations Example

Nonlinear Algebraic Equations Example Continuous Stirred Tank Reactor (CSTR). Look for steady state concentrations & temperature. s r (in) p,i (in) i In: N spieces with concentrations c, heat capacities

### Dynamical Systems Analysis II: Evaluating Stability, Eigenvalues

Dynamical Systems Analysis II: Evaluating Stability, Eigenvalues By Peter Woolf pwoolf@umich.edu) University of Michigan Michigan Chemical Process Dynamics and Controls Open Textbook version 1.0 Creative

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Høgskolen i Narvik Sivilingeniørutdanningen STE6237 ELEMENTMETODER. Oppgaver

Høgskolen i Narvik Sivilingeniørutdanningen STE637 ELEMENTMETODER Oppgaver Klasse: 4.ID, 4.IT Ekstern Professor: Gregory A. Chechkin e-mail: chechkin@mech.math.msu.su Narvik 6 PART I Task. Consider two-point

### EE 580 Linear Control Systems VI. State Transition Matrix

EE 580 Linear Control Systems VI. State Transition Matrix Department of Electrical Engineering Pennsylvania State University Fall 2010 6.1 Introduction Typical signal spaces are (infinite-dimensional vector

### Principles of Scientific Computing Nonlinear Equations and Optimization

Principles of Scientific Computing Nonlinear Equations and Optimization David Bindel and Jonathan Goodman last revised March 6, 2006, printed March 6, 2009 1 1 Introduction This chapter discusses two related

### CITY UNIVERSITY LONDON. BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION

No: CITY UNIVERSITY LONDON BEng Degree in Computer Systems Engineering Part II BSc Degree in Computer Systems Engineering Part III PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2005 Date: August

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver. Finite Difference Methods for Partial Differential Equations As you are well aware, most differential equations are much too complicated to be solved by

### Computational Methods. Least Squares Approximation/Optimization

Computational Methods Least Squares Approximation/Optimization Manfred Huber 2010 1 Least Squares Least squares methods are aimed at finding approximate solutions when no precise solution exists Find the

### The Heat Equation. Lectures INF2320 p. 1/88

The Heat Equation Lectures INF232 p. 1/88 Lectures INF232 p. 2/88 The Heat Equation We study the heat equation: u t = u xx for x (,1), t >, (1) u(,t) = u(1,t) = for t >, (2) u(x,) = f(x) for x (,1), (3)

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### 3.7 Non-autonomous linear systems of ODE. General theory

3.7 Non-autonomous linear systems of ODE. General theory Now I will study the ODE in the form ẋ = A(t)x + g(t), x(t) R k, A, g C(I), (3.1) where now the matrix A is time dependent and continuous on some

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12,

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, 2000 45 4 Iterative methods 4.1 What a two year old child can do Suppose we want to find a number x such that cos x = x (in radians). This is a nonlinear

### Nonlinear Algebraic Equations. Lectures INF2320 p. 1/88

Nonlinear Algebraic Equations Lectures INF2320 p. 1/88 Lectures INF2320 p. 2/88 Nonlinear algebraic equations When solving the system u (t) = g(u), u(0) = u 0, (1) with an implicit Euler scheme we have

### Numerical Analysis Introduction. Student Audience. Prerequisites. Technology.

Numerical Analysis Douglas Faires, Youngstown State University, (Chair, 2012-2013) Elizabeth Yanik, Emporia State University, (Chair, 2013-2015) Graeme Fairweather, Executive Editor, Mathematical Reviews,

### ASEN 3112 - Structures. MDOF Dynamic Systems. ASEN 3112 Lecture 1 Slide 1

19 MDOF Dynamic Systems ASEN 3112 Lecture 1 Slide 1 A Two-DOF Mass-Spring-Dashpot Dynamic System Consider the lumped-parameter, mass-spring-dashpot dynamic system shown in the Figure. It has two point

### Portable Assisted Study Sequence ALGEBRA IIA

SCOPE This course is divided into two semesters of study (A & B) comprised of five units each. Each unit teaches concepts and strategies recommended for intermediate algebra students. The first half of

### 3.3. Solving Polynomial Equations. Introduction. Prerequisites. Learning Outcomes

Solving Polynomial Equations 3.3 Introduction Linear and quadratic equations, dealt within Sections 3.1 and 3.2, are members of a class of equations, called polynomial equations. These have the general

### A Second Course in Elementary Differential Equations: Problems and Solutions. Marcel B. Finan Arkansas Tech University c All Rights Reserved

A Second Course in Elementary Differential Equations: Problems and Solutions Marcel B. Finan Arkansas Tech University c All Rights Reserved Contents 8 Calculus of Matrix-Valued Functions of a Real Variable

### ORDINARY DIFFERENTIAL EQUATIONS

ORDINARY DIFFERENTIAL EQUATIONS GABRIEL NAGY Mathematics Department, Michigan State University, East Lansing, MI, 48824. SEPTEMBER 4, 25 Summary. This is an introduction to ordinary differential equations.

### What s Ahead. Interpolating Methods: Lagrange, Newton, Hermite, Monomial, and Chebyshev

Chapter 6 Interpolation ***** Updated 5/21/12 HG What s Ahead Polynomial Interpolation: Existence and Uniqueness Interpolation Error Interpolating Methods: Lagrange, Newton, Hermite, Monomial, and Chebyshev

### SIXTY STUDY QUESTIONS TO THE COURSE NUMERISK BEHANDLING AV DIFFERENTIALEKVATIONER I

Lennart Edsberg, Nada, KTH Autumn 2008 SIXTY STUDY QUESTIONS TO THE COURSE NUMERISK BEHANDLING AV DIFFERENTIALEKVATIONER I Parameter values and functions occurring in the questions belowwill be exchanged

### Roots of Equations (Chapters 5 and 6)

Roots of Equations (Chapters 5 and 6) Problem: given f() = 0, find. In general, f() can be any function. For some forms of f(), analytical solutions are available. However, for other functions, we have

### Variance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers

Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications

### MATHEMATICS (MATH) 3. Provides experiences that enable graduates to find employment in sciencerelated

194 / Department of Natural Sciences and Mathematics MATHEMATICS (MATH) The Mathematics Program: 1. Provides challenging experiences in Mathematics, Physics, and Physical Science, which prepare graduates

### Reaction diffusion systems and pattern formation

Chapter 5 Reaction diffusion systems and pattern formation 5.1 Reaction diffusion systems from biology In ecological problems, different species interact with each other, and in chemical reactions, different

### Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and Non-Square Systems

Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and Non-Square Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001,

### since by using a computer we are limited to the use of elementary arithmetic operations

> 4. Interpolation and Approximation Most functions cannot be evaluated exactly: x, e x, ln x, trigonometric functions since by using a computer we are limited to the use of elementary arithmetic operations

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### Introduction to Partial Differential Equations. John Douglas Moore

Introduction to Partial Differential Equations John Douglas Moore May 2, 2003 Preface Partial differential equations are often used to construct models of the most basic theories underlying physics and

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### Chapter 5. Methods for ordinary differential equations. 5.1 Initial-value problems

Chapter 5 Methods for ordinary differential equations 5.1 Initial-value problems Initial-value problems (IVP) are those for which the solution is entirely known at some time, say t = 0, and the question

### HW6 Solutions. MATH 20D Fall 2013 Prof: Sun Hui TA: Zezhou Zhang (David) November 14, 2013. Checklist: Section 7.8: 1c, 2, 7, 10, [16]

HW6 Solutions MATH D Fall 3 Prof: Sun Hui TA: Zezhou Zhang David November 4, 3 Checklist: Section 7.8: c,, 7,, [6] Section 7.9:, 3, 7, 9 Section 7.8 In Problems 7.8. thru 4: a Draw a direction field and

### Numerical Recipes in C

2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Numerical Recipes in C The Art of Scientific Computing Second Edition

### HETEROGENEOUS AGENTS AND AGGREGATE UNCERTAINTY. Daniel Harenberg daniel.harenberg@gmx.de. University of Mannheim. Econ 714, 28.11.

COMPUTING EQUILIBRIUM WITH HETEROGENEOUS AGENTS AND AGGREGATE UNCERTAINTY (BASED ON KRUEGER AND KUBLER, 2004) Daniel Harenberg daniel.harenberg@gmx.de University of Mannheim Econ 714, 28.11.06 Daniel Harenberg

### NUMERICAL ANALYSIS PROGRAMS

NUMERICAL ANALYSIS PROGRAMS I. About the Program Disk This disk included with Numerical Analysis, Seventh Edition by Burden and Faires contains a C, FORTRAN, Maple, Mathematica, MATLAB, and Pascal program

### CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation

Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in

### Finite Difference Approach to Option Pricing

Finite Difference Approach to Option Pricing February 998 CS5 Lab Note. Ordinary differential equation An ordinary differential equation, or ODE, is an equation of the form du = fut ( (), t) (.) dt where

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### Lecture 13 Linear quadratic Lyapunov theory

EE363 Winter 28-9 Lecture 13 Linear quadratic Lyapunov theory the Lyapunov equation Lyapunov stability conditions the Lyapunov operator and integral evaluating quadratic integrals analysis of ARE discrete-time

### Basics of Polynomial Theory

3 Basics of Polynomial Theory 3.1 Polynomial Equations In geodesy and geoinformatics, most observations are related to unknowns parameters through equations of algebraic (polynomial) type. In cases where

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### Fourth-Order Compact Schemes of a Heat Conduction Problem with Neumann Boundary Conditions

Fourth-Order Compact Schemes of a Heat Conduction Problem with Neumann Boundary Conditions Jennifer Zhao, 1 Weizhong Dai, Tianchan Niu 1 Department of Mathematics and Statistics, University of Michigan-Dearborn,

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### How High a Degree is High Enough for High Order Finite Elements?

This space is reserved for the Procedia header, do not use it How High a Degree is High Enough for High Order Finite Elements? William F. National Institute of Standards and Technology, Gaithersburg, Maryland,

### Lecture 2 Matrix Operations

Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or

### ALGEBRAIC EIGENVALUE PROBLEM

ALGEBRAIC EIGENVALUE PROBLEM BY J. H. WILKINSON, M.A. (Cantab.), Sc.D. Technische Universes! Dsrmstedt FACHBEREICH (NFORMATiK BIBL1OTHEK Sachgebieto:. Standort: CLARENDON PRESS OXFORD 1965 Contents 1.

### Explicit inverses of some tridiagonal matrices

Linear Algebra and its Applications 325 (2001) 7 21 wwwelseviercom/locate/laa Explicit inverses of some tridiagonal matrices CM da Fonseca, J Petronilho Depto de Matematica, Faculdade de Ciencias e Technologia,

### Introduction to the Finite Element Method (FEM)

Introduction to the Finite Element Method (FEM) ecture First and Second Order One Dimensional Shape Functions Dr. J. Dean Discretisation Consider the temperature distribution along the one-dimensional

### Class Meeting # 1: Introduction to PDEs

MATH 18.152 COURSE NOTES - CLASS MEETING # 1 18.152 Introduction to PDEs, Fall 2011 Professor: Jared Speck Class Meeting # 1: Introduction to PDEs 1. What is a PDE? We will be studying functions u = u(x

### General Theory of Differential Equations Sections 2.8, 3.1-3.2, 4.1

A B I L E N E C H R I S T I A N U N I V E R S I T Y Department of Mathematics General Theory of Differential Equations Sections 2.8, 3.1-3.2, 4.1 Dr. John Ehrke Department of Mathematics Fall 2012 Questions

### POLYNOMIAL HISTOPOLATION, SUPERCONVERGENT DEGREES OF FREEDOM, AND PSEUDOSPECTRAL DISCRETE HODGE OPERATORS

POLYNOMIAL HISTOPOLATION, SUPERCONVERGENT DEGREES OF FREEDOM, AND PSEUDOSPECTRAL DISCRETE HODGE OPERATORS N. ROBIDOUX Abstract. We show that, given a histogram with n bins possibly non-contiguous or consisting

### Sound propagation in a lined duct with flow

Sound propagation in a lined duct with flow Martien Oppeneer supervisors: Sjoerd Rienstra and Bob Mattheij CASA day Eindhoven, April 7, 2010 1 / 47 Outline 1 Introduction & Background 2 Modeling the problem

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Fourier Series. A Fourier series is an infinite series of the form. a + b n cos(nωx) +

Fourier Series A Fourier series is an infinite series of the form a b n cos(nωx) c n sin(nωx). Virtually any periodic function that arises in applications can be represented as the sum of a Fourier series.

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

### 1 Error in Euler s Method

1 Error in Euler s Method Experience with Euler s 1 method raises some interesting questions about numerical approximations for the solutions of differential equations. 1. What determines the amount of

### F Matrix Calculus F 1

F Matrix Calculus F 1 Appendix F: MATRIX CALCULUS TABLE OF CONTENTS Page F1 Introduction F 3 F2 The Derivatives of Vector Functions F 3 F21 Derivative of Vector with Respect to Vector F 3 F22 Derivative

### Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

### i = 0, 1,, N p(x i ) = f i,

Chapter 4 Curve Fitting We consider two commonly used methods for curve fitting, namely interpolation and least squares For interpolation, we use first polynomials then splines Then, we introduce least

### Iterative Methods for Computing Eigenvalues and Eigenvectors

The Waterloo Mathematics Review 9 Iterative Methods for Computing Eigenvalues and Eigenvectors Maysum Panju University of Waterloo mhpanju@math.uwaterloo.ca Abstract: We examine some numerical iterative

### Weighted Residual Methods دانشگاه صنعتي اصفهان- دانشكده مكانيك

Weighted Residual Methods 1 Formulation of FEM Model Direct Method Formulation of FEM Model Variational Method Weighted Residuals Several approaches can be used to transform the physical formulation of

### Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8

Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e

### A First Course in Elementary Differential Equations. Marcel B. Finan Arkansas Tech University c All Rights Reserved

A First Course in Elementary Differential Equations Marcel B. Finan Arkansas Tech University c All Rights Reserved 1 Contents 1 Basic Terminology 4 2 Qualitative Analysis: Direction Field of y = f(t, y)

### Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### Equations, Inequalities & Partial Fractions

Contents Equations, Inequalities & Partial Fractions.1 Solving Linear Equations 2.2 Solving Quadratic Equations 1. Solving Polynomial Equations 1.4 Solving Simultaneous Linear Equations 42.5 Solving Inequalities

### Numerical Methods I Eigenvalue Problems

Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 September 30th, 2010 A. Donev (Courant Institute)

### Applied Linear Algebra

Applied Linear Algebra OTTO BRETSCHER http://www.prenhall.com/bretscher Chapter 7 Eigenvalues and Eigenvectors Chia-Hui Chang Email: chia@csie.ncu.edu.tw National Central University, Taiwan 7.1 DYNAMICAL

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Math 215 Exam #1 Practice Problem Solutions

Math 5 Exam # Practice Problem Solutions For each of the following statements, say whether it is true or false If the statement is true, prove it If false, give a counterexample (a) If A is a matrix such

### Summary of week 8 (Lectures 22, 23 and 24)

WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

### Second Order Linear Partial Differential Equations. Part I

Second Order Linear Partial Differential Equations Part I Second linear partial differential equations; Separation of Variables; - point boundary value problems; Eigenvalues and Eigenfunctions Introduction

### Object-oriented scientific computing

Object-oriented scientific computing Pras Pathmanathan Summer 2012 The finite element method Advantages of the FE method over the FD method Main advantages of FE over FD 1 Deal with Neumann boundary conditions

### (Refer Slide Time: 01:11-01:27)

Digital Signal Processing Prof. S. C. Dutta Roy Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 6 Digital systems (contd.); inverse systems, stability, FIR and IIR,

### 3. Regression & Exponential Smoothing

3. Regression & Exponential Smoothing 3.1 Forecasting a Single Time Series Two main approaches are traditionally used to model a single time series z 1, z 2,..., z n 1. Models the observation z t as a

### Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 01

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras Lecture - 01 Welcome to the series of lectures, on finite element analysis. Before I start,

### LINEAR SYSTEMS. Consider the following example of a linear system:

LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x +2x 2 +3x 3 = 5 x + x 3 = 3 3x + x 2 +3x 3 = 3 x =, x 2 =0, x 3 = 2 In general we want to solve n equations in

### A QUICK GUIDE TO THE FORMULAS OF MULTIVARIABLE CALCULUS

A QUIK GUIDE TO THE FOMULAS OF MULTIVAIABLE ALULUS ontents 1. Analytic Geometry 2 1.1. Definition of a Vector 2 1.2. Scalar Product 2 1.3. Properties of the Scalar Product 2 1.4. Length and Unit Vectors