Lecture 11 Fast Fourier Transform (FFT)

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Lecture 11 Fast Fourier Transform (FFT)"

Transcription

1 Lecture 11 Fast Fourier Transform (FFT) Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, 2 School of Mathematical Sciences, Peking University, No.1 Science Building, 1575

2 Outline Examples Fast Fourier Transform Applications

3 Signal processing Filtering: a polluted signal High pass and low pass filter (signal and noise) How to obtain the high frequency and low frequency quickly?

4 Solving PDEs on rectangular mesh Solving the Poisson equations u = f in Ω u = 0 on Ω in the rectangular domain After discretization we will obtain the linear system with about N 2 unknowns ui+1,j + ui 1,j + ui,j+1 + ui,j 1 4ui,j 4h 2 = f ij The FFT would give a fast algorithm to solve the system above with computational efforts O(N 2 log 2 N).

5 Computing convolution ( ) Suppose h(x) = 2π 0 f(x y)g(y)dy is the convolution of f and g, where f(x), g(x) C 2π are period 2π functions. Take x j = jδ, j = 0, 1,..., N 1, δ = 2π N discretization h(x i) N 1 j=0 and apply simple rectangular f(x i x j)g(x j) δ i = 0, 1,..., N 1 Define f i = f(x i), g i = g(x i), and let f i is period N respect to the subscript i, define h i = N 1 j=0 The direct computation is O(N 2 ). f i jg j δ i = 0, 1,..., N 1

6 Fast Fourier Transform Fast Fourier Transform is one of the top 10 algorithms in 20th century. But its idea is quite simple, even for a high school student!

7 Outline Examples Fast Fourier Transform Applications

8 Fourier Transform Suppose f(x) is absolutely integrable in (, + ), then the Fourier transform of f(x) is ˆf(k) = + f(x)e ikx dx. Moreover if f(x) is square integrable, then the inverse Fourier transform of ˆf(k) is f(x) = 1 + ˆf(k)e ikx dk. 2π

9 Properties of Fourier transform Some important properties of Fourier transform: 1. Derivative to coefficient: 2. Translation property: 3. Convolution to multiplication: (f (x))(k) = ik ˆf(k); (f(x a))(k) = e ika ˆf(k); (f g)(k) = ˆf(k)ĝ(k); where (f g)(x) = + f(x y)g(y)dy. 4. Parseval s identity: + f(x) 2 dx = 1 2π + ˆf(k) 2 dk.

10 Discrete Fourier transform (DFT) Suppose we have a = (a 0, a 1,, a N 1) T, define DFT of a as c = (c 0, c 1,, c N 1) T = â, where c k = N 1 j=0 jk 2πi a je N, k = 0, 1,..., N 1. Here i is the imaginary unit, e 2πi N = ω is the N-th root of unity. a is the inverse discrete Fourier transform of c defined as a j = 1 N N 1 k=0 jk 2πi c k e N, j = 0, 1,..., N 1. DFT is closely related to the trigonometric interpolation for 2π-periodic function T (x) = N 2 k= N 2 +1 b k e ikx. such that at x j = 2jπ, T (xj) = aj, j = 0, 1,..., N 1. The readers N may find the relation between c k and b k.

11 Remark on DFT DFT can be considered as a linear transformation. Define Fourier matrix F = 1 ω ω N 1 = (ωjk ) N 1 1 ω N 1 ω (N 1)2 j,k=0 where ω is the N-th root of unity. c is the Fourier transform of a can be represented as c = F a

12 Remark on DFT Inverse DFT can also be considered as a linear transformation. Define inverse Fourier matrix F 1 = G = 1 1 ω 1 ω (N 1) N = (ω jk ) N 1 1 ω (N 1) ω (N 1)2 j,k=0 where ω is the N-th root of unity. a is the inverse Fourier transform of c can be represented as a = Gc

13 Properties of DFT Convolution to multiplication: where (f g) k = ˆf k ĝ k k = 0, 1,..., N 1 (f g) l = N 1 j=0 f l j g j l = 0, 1,..., N 1, and f l is period N with respect to index l, i.e. f 1 = f N 1, f 2 = f N 2,... Parseval s identity: N N 1 j=0 a j 2 = N 1 k=0 c k 2

14 FFT idea FFT is proposed by J.W. Cooley and J.W. Tukey in 1960s, but the idea may be traced back to Gauss. The basic motivation is if we compute DFT directly, i.e. c = F a we need N 2 multiplications and N(N 1) additions. Is it possible to reduce the computation effort? First consider the case N = (a 0 + a 2) + (a 1 + a 3) F = 1 i 1 i , F a = (a 0 a 2) i(a 1 a 3) (a 0 + a 2) (a 1 + a 3) 1 i 1 i (a 0 a 2) + i(a 1 a 3)

15 FFT idea From the concrete form of DFT, we actually need 2 multiplications (timing ±i) and 8 additions (a 0 + a 2, a 1 + a 3, a 0 a 2, a 1 a 3 and the additions in the middle). This observation may reduce the computational effort from O(N 2 ) into O(N log 2 N) Because It is a typical fast algorithm. log lim 2 N N N = 0 Fast algorithms of this type of recursive halving are very typical in scientific computing.

16 Construction of FFT Consider N = 2 m and denote p(x) = a 0 + a 1x + + a N 1x N 1, divide p(x) into odd ( ) and even ( ) power parts p(x) = (a 0 + a 2x 2 + ) + x(a 1 + a 3x 2 + ) = p e(x 2 ) + xp o(x 2 ) where p e(t) = a 0 + a 2t a N 2t N 2 1, p o(t) = a 1 + a 3t a N 1t N 2 1 Define ω k = e 2πi k (k-th root of unity), then when j = 0, 1,..., N 2 1 c j c N 2 +j = p e(ω 2j N ) + ωj N po(ω2j N ) = p e(ω 2( N 2 +j) N ) + ω N 2 +j N p o(ω 2( N 2 +j) N )

17 Construction of FFT Pay attention that ω 2j N = ωj N 2, ω N 2 +j N = ω j N, ωn+2j N = ω j N 2 then c j = v j + ω j N uj, c j+ N 2 = v j ω j N uj j = 0, 1,..., N 2 1 where v j = p e(ω j N 2 ), u j = p o(ω j N 2 ) The formula above show that the DFT of N components vector a could be converted to compute the DFT of two N components vectors ae, ao 2 and some simple additions and multiplications. This is called Danielson-Lanczos algorithm. The recursive application of this idea gives FFT.

18 A simple example: N = 8 Suppose the array a = (a 0, a 1,, a 7) T Step A: Splitting (reordering) (odd parts and even parts): Step 1 Step 2 Step 3 a e = (a 0, a 2, a 4, a 6 ) T, a o = (a 1, a 3, a 5, a 7 ) T ; a ee = (a 0, a 4 ) T, a eo = (a 2, a 6 ) T, a oe = (a 1, a 5 ) T, a oo = (a 3, a 7 ) T ; a eee a eeo a eoe a eoo a oee a oeo a ooe a ooo a 0 a 4 a 2 a 6 a 1 a 5 a 3 a 7

19 A simple example: N = 8 Step B: Combination: Step 1 c ee = (a 0 + ω2a 0 4, a 0 ω2a 0 4) T, c eo = (a 2 + ω2a 0 6, a 2 ω2a 0 6) T, c oe = (a 1 + ω2a 0 5, a 1 ω2a 0 5) T, c oo = (a 3 + ω2a 0 7, a 3 ω2a 0 7) T, Define the notations w 4 (w 0 4, w 1 4) T, w 8 (w 0 8, w 1 8, w 2 8, w 3 8) T, and X Y (x jy j) j as the vector product through multiplication by components.

20 A simple example: N = 8 Step B: Combination: Step 2 Step 3 [ ] [ ] cee + w 4 c eo coe + w 4 c oo c e =, c o =, c ee w 4 c eo c oe w 4 c oo [ ] ce + w 8 c 0 c = c e w 8 c 0

21 A simple sketch of FFT (N = 8) a reordering a 0 a 4 a 2 a 6 a 1 a 5 a 3 a 7 combination 1 c ee c eo c eo c oo combination 2 [ ce c o ] combination 3 c

22 A remark on the reordering If we map e to 0, and o to 1 we can find the binary representation of the indices after reordering is just the bit reversal before reordering 0= = = = = = = = Bit reversal = = = = = = = = 7

23 Outline Examples Fast Fourier Transform Applications

24 Compute the convolution From the discretization at the beginning, we have h i = N 1 j=0 f i jg j δ i = 0, 1,..., N 1 thus h = (ĥ) = (δ ˆf ĝ) After using FFT, N 2 + N multiplications and N(N 1) additions are reduced to 3 2 N log 2 N + 2N multiplications and 3N log 2 N additions.

25 Solving the linear system with loop matrix Let L = c 0 c N 1 c 1 c 1 c 0 c 2 Solving Lx = b. L is a loop matrix. We have c N 1 c N 2 c 0 (Lx) i = N 1 j=0 c i jx j where we assume c is period N with respect to the subscripts, and x = (x 0, x 1,..., x N 1) T.

26 Solving the linear system with loop matrix First consider the Jordan form of L. From the formula before Lx = c x = λx Take DFT we have then eigenvalues ĉ ˆx = λˆx The eigenvectors where δ kj is Kronecker s δ. Take inverse transform we obtain λ k = ĉ k ˆx (k) j = δ kj, (j, k = 0, 1,..., N 1) x (0) = (1, 1,..., 1) T, x (1) = (1, ω 1,..., ω (N 1) ) T, x (N 1) = (1, ω (N 1),..., ω (N 1)2 ) T

27 Solving the linear system with loop matrix Spectral decomposition of L L = (x (0) x (1) x (N 1) ) λ 0 λ 1... (x (0) x (1) x (N 1) ) 1 = (NF 1 )Λ (NF 1 ) 1 = F 1 ΛF λ N 1 Solving Lx = b is equivalent to F 1 ΛF x = b, i.e. Λ(F x) = F b. Then it is composed of three steps: Step 1: Compute F b i.e. apply FFT to b to obtain ˆb; Step 2: Compute Λ i.e. apply FFT to c to obtain ĉ; Step 3: Compute ˆx k = ˆb k /ĉ k, and then compute (ˆx) to obtainx.

28 Homework assignment Familiarize the FFT and IFFT command in MATLAB; Compute the convolution for h(x) = 2π 0 sin(x y)e cos y dy

Examination paper for Solutions to Matematikk 4M and 4N

Examination paper for Solutions to Matematikk 4M and 4N Department of Mathematical Sciences Examination paper for Solutions to Matematikk 4M and 4N Academic contact during examination: Trygve K. Karper Phone: 99 63 9 5 Examination date:. mai 04 Examination

More information

Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Battle Plan Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

More information

THE FAST FOURIER TRANSFORM

THE FAST FOURIER TRANSFORM THE FAST FOURIER TRASFORM LOG CHE ABSTRACT. Fast Fourier transform (FFT) is a fast algorithm to compute the discrete Fourier transform in O( log ) operations for an array of size 2 J. It is based on the

More information

Solutions to Linear Algebra Practice Problems

Solutions to Linear Algebra Practice Problems Solutions to Linear Algebra Practice Problems. Find all solutions to the following systems of linear equations. (a) x x + x 5 x x x + x + x 5 (b) x + x + x x + x + x x + x + 8x Answer: (a) We create the

More information

The continuous and discrete Fourier transforms

The continuous and discrete Fourier transforms FYSA21 Mathematical Tools in Science The continuous and discrete Fourier transforms Lennart Lindegren Lund Observatory (Department of Astronomy, Lund University) 1 The continuous Fourier transform 1.1

More information

4. MATRICES Matrices

4. MATRICES Matrices 4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

More information

1.4 Fast Fourier Transform (FFT) Algorithm

1.4 Fast Fourier Transform (FFT) Algorithm 74 CHAPTER AALYSIS OF DISCRETE-TIME LIEAR TIME-IVARIAT SYSTEMS 4 Fast Fourier Transform (FFT Algorithm Fast Fourier Transform, or FFT, is any algorithm for computing the -point DFT with a computational

More information

MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform

MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish

More information

FFT Algorithms. Chapter 6. Contents 6.1

FFT Algorithms. Chapter 6. Contents 6.1 Chapter 6 FFT Algorithms Contents Efficient computation of the DFT............................................ 6.2 Applications of FFT................................................... 6.6 Computing DFT

More information

Fast Fourier Transforms and Power Spectra in LabVIEW

Fast Fourier Transforms and Power Spectra in LabVIEW Application Note 4 Introduction Fast Fourier Transforms and Power Spectra in LabVIEW K. Fahy, E. Pérez Ph.D. The Fourier transform is one of the most powerful signal analysis tools, applicable to a wide

More information

Sine and Cosine Series; Odd and Even Functions

Sine and Cosine Series; Odd and Even Functions Sine and Cosine Series; Odd and Even Functions A sine series on the interval [, ] is a trigonometric series of the form k = 1 b k sin πkx. All of the terms in a series of this type have values vanishing

More information

Wavelet analysis. Wavelet requirements. Example signals. Stationary signal 2 Hz + 10 Hz + 20Hz. Zero mean, oscillatory (wave) Fast decay (let)

Wavelet analysis. Wavelet requirements. Example signals. Stationary signal 2 Hz + 10 Hz + 20Hz. Zero mean, oscillatory (wave) Fast decay (let) Wavelet analysis In the case of Fourier series, the orthonormal basis is generated by integral dilation of a single function e jx Every 2π-periodic square-integrable function is generated by a superposition

More information

C 1 x(t) = e ta C = e C n. 2! A2 + t3

C 1 x(t) = e ta C = e C n. 2! A2 + t3 Matrix Exponential Fundamental Matrix Solution Objective: Solve dt A x with an n n constant coefficient matrix A x (t) Here the unknown is the vector function x(t) x n (t) General Solution Formula in Matrix

More information

Chapter 3 Joint Distributions

Chapter 3 Joint Distributions Chapter 3 Joint Distributions 3.6 Functions of Jointly Distributed Random Variables Discrete Random Variables: Let f(x, y) denote the joint pdf of random variables X and Y with A denoting the two-dimensional

More information

Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

More information

G.A. Pavliotis. Department of Mathematics. Imperial College London

G.A. Pavliotis. Department of Mathematics. Imperial College London EE1 MATHEMATICS NUMERICAL METHODS G.A. Pavliotis Department of Mathematics Imperial College London 1. Numerical solution of nonlinear equations (iterative processes). 2. Numerical evaluation of integrals.

More information

The Fast Fourier Transform (FFT) and MATLAB Examples

The Fast Fourier Transform (FFT) and MATLAB Examples The Fast Fourier Transform (FFT) and MATLAB Examples Learning Objectives Discrete Fourier transforms (DFTs) and their relationship to the Fourier transforms Implementation issues with the DFT via the FFT

More information

Mathematical Background

Mathematical Background Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,

More information

SGN-1158 Introduction to Signal Processing Test. Solutions

SGN-1158 Introduction to Signal Processing Test. Solutions SGN-1158 Introduction to Signal Processing Test. Solutions 1. Convolve the function ( ) with itself and show that the Fourier transform of the result is the square of the Fourier transform of ( ). (Hints:

More information

2 Integrating Both Sides

2 Integrating Both Sides 2 Integrating Both Sides So far, the only general method we have for solving differential equations involves equations of the form y = f(x), where f(x) is any function of x. The solution to such an equation

More information

Introduction to Sturm-Liouville Theory

Introduction to Sturm-Liouville Theory Introduction to Ryan C. Trinity University Partial Differential Equations April 10, 2012 Inner products with weight functions Suppose that w(x) is a nonnegative function on [a,b]. If f (x) and g(x) are

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

More information

The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23 (copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

More information

Fourier Transform and Image Filtering. CS/BIOEN 6640 Lecture Marcel Prastawa Fall 2010

Fourier Transform and Image Filtering. CS/BIOEN 6640 Lecture Marcel Prastawa Fall 2010 Fourier Transform and Image Filtering CS/BIOEN 6640 Lecture Marcel Prastawa Fall 2010 The Fourier Transform Fourier Transform Forward, mapping to frequency domain: Backward, inverse mapping to time domain:

More information

Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

More information

1 Review of Newton Polynomials

1 Review of Newton Polynomials cs: introduction to numerical analysis 0/0/0 Lecture 8: Polynomial Interpolation: Using Newton Polynomials and Error Analysis Instructor: Professor Amos Ron Scribes: Giordano Fusco, Mark Cowlishaw, Nathanael

More information

ECE 842 Report Implementation of Elliptic Curve Cryptography

ECE 842 Report Implementation of Elliptic Curve Cryptography ECE 842 Report Implementation of Elliptic Curve Cryptography Wei-Yang Lin December 15, 2004 Abstract The aim of this report is to illustrate the issues in implementing a practical elliptic curve cryptographic

More information

Theta Functions. Lukas Lewark. Seminar on Modular Forms, 31. Januar 2007

Theta Functions. Lukas Lewark. Seminar on Modular Forms, 31. Januar 2007 Theta Functions Lukas Lewark Seminar on Modular Forms, 31. Januar 007 Abstract Theta functions are introduced, associated to lattices or quadratic forms. Their transformation property is proven and the

More information

SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

More information

NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

Note 5.1 Stress range histories and Rain Flow counting

Note 5.1 Stress range histories and Rain Flow counting June 2009/ John Wægter Note 5.1 Stress range histories and Rain Flow counting Introduction...2 Stress range histories...3 General...3 Characterization of irregular fatigue loading...4 Stress range histogram...5

More information

Analysis/resynthesis with the short time Fourier transform

Analysis/resynthesis with the short time Fourier transform Analysis/resynthesis with the short time Fourier transform summer 2006 lecture on analysis, modeling and transformation of audio signals Axel Röbel Institute of communication science TU-Berlin IRCAM Analysis/Synthesis

More information

MATH 110 Spring 2015 Homework 6 Solutions

MATH 110 Spring 2015 Homework 6 Solutions MATH 110 Spring 2015 Homework 6 Solutions Section 2.6 2.6.4 Let α denote the standard basis for V = R 3. Let α = {e 1, e 2, e 3 } denote the dual basis of α for V. We would first like to show that β =

More information

On Polynomial Multiplication Complexity in Chebyshev Basis

On Polynomial Multiplication Complexity in Chebyshev Basis On Polynomial Multiplication Complexity in Chebyshev Basis Pascal Giorgi INRIA Rocquencourt, Algorithms Project s Seminar November 29, 2010. Outline 1 Polynomial multiplication in monomial basis 2 Polynomial

More information

Inner Product Spaces

Inner Product Spaces Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

More information

Final Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones

Final Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones Final Year Project Progress Report Frequency-Domain Adaptive Filtering Myles Friel 01510401 Supervisor: Dr.Edward Jones Abstract The Final Year Project is an important part of the final year of the Electronic

More information

33 Cauchy Integral Formula

33 Cauchy Integral Formula 33 AUHY INTEGRAL FORMULA October 27, 2006 PROOF Use the theorem to write f ()d + 1 f ()d + 1 f ()d = 0 2 f ()d = f ()d f ()d. 2 1 2 This is the deformation principle; if you can continuously deform 1 to

More information

equations Karl Lundengård December 3, 2012 MAA704: Matrix functions and matrix equations Matrix functions Matrix equations Matrix equations, cont d

equations Karl Lundengård December 3, 2012 MAA704: Matrix functions and matrix equations Matrix functions Matrix equations Matrix equations, cont d and and Karl Lundengård December 3, 2012 Solving General, Contents of todays lecture and (Kroenecker product) Solving General, Some useful from calculus and : f (x) = x n, x C, n Z + : f (x) = n x, x R,

More information

Basic Quantum Mechanics

Basic Quantum Mechanics Basic Quantum Mechanics Postulates of QM - The state of a system with n position variables q, q, qn is specified by a state (or wave) function Ψ(q, q, qn) - To every observable (physical magnitude) there

More information

Similarity and Diagonalization. Similar Matrices

Similarity and Diagonalization. Similar Matrices MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

More information

3. Interpolation. Closing the Gaps of Discretization... Beyond Polynomials

3. Interpolation. Closing the Gaps of Discretization... Beyond Polynomials 3. Interpolation Closing the Gaps of Discretization... Beyond Polynomials Closing the Gaps of Discretization... Beyond Polynomials, December 19, 2012 1 3.3. Polynomial Splines Idea of Polynomial Splines

More information

FAST FOURIER TRANSFORM ALGORITHMS WITH APPLICATIONS. A Dissertation Presented to the Graduate School of Clemson University

FAST FOURIER TRANSFORM ALGORITHMS WITH APPLICATIONS. A Dissertation Presented to the Graduate School of Clemson University FAST FOURIER TRANSFORM ALGORITHMS WITH APPLICATIONS A Dissertation Presented to the Graduate School of Clemson University In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy

More information

8 Polynomials Worksheet

8 Polynomials Worksheet 8 Polynomials Worksheet Concepts: Quadratic Functions The Definition of a Quadratic Function Graphs of Quadratic Functions - Parabolas Vertex Absolute Maximum or Absolute Minimum Transforming the Graph

More information

Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

More information

5. Orthogonal matrices

5. Orthogonal matrices L Vandenberghe EE133A (Spring 2016) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

More information

EC9A0: Pre-sessional Advanced Mathematics Course

EC9A0: Pre-sessional Advanced Mathematics Course University of Warwick, EC9A0: Pre-sessional Advanced Mathematics Course Peter J. Hammond & Pablo F. Beker 1 of 55 EC9A0: Pre-sessional Advanced Mathematics Course Slides 1: Matrix Algebra Peter J. Hammond

More information

DERIVATIVES AS MATRICES; CHAIN RULE

DERIVATIVES AS MATRICES; CHAIN RULE DERIVATIVES AS MATRICES; CHAIN RULE 1. Derivatives of Real-valued Functions Let s first consider functions f : R 2 R. Recall that if the partial derivatives of f exist at the point (x 0, y 0 ), then we

More information

November 16, 2015. Interpolation, Extrapolation & Polynomial Approximation

November 16, 2015. Interpolation, Extrapolation & Polynomial Approximation Interpolation, Extrapolation & Polynomial Approximation November 16, 2015 Introduction In many cases we know the values of a function f (x) at a set of points x 1, x 2,..., x N, but we don t have the analytic

More information

TMA4213/4215 Matematikk 4M/N Vår 2013

TMA4213/4215 Matematikk 4M/N Vår 2013 Norges teknisk naturvitenskapelige universitet Institutt for matematiske fag TMA43/45 Matematikk 4M/N Vår 3 Løsningsforslag Øving a) The Fourier series of the signal is f(x) =.4 cos ( 4 L x) +cos ( 5 L

More information

Matrix Differentiation

Matrix Differentiation 1 Introduction Matrix Differentiation ( and some other stuff ) Randal J. Barnes Department of Civil Engineering, University of Minnesota Minneapolis, Minnesota, USA Throughout this presentation I have

More information

LEAST SQUARES APPROXIMATION

LEAST SQUARES APPROXIMATION LEAST SQUARES APPROXIMATION Another approach to approximating a function f(x) on an interval a x b is to seek an approximation p(x) with a small average error over the interval of approximation. A convenient

More information

Factorization Theorems

Factorization Theorems Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

More information

1 Introduction to Matrices

1 Introduction to Matrices 1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

More information

FINAL EXAM SECTIONS AND OBJECTIVES FOR COLLEGE ALGEBRA

FINAL EXAM SECTIONS AND OBJECTIVES FOR COLLEGE ALGEBRA FINAL EXAM SECTIONS AND OBJECTIVES FOR COLLEGE ALGEBRA 1.1 Solve linear equations and equations that lead to linear equations. a) Solve the equation: 1 (x + 5) 4 = 1 (2x 1) 2 3 b) Solve the equation: 3x

More information

Separable First Order Differential Equations

Separable First Order Differential Equations Separable First Order Differential Equations Form of Separable Equations which take the form = gx hy or These are differential equations = gxĥy, where gx is a continuous function of x and hy is a continuously

More information

Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

More information

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form 1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

More information

Quantum Computing Lecture 7. Quantum Factoring. Anuj Dawar

Quantum Computing Lecture 7. Quantum Factoring. Anuj Dawar Quantum Computing Lecture 7 Quantum Factoring Anuj Dawar Quantum Factoring A polynomial time quantum algorithm for factoring numbers was published by Peter Shor in 1994. polynomial time here means that

More information

SYSTEMS OF EQUATIONS

SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS 1. Examples of systems of equations Here are some examples of systems of equations. Each system has a number of equations and a number (not necessarily the same) of variables for which

More information

Definition of Vertical Asymptote The line x = a is called a vertical asymptote of f (x) if at least one of the following is true: f (x) =

Definition of Vertical Asymptote The line x = a is called a vertical asymptote of f (x) if at least one of the following is true: f (x) = Vertical Asymptotes Definition of Vertical Asymptote The line x = a is called a vertical asymptote of f (x) if at least one of the following is true: lim f (x) = x a lim f (x) = lim x a lim f (x) = x a

More information

FIXED POINT ITERATION

FIXED POINT ITERATION FIXED POINT ITERATION We begin with a computational example. solving the two equations Consider E1: x =1+.5sinx E2: x =3+2sinx Graphs of these two equations are shown on accompanying graphs, with the solutions

More information

n th roots of complex numbers

n th roots of complex numbers n th roots of complex numbers Nathan Pflueger 1 October 014 This note describes how to solve equations of the form z n = c, where c is a complex number. These problems serve to illustrate the use of polar

More information

Chapter 8 - Power Density Spectrum

Chapter 8 - Power Density Spectrum EE385 Class Notes 8/8/03 John Stensby Chapter 8 - Power Density Spectrum Let X(t) be a WSS random process. X(t) has an average power, given in watts, of E[X(t) ], a constant. his total average power is

More information

Introduction to Complex Fourier Series

Introduction to Complex Fourier Series Introduction to Complex Fourier Series Nathan Pflueger 1 December 2014 Fourier series come in two flavors. What we have studied so far are called real Fourier series: these decompose a given periodic function

More information

Inner product. Definition of inner product

Inner product. Definition of inner product Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

More information

Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

More information

The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1]. Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

More information

Introduction to Monte-Carlo Methods

Introduction to Monte-Carlo Methods Introduction to Monte-Carlo Methods Bernard Lapeyre Halmstad January 2007 Monte-Carlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities

More information

13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions. 3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

More information

1 Lecture: Integration of rational functions by decomposition

1 Lecture: Integration of rational functions by decomposition Lecture: Integration of rational functions by decomposition into partial fractions Recognize and integrate basic rational functions, except when the denominator is a power of an irreducible quadratic.

More information

Representation of functions as power series

Representation of functions as power series Representation of functions as power series Dr. Philippe B. Laval Kennesaw State University November 9, 008 Abstract This document is a summary of the theory and techniques used to represent functions

More information

Math 5311 Gateaux differentials and Frechet derivatives

Math 5311 Gateaux differentials and Frechet derivatives Math 5311 Gateaux differentials and Frechet derivatives Kevin Long January 26, 2009 1 Differentiation in vector spaces Thus far, we ve developed the theory of minimization without reference to derivatives.

More information

GEC320 COURSE COMPACT. Four hours per week for 15 weeks (60 hours)

GEC320 COURSE COMPACT. Four hours per week for 15 weeks (60 hours) GEC320 COURSE COMPACT Course Course code: GEC 320 Course title: Course status: Course Duration Numerical Methods (2 units) Compulsory Four hours per week for 15 weeks (60 hours) Lecturer Data Name: Engr.

More information

The Convolution Operation

The Convolution Operation The Convolution Operation Convolution is a very natural mathematical operation which occurs in both discrete and continuous modes of various kinds. We often encounter it in the course of doing other operations

More information

Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability

Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability S. Widnall 16.07 Dynamics Fall 2009 Version 1.0 Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability Vibration, Instability An important class of problems in dynamics concerns the free

More information

Microeconomic Theory: Basic Math Concepts

Microeconomic Theory: Basic Math Concepts Microeconomic Theory: Basic Math Concepts Matt Van Essen University of Alabama Van Essen (U of A) Basic Math Concepts 1 / 66 Basic Math Concepts In this lecture we will review some basic mathematical concepts

More information

Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

More information

Roots and Coefficients of a Quadratic Equation Summary

Roots and Coefficients of a Quadratic Equation Summary Roots and Coefficients of a Quadratic Equation Summary For a quadratic equation with roots α and β: Sum of roots = α + β = and Product of roots = αβ = Symmetrical functions of α and β include: x = and

More information

Taylor Polynomials and Taylor Series Math 126

Taylor Polynomials and Taylor Series Math 126 Taylor Polynomials and Taylor Series Math 26 In many problems in science and engineering we have a function f(x) which is too complicated to answer the questions we d like to ask. In this chapter, we will

More information

We shall turn our attention to solving linear systems of equations. Ax = b

We shall turn our attention to solving linear systems of equations. Ax = b 59 Linear Algebra We shall turn our attention to solving linear systems of equations Ax = b where A R m n, x R n, and b R m. We already saw examples of methods that required the solution of a linear system

More information

1 Determinants and the Solvability of Linear Systems

1 Determinants and the Solvability of Linear Systems 1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped

More information

ALGORITHMS FOR ALGEBRAIC CURVES

ALGORITHMS FOR ALGEBRAIC CURVES ALGORITHMS FOR ALGEBRAIC CURVES SUMMARY OF LECTURE 4 1. SCHOOF S ALGORITHM Let K be a finite field with q elements. Let p be its characteristic. Let X be an elliptic curve over K. To simplify we assume

More information

Nonlinear Algebraic Equations Example

Nonlinear Algebraic Equations Example Nonlinear Algebraic Equations Example Continuous Stirred Tank Reactor (CSTR). Look for steady state concentrations & temperature. s r (in) p,i (in) i In: N spieces with concentrations c, heat capacities

More information

4.3 Lagrange Approximation

4.3 Lagrange Approximation 206 CHAP. 4 INTERPOLATION AND POLYNOMIAL APPROXIMATION Lagrange Polynomial Approximation 4.3 Lagrange Approximation Interpolation means to estimate a missing function value by taking a weighted average

More information

A note on companion matrices

A note on companion matrices Linear Algebra and its Applications 372 (2003) 325 33 www.elsevier.com/locate/laa A note on companion matrices Miroslav Fiedler Academy of Sciences of the Czech Republic Institute of Computer Science Pod

More information

(x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

(x + a) n = x n + a Z n [x]. Proof. If n is prime then the map 22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

More information

Discrete Mathematics: Solutions to Homework (12%) For each of the following sets, determine whether {2} is an element of that set.

Discrete Mathematics: Solutions to Homework (12%) For each of the following sets, determine whether {2} is an element of that set. Discrete Mathematics: Solutions to Homework 2 1. (12%) For each of the following sets, determine whether {2} is an element of that set. (a) {x R x is an integer greater than 1} (b) {x R x is the square

More information

Homework set 1: posted on website

Homework set 1: posted on website Homework set 1: posted on website 1 Last time: Interpolation, fitting and smoothing Interpolated curve passes through all data points Fitted curve passes closest (by some criterion) to all data points

More information

The Characteristic Polynomial

The Characteristic Polynomial Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

More information

Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks

Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks Thnkwell s Homeschool Precalculus Course Lesson Plan: 36 weeks Welcome to Thinkwell s Homeschool Precalculus! We re thrilled that you ve decided to make us part of your homeschool curriculum. This lesson

More information

9 Fourier Transform Properties

9 Fourier Transform Properties 9 Fourier Transform Properties The Fourier transform is a major cornerstone in the analysis and representation of signals and linear, time-invariant systems, and its elegance and importance cannot be overemphasized.

More information

1 Review of complex numbers

1 Review of complex numbers 1 Review of complex numbers 1.1 Complex numbers: algebra The set C of complex numbers is formed by adding a square root i of 1 to the set of real numbers: i = 1. Every complex number can be written uniquely

More information

Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

More information

Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

More information

NOTRE DAME MATHEMATICAL Number 2 GALOIS THEORY. Lectures delivered at the University of Notre Dame

NOTRE DAME MATHEMATICAL Number 2 GALOIS THEORY. Lectures delivered at the University of Notre Dame NOTRE DAME MATHEMATICAL Number 2 LECTURES GALOIS THEORY Lectures delivered at the University of Notre Dame by DR. EMIL ARTIN Professor of Mathematics, Princeton University Edited and supplemented with

More information

Chebyshev Expansions

Chebyshev Expansions Chapter 3 Chebyshev Expansions The best is the cheapest. Benjamin Franklin 3.1 Introduction In Chapter, approximations were considered consisting of expansions around a specific value of the variable (finite

More information

Algebra 1 Course Information

Algebra 1 Course Information Course Information Course Description: Students will study patterns, relations, and functions, and focus on the use of mathematical models to understand and analyze quantitative relationships. Through

More information

10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES

10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES 58 CHAPTER NUMERICAL METHODS. POWER METHOD FOR APPROXIMATING EIGENVALUES In Chapter 7 you saw that the eigenvalues of an n n matrix A are obtained by solving its characteristic equation n c nn c nn...

More information

1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates

1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates The maximum-likelihood method Volker Blobel University of Hamburg March 2005 1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates Keys during display: enter = next page; =

More information