Discrete Convolution and the Discrete Fourier Transform


 Jerome Reynolds
 1 years ago
 Views:
Transcription
1 Discrete Convolution and the Discrete Fourier Transform Discrete Convolution First of all we need to introduce what we might call the wraparound convention Because the complex numbers w j e i πj N have the property w j±n w j, which readily extends to w j+mn w j for any integer m, and since in the discrete Fourier context we represent all Ndimensional vectors as linear combinations of the Fourier vectors W k whose components are w kj, we make the convention that for any vector Z E N with components z k, k,,,n, if we write z k with k lying outside the range through N whatismeantisz k+mn where m is the unique integer such that k + mn does lie in this range Thus, for example, if N 8 and we write z,whatwemeanisz + 8 z This convention may initially seem rather strange and arbitrary but, in fact, it is quite essential for effective use of and computation with the discrete Fourier transform Now suppose we have two real or complex vectors in E N : Z (z z z N ) t, Y (y y y N ) t The discrete convolution of these two vectors is another vector, which we denote by Z Y, defined componentwise by (Z Y ) k N j z k j y j, k,,, In making this definition we have to take into account the fact that if j > k the index k j will be negative According to the convention just introduced, what is meant instead of z k j is z k j+n Setting l k j we can see that (Z Y ) k N j z k j y j k N+ lk z l y k l N l y k l z l (Y Z) k,
2 the second inequality again following from the wraparound convention As a consequence we conclude that discrete convolution is commutative: Z Y Y Z Now we compute some discrete convolutions Example If Z ( ) t, then for any Y we have (Z Y ) k N j z k j y j y k because z k j when k j and equals when k j Thus the vector Z as shown here plays the role of the convolution identity Example If Z ( ) t, then for any Y we have (Z Y ) k N and thus with Z as shown j z k j y j N j y j s Y Z Y (s Y s Y s Y ) t (s Y )Z In particular, Z Z NZ in this case Example Let for N 5andlet Z ( ) t Y ( ) t for the same value of N; let us suppose the two s are in positions k and k + In this case we can verify that Z Y has the form Z Y ( ) t ; the effect of the convolution is to blur the two distinct s into the extended pattern shown
3 Just as the importance of the Laplace transform derives in large part from its behavior with respect to convolution of functions on [, ), the significance of the discrete Fourier transform also largely derives from its relationship to discrete convolution Theorem For any two Ndimensional complex vectors Z and Y we have F(Z Y )N F(Z) F(Y ) where the product F(Z) F(Y ) indicates the vector obtained with component by component multiplication; ie, (F(Z) F(Y )) k (F(Z)) k (F(Y )) k, the right hand side being the ordinary numerical product of complex numbers Proof We compute F(Z Y ) k N N j w kj (Z Y ) j N N j w kj ( N l z j l y l ) N N (setting m j l) N N l N j N l N l w kl y l ( N w k(j l) z j l w kl y l j w kl y l ( N l m l w k(j l) z j l ) (taking the wraparound effect into account) N ( N N l w kl y l )( N w km z m ) N m w km z m ) N (F(Y )) k (F(Z)) k (F(Y ) F(Z)) k,
4 which completes the proof Convolution Equations A (discrete) convolution equation is an equation of the form A Y B, wherea and B are known vectors in E N and Y is the vector in E N to be determined Such an equation is simply a linear algebraic equation of a special type If we define the N N matrix A a a N a N a a a a N a a a a a a N a N a N a whose first column is A and whose subsequent columns are rotations of A, one can easily see that the convolution product A Y is the same as the matrixvector product A Y and the convolution equation takes the form A Y B Then we have the following Proposition The convolution equation A Y B is solvable for arbitrary B E N (equivalently, the matrix A is nonsingular, so that A exists) if and only if the discrete Fourier transform of A has no zero components; ie, (F(A)) k, k,,,, N, From A Y B we obtain, taking the discrete Fourier trans Proof form, F(A Y )F(B) Using the result from Section on the discrete Fourier transform of a convolution product, this becomes N F(A) F(Y )F(B) Looking at the kth component on each side we have N (F(A)) k (F(Y )) k (F(B)) k, k,,,, N 4
5 If (F(A)) k foranyk then it is clear that the corresponding component, (F(B)) k,off(b) must also be zero and the convolution equation cannot be solved for arbitrary vectors B On the other hand, if all of the components of F(A) are different from zero, we simply divide to obtain (F(Y )) k (F(B)) k N (F(A)) k We will write this vectorially as F(Y ) F(B) F(A), N where the symbol will be used to indicate component by component division of the first vector by the second Then, applying the inverse transform, we have Y F ( F(B) F(A)) N This completes the proof of the proposition Solving Discrete Convolution Equations Example We consider the convolution equation in R 4 : Y The discrete Fourier transform solution method involves application of the discrete Fourier transform to both sides of this equation and use of the property F(A Y )N F(A) F(Y ) First of all, we have F(A) 4 i i i i 4 5 5
6 In this case none of the components of F(A) are zero so the solution process is a straightforward one Computing F(B) 4 i i i i + i i, and letting Z F(Y ), we have the equation F(A Y )4F(A) Z 5 z z z z + i i This is solved readily to give Z + i i Then, applying the inverse transform, we have Y F (Z) i i i i + i i The fact that the solution Y turns out to be the same as the original right hand side B is a coincidence particular to this example Example In this example the solution process is complicated by the fact that one of the components of F(A) turns out to be zero The convolution equation in question is 6 Y d 4 d 4 + d B + db, 6
7 where d is a parameter, initially undetermined Here we have F(A) 4 i i i i 6 i +i ; FB 4 i i i i 4 ; FB 4 i i i i 4 i 4 4 i 4 Since the transform is linear, letting Z F(Y ), we have 4 i +i z z z z + d 4 i 4 4 i 4 From examination of the component on both sides it is clear that we have to take d for consistency Then z is arbitrary and we have ( 8i) z i z 5 i 68 ; 8 z + z 8 ; ( +8i) z + i z 5+i 68 Then Y i i i i z 5 i i 68 z
8 Our next example shows that the discrete Fourier transform can be applied to solve, at least approximately, certain problems of classical mechanics The example also shows that zero components of F(A) are not just an inconvenience but may correspond to degrees of freedom in the solution of the convolution equation A Y B which we would quite naturally expect to find Example 4 A cable is strung between two vertical supporting structures of equal height and is displaced from the horizontal due to the effect of gravity acting on the cable Letting < x< be the parametrization in the horizontal direction and letting y(x) denote the vertical displacement of the cable, we have τ d y dx gρ(x), < x<, where τ is the tension in the cable, g is the gravitational constant and ρ(x) is the cable mass per unit length It is not difficult to solve this problem analytically but we will carry out an approximate solution as an example of application of Fourier solution of discrete convolution equations We discretize the problem by dividing the interval [, ) into N subintervals [x k,x k ),k,,, N, ofequallengthh and we replace N the unknown displacement function y(x) byanndimensional vector Y [y,y,, y N ] T representing approximate displacements at the points x k Since the supports at the endpoints are of equal height we can identify y with y N as we need to do in discrete Fourier analysis (In effect, we are assuming a periodic structure consisting of supports of equal height placed one unit apart with identical cables suspended between each adjacent pair of them) We replace the second derivative operator by the standard second order divided difference approximation d y dx (x k) y k+ y k + y k h and we discretize the gravitational force to g k gρ(x k )atthepoints x k,k,,,, N Then we let B be the vector whose components 8
9 are { g b k s gρ() s, k, g k gρ(x k ), k,,, N, where s is the force exerted by the supporting structure Then we have τn A Y B A Y τn B, where A ( ) T, the omitted terms all being zero Taking the discrete Fourier transform and using the convolution property of the transform we have NF(A) F(Y ) τn F(B) F(Y ) k F(B) k, F(A) τn k F(A) k In this case, because + ( ) +, the component F(A) For consistency, then, we also must have F(B) But this last component is just g s + N k g k, so we conclude that we must have s N k which is just the statement that the magnitude of the supporting force s should be the sum of the magnitudes of he gravitational forces  very much to be expected from the principles of statics Having made this choice, the component F(Y ) N the average value of the vertical displacements, is arbitrary But this is to be expected because we have not specified the height of the supporting structures g k, N For our computational treatment of this problem we will take τ, g, N 6 and we will suppose that ρ(x) {, < x< 5 4, k 6, w k, < x< 6 6 < x<,
10 This gives B ( s ) The supporting force s is selected so the sum of the components of B is zero, ie, s 4 We used MATLAB R to compute C F ( 56 B) ; because of the choice of s the first component of this discrete Fourier transform is zero There is little point in providing the whole list of the components of C here We also computed F(A); the first component is again zero and all other components are nonzero We thus let F(Y ), F(Y ) k F( 56 B) k 6 F(A) k, k,,, 5 At this point all components of F(Y ) have been determined except the first Thus we can write F(Y ) F(Y )+r ( ) t, where r is yet to be determined Then Y Y + r F (( ) t ) Y + r 6 ( )t It turns out in this example that Y ( ) t We suppose the height of the supporting posts is Then we set r to realize the value as the first component of Y To display the right hand supporting post we introduce a new component y 6 y and we plot the result in Figure
11 Figure 8 6 > y > x Supported Cable as Constructed Using Discrete Fourier Transform
Elements of Abstract Group Theory
Chapter 2 Elements of Abstract Group Theory Mathematics is a game played according to certain simple rules with meaningless marks on paper. David Hilbert The importance of symmetry in physics, and for
More informationSection 3.7. Rolle s Theorem and the Mean Value Theorem. Difference Equations to Differential Equations
Difference Equations to Differential Equations Section.7 Rolle s Theorem and the Mean Value Theorem The two theorems which are at the heart of this section draw connections between the instantaneous rate
More informationBasic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011
Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011 A. Miller 1. Introduction. The definitions of metric space and topological space were developed in the early 1900 s, largely
More information3. INNER PRODUCT SPACES
. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More information1 Sets and Set Notation.
LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most
More informationPROOFS BY DESCENT KEITH CONRAD
PROOFS BY DESCENT KEITH CONRAD As ordinary methods, such as are found in the books, are inadequate to proving such difficult propositions, I discovered at last a most singular method... that I called the
More informationDiscreteTime Signals and Systems
2 DiscreteTime Signals and Systems 2.0 INTRODUCTION The term signal is generally applied to something that conveys information. Signals may, for example, convey information about the state or behavior
More informationGroup Fundamentals. Chapter 1. 1.1 Groups and Subgroups. 1.1.1 Definition
Chapter 1 Group Fundamentals 1.1 Groups and Subgroups 1.1.1 Definition A group is a nonempty set G on which there is defined a binary operation (a, b) ab satisfying the following properties. Closure: If
More information4. FIRST STEPS IN THE THEORY 4.1. A
4. FIRST STEPS IN THE THEORY 4.1. A Catalogue of All Groups: The Impossible Dream The fundamental problem of group theory is to systematically explore the landscape and to chart what lies out there. We
More informationSwitching Algebra and Logic Gates
Chapter 2 Switching Algebra and Logic Gates The word algebra in the title of this chapter should alert you that more mathematics is coming. No doubt, some of you are itching to get on with digital design
More informationChapter 5. Banach Spaces
9 Chapter 5 Banach Spaces Many linear equations may be formulated in terms of a suitable linear operator acting on a Banach space. In this chapter, we study Banach spaces and linear operators acting on
More informationONEDIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK
ONEDIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK Definition 1. A random walk on the integers with step distribution F and initial state x is a sequence S n of random variables whose increments are independent,
More informationWHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT?
WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT? introduction Many students seem to have trouble with the notion of a mathematical proof. People that come to a course like Math 216, who certainly
More information2 Integrating Both Sides
2 Integrating Both Sides So far, the only general method we have for solving differential equations involves equations of the form y = f(x), where f(x) is any function of x. The solution to such an equation
More informationSUBGROUPS OF CYCLIC GROUPS. 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by
SUBGROUPS OF CYCLIC GROUPS KEITH CONRAD 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by g = {g k : k Z}. If G = g, then G itself is cyclic, with g as a generator. Examples
More informationYou know from calculus that functions play a fundamental role in mathematics.
CHPTER 12 Functions You know from calculus that functions play a fundamental role in mathematics. You likely view a function as a kind of formula that describes a relationship between two (or more) quantities.
More informationA Course on Number Theory. Peter J. Cameron
A Course on Number Theory Peter J. Cameron ii Preface These are the notes of the course MTH6128, Number Theory, which I taught at Queen Mary, University of London, in the spring semester of 2009. There
More informationLinear Control Systems
Chapter 3 Linear Control Systems Topics : 1. Controllability 2. Observability 3. Linear Feedback 4. Realization Theory Copyright c Claudiu C. Remsing, 26. All rights reserved. 7 C.C. Remsing 71 Intuitively,
More informationOrthogonal Bases and the QR Algorithm
Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries
More informationGroup Theory. Contents
Group Theory Contents Chapter 1: Review... 2 Chapter 2: Permutation Groups and Group Actions... 3 Orbits and Transitivity... 6 Specific Actions The Right regular and coset actions... 8 The Conjugation
More informationCommunication Theory of Secrecy Systems
Communication Theory of Secrecy Systems By C. E. SHANNON 1 INTRODUCTION AND SUMMARY The problems of cryptography and secrecy systems furnish an interesting application of communication theory 1. In this
More informationHow many numbers there are?
How many numbers there are? RADEK HONZIK Radek Honzik: Charles University, Department of Logic, Celetná 20, Praha 1, 116 42, Czech Republic radek.honzik@ff.cuni.cz Contents 1 What are numbers 2 1.1 Natural
More informationThe Backpropagation Algorithm
7 The Backpropagation Algorithm 7. Learning as gradient descent We saw in the last chapter that multilayered networks are capable of computing a wider range of Boolean functions than networks with a single
More informationA HOPF DIFFERENTIAL FOR CONSTANT MEAN CURVATURE SURFACES IN S 2 R AND H 2 R
A HOPF DIFFERENTIAL FOR CONSTANT MEAN CURVATURE SURFACES IN S 2 R AND H 2 R UWE ABRESCH AND HAROLD ROSENBERG Dedicated to Hermann Karcher on the Occasion of his 65 th Birthday Abstract. A basic tool in
More information4.1 Learning algorithms for neural networks
4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to
More informationHow to Gamble If You Must
How to Gamble If You Must Kyle Siegrist Department of Mathematical Sciences University of Alabama in Huntsville Abstract In red and black, a player bets, at even stakes, on a sequence of independent games
More informationNotes on finite group theory. Peter J. Cameron
Notes on finite group theory Peter J. Cameron October 2013 2 Preface Group theory is a central part of modern mathematics. Its origins lie in geometry (where groups describe in a very detailed way the
More informationPRINCIPAL COMPONENT ANALYSIS
1 Chapter 1 PRINCIPAL COMPONENT ANALYSIS Introduction: The Basics of Principal Component Analysis........................... 2 A Variable Reduction Procedure.......................................... 2
More informationAn Introduction to the NavierStokes InitialBoundary Value Problem
An Introduction to the NavierStokes InitialBoundary Value Problem Giovanni P. Galdi Department of Mechanical Engineering University of Pittsburgh, USA Rechts auf zwei hohen Felsen befinden sich Schlösser,
More information