Orthogonl Functions My 9, 3 1 Introduction A prticulrly importnt clss of pplictions of liner lgebr within mthemtics rises from treting functions s vectors, with the rules (f + g)(x) = f(x) + g(x) (αf)(x) = αf(x), where α is sclr. f g = f(x)g(x) dx. Here, the intervl [, b] over which we integrte is regrded s fixed within ny prticulr discussion. Two importnt exmples re 1. [, b] = [, 1] nd the functions in question re tken to be polynomils.. [, b] = [, π] nd the functions re periodic, e.g. trigonometric polynomils. A set of functions on [,b] is linerly independent if no liner combintion of those functions is identiclly on [,b]. In prticulr, the powers of x re linerly independent on ny intervl, s re the functions sin(nx), cos(nx) nd 1 on [, π], for n nd m positive integers. However, the ltter set hs n dditionl importnt property: they re orthogonl. Tht is π sin(nx) cos(mx) dx = π sin(nx) sin(mx) dx = Preliminry version: subject to lter expnsion π cos(nx) cos(mx) dx = 1
unless m = n, in which cse π sin(nx) cos(nx) dx = ; π sin (nx) dx = π cos (nx) dx = π. The powers of x re not orthogonl on ny intervl. However if we confine our ttention to ny prticulr intervl, such s [, 1], we cn use the Grm- Schmidt orthogonliztion lgorithm to produce orthogonl polynomils. Suppose we hve orthogonl functions {f i } i n, nd function g = n i= if i, which is liner combintion of the functions f i. Then g(x)f j (x) dx = n i f i (x)f j (x) dx = j f j (x) dx, i= so tht j = g(x)f j(x) dx f j(x) dx, nd the coefficients j cn be recovered by integrtion. Complex Exponentils nd the Discrete Fourier Trnsform In deling with periodic functions on [, π], it is prticulrly useful to consider the complex exponentil functions e inx s n vries over the integers. From the power series e x = x n n = 1 + x + x + x3 3 +, cos(x) = 1 x + x4 4 x6 6 +, nd sin(x) = x x3 3 + x5 5 x7 7 +, we recover the identity e ix = cos(x) + i sin(x),
which is lso consistent with the trigonometric ddition formule. Becuse we re considering functions tht tke complex vlues (lthough only s functions of the rel vrible x), we re, for the first time, considering complex vector spce. This mens tht sclrs re llowed to be complex numbers, nd lso requires n djustment to the dot product. We recll tht if z = x + iy, then z = x iy, nd set f g = π f(x)g(x) dx. With this definition, e inx e imx = π if n = m, nd otherwise, since e inx = e inx. In prticulr, if f(x) = n e inx, then f e imx = π f(x)e imx = π m, nd we cn recover the coefficients m by integrtion, s in the strictly rel cse. Let us now consider how to best pproximte the coefficients m for m 1 when we know the vlues f( πk ) for rel vlued function f. From the fct tht f is rel vlues, it follows tht m = m for ll m, so tht we need only determine the coefficients m for m. The next observtion is tht we cn pproximte the integrl tht gives us π m by the Riemnn sum so tht m 1 1 k= π 1 k= f( πk πimk )e f( πk πimk )e, = 1 1 k= f( πk mk )W, where we define W to be e πi. If we set v to be the vector whose k t h component is f( π(k 1) ) nd M to be the complex symmetric mtrix whose (m, k) th entry is W (m 1)(k 1), the components of M v re pproximtions to π m m 1. This pproximtion is clled the discrete Fourier trnsform. It tends to be rther crude for vlues of m close to. 3
3 The Fst Fourier Trnsform In order to pprecite wht follows, one must think of s very lrge (perhps s lrge s 1 6 ), so tht the multiplictions tht re necessry to compute the discrete Fourier trnsform will tke significnt time. We will see now tht there is trick tht llows us to do the computtion with mny fewer multiplictions if is power of. The ssumption is tht multipliction is much more time consuming thn ddition, so tht it is much more efficient to compute (b + c + d) then b + c + d. It will be more pproprite for wht follows to shift the components nd indices so tht they go from to 1, v k = f( π(k ) nd the (m, k)th component of M is W (m)(k). We mke the observtion tht, for even, W = W, nd W = 1. We write v even for the vector formed from the even components of v tken in order, nd v odd for the vector formed from the odd components of v, lso tken in order. From this it follows tht the first components of M v re the components of the vector of length whose j t h component, (ending with j = 1) is (M v even ) j + W j (M v odd ) j, while the (j + )th component of M v (strting with j = ) is (M v even ) j W j (M v odd ) j. If we write µ() for the minimum number of multiplictions required to compute M v, it follows from the foregoing tht, for even, µ() µ( ) +. We re not counting the multiplictions required to compute the powers of W, becuse these cn be stored nd used repetedly, nd we re relly interested in the speed for which the computtion cn be crried out for fixed nd constntly chnging v. Since M = 1 1 1 1, it follows tht µ() 1 nd, from the foregoing by induction, tht µ( p ) p p 1. 4
4 Problems 1. Let f(x) be the function on [, 1] defined by f(x) = x, x 1 ; f(x) = 1 x, 1 x 1. Find the orthogonl projection of f on the spce of qurtic polynomils. Begin by finding n orthogonl bsis for the spce of qurtic polynomils by pplying Grm-Schmidt to the functions {1, x, x, x 3, x 4 }. You re encourged to use MATLAB for this purpose.. Let f(x) = π π x for x π. () Find the Fourier coefficients m = 1 direct integrtion for m 4. π π f(x)e imx dx of f by (b) Use the vlues of f t multiples of π nd the discrete Fourier trnsform to pproximte the sme coefficients. 5