. Hilbert spaces. Defiitios.. Vector spaces Defiitio. Vector space (*9&)8& "(9/). A vector space over a field F is a set V that has the structure of a additive group. Moreover, a product F V V, deoted (a,x) ax, is defied, satisfyig: Distributivity i V : a(x +y) = ax +ay. Distributivity i F : (a +b)x = ax +bx. Homogeeity i F : a(b x) = (ab)x. Scalar uit elemet: x = x. The elemets of V are called vectors; the elemets of F are called scalars. Throughout this course the field F will be either the field of complex umbers C (V is a complex vector space) or the field of reals R (V is a real vector space). Defiitio.2 Let V be a vector space. A (fiite) set of vectors {x,...,x } V is called liearly idepedet (;*9!**-.**&-; *;-") if the idetity k= a k x k = implies that a k = for all k. Otherwise, this set of elemets is said to be liearly depedet.
8 Hilbert spaces Defiitio.3 If a vector space V cotais liearly idepedet vectors ad every + vectors are liearly depedet, the we say that V has dimesio : dimv =. If dimv for every N (for every there exist liearly idepedet vectors) the we say that V has ifiite dimesio. Propositio. Let V be a vector space. Suppose that dimv = ad let (x,...,x ) be liearly idepedet (a basis). The, every y V has a uique represetatio y = a k x k. k= Proof. Obvious. Defiitio.4 Let V be a vector space. A subset Y V is called a vector subspace (*9&)8& "(9/ ;;) (or a liear subspace) if it is a vector space with respect to the same additio ad scalar multiplicatio operatios (the vector space structure o Y is iherited from the vector space structure o V ). Propositio.2 Let V be a vector space. A subset Y V is a vector subspace if ad oly if Y ad for all y,y 2 Y ad a,a 2 F, a y +a 2 y 2 Y, i.e., the subset Y is closed uder liear combiatios. Proof. Easy. Commet. By defiitio, every liear subspace is closed uder vector space operatios (it is algebraically closed). This should ot be cofused with the topological otio of closedess, which is defied oce we edow the vector space with a topology. A liear subspace may ot be closed i the topological sese. Defiitio.5 Let V ad Y be vector spaces over the same field F ; let D V be a vector subspace. A mappig T D Y is said to be a liear trasformatio (;*9!**- %8;3%) if for all x,x 2 D ad a a 2 F : T (a x +a 2 x 2 ) = a T (x )+a 2 T (x 2 ).
. Defiitios 9 The set D is called the domai (.&(;) of T. The set Image(T ) = {T (x) x D} Y is called the image (%&/;) of T. If D = V = Y we call T a liear trasformatio o V. If Y = F we call T a liear fuctioal (*9!**- -&*78&5). Commet.2 Liear trasformatios preserve the vector space operatios, ad are therefore the atural isomorphisms i the category of vector spaces. This should be kept i mid, as the atural isomorphisms may chage as we edow the vector space with additioal structure. Iverse trasformatio If T Domai(T ) Image(T ) is oe-to-oe (ijective) the we ca defie a iverse trasformatio such that for all x Domai(T ) ad y Image(T ). T Image(T ) Domai(T ), T (Tx) = x ad T (T y) = y Notatio. I these otes we will use A B to deote ijectios, A B to deote surjectios, ad A B to deote bijectios. Propositio.3 Let V be a vector space ad D V a liear subspace. Let T D Y be a liear trasformatio. The, Image(T ) is a liear subspace of Y. Proof. Sice D, Image(T ) T () =. Let x,y Image(T ). By defiitio, there exist u,v D such that x = T (u) ad y = T (v). By the liearity of T, for every a,b F : Image(T ) T (au+bv) = ax +by. Thus, Image(T ) is closed uder liear combiatios. A digressio o categories: A category is a algebraic structure that comprises objects that are liked by morphisms. A category has two basic properties: the ability to compose the morphisms associatively ad the existece of a idetity morphism for each object. A simple example is the category of sets, whose morphisms are fuctios. Aother example is the category of groups, whose morphisms are homomorphisms. A third example is the category of topological spaces, whose morphisms are the cotiuous fuctios. As you ca see, the chose morphisms are ot just arbitrary associative maps. They are maps that preserve a certai structure i each class of objects.
Hilbert spaces..2 Normed spaces A vector space is a set edowed with a algebraic structure. We ow edow vector spaces with additioal structures all of them ivolvig topologies. Thus, the vector space is edowed with a otio of covergece. Defiitio.6 Metric space. A metric space (*9)/ "(9/) is a set X, edowed with a fuctio d X X R, such that Positivity: d(x,y) with equality iff x = y. Symmetry: d(x,y) = d(y,x). Triagle iequality: d(x,y) d(x,z) + d(z,y). Please ote that a metric space does ot eed to be a vector space. O the other had, a metric defies a topology o X geerated by ope balls, B(x,r) = {y X d(x,y) < r}. As topological spaces, metric spaces are paracompact (every ope cover has a ope refiemet that is locally fiite), Hausdorff spaces, ad hece ormal (give ay disjoit closed sets E ad F, there are ope eighborhoods U of E ad V of F that are also disjoit). Metric spaces are first coutable (each poit has a coutable eighborhood base) sice oe ca use balls with ratioal radius as a eighborhood base. Defiitio.7 Norm. A orm (%/9&) over a vector space V is a mappig V R such that Positivity: x with equality iff x =. Homogeeity: ax = ax. Triagle iequality: x +y x+y. A ormed space (*/9& "(9/) is a pair (V, ), where V is a vector space ad is a orm over V. A orm is a fuctio that assigs a size to vectors. Ay orm o a vector space iduces a metric: Propositio.4 Let (V, ) be a ormed space. The d(x,y) = x y
. Defiitios is a metric o V. Proof. Obvious. The coverse is ot ecessarily true uless certai coditios hold: Propositio.5 Let V be a vector space edowed with a metric d. If the followig two coditios hold: Traslatio ivariace: d(x + z,y + z) = d(x,y) Homogeeity: d(ax,ay) = ad(x,y), the x = d(x,) is a orm o V. Exercise. Prove Prop..5...3 Ier-product spaces Vector spaces are a very useful costruct (e.g., i physics). But to be eve more useful, we ofte eed to edow them with structure beyod the otio of a size. Defiitio.8 Ier product space. A complex vector field V is called a ier-product space (;*/*5 %-5,/ "(9/) if there exists a product (, ) V V C, satisfyig: Symmetry: (x,y) = (y,x). Biliearity: (x +y,z) = (x,z)+(y,z). Homogeeity: (ax,y) = a(x,y). Positivity: (x,x) with equality iff x =. A ier-product space is also called a pre-hilbert space. Propositio.6 A ier-product (H,(, )) space satisfies:
2 Hilbert spaces (x,y+z) = (x,y)+(x,z). (x,ay) = a(x,y). Proof. Obvious. Propositio.7 Cauchy-Schwarz iequality. Let (H,(, )) be a ierproduct space. Defie = (, ) 2. The, for every x,y H, (x,y) xy. Proof. There are may differet proofs to this propositio 2. For x,y H defie u = xx ad v = yy. Usig the positivity, symmetry, ad biliearity of the ier-product: That is, (u (u,v)v,u (u,v)v) = +(u,v) 2 (u,v) 2 (u,v) 2 = (u,v) 2. (x,y) xy. Equality holds if ad oly if u ad v are co-liear, i.e., if ad oly if x ad y are co-liear. Corollary.8 Triagle iequality. I every ier product space H, x +y x+y. Proof. Applyig the Cauchy-Schwarz iequality x +y 2 = (x +y,x +y) = x 2 +y 2 +2Re(x,y) x 2 +y 2 +2(x,y) x 2 +y 2 +2xy = (x+y) 2. 2 There is a book called The Cauchy-Schwarz Master Class which presets more proofs that you wat.
. Defiitios 3 Corollary.9 A ier-product space is a ormed space with respect to the orm: x = (x,x) 2. Proof. Obvious. Thus, every ier-product space is automatically a ormed space ad cosequetly a metric space. The (default) topology associated with a ier-product space is that iduced by the metric (i..e, the ope sets are geerated by ope metric balls). Exercise.2 Show that the ier product (H,(, )) is cotiuous with respect to each of its argumets: ( x,y H)( e > )( d > ) ( z H z x < d)((z,y) (x,y) < e). Exercise.3 Let (H,(, )) be a complex ier-product space. Defie x,y = Re(x,y). Show that (H,, ) is a real ier-product space. Exercise.4 Prove that if a collectio of o-zero vectors {x,...,x } i a ier-product space are mutually orthogoal the they are liearly idepedet. Exercise.5 Prove that i a ier-product space x = iff (x,y) = for all y. Exercise.6 Let (H,(, )) be a ier-product space. Show that the followig coditios are equivalet: (x,y) =. x +ly = x ly for all l C. x x +ly for all l C.
4 Hilbert spaces Exercise.7 Cosider the vector space V = C [,] (cotiuouslydifferetiable fuctios over the uit iterval) ad defie the product (, ) V V C: Is (, ) a ier-product? ( f,g) = f (x)g(x)dx. Set V = { f V f () = }. Is (V,(, )) a ier-product? Propositio. Parallelogram idetity (;*-*"8/% &*&&:). I every ierproduct space (H,(, )): x +y 2 +x y 2 = 2x 2 +y 2. (This equatio is called the parallelogram idetity because it asserts that i a parallelogram the sum of the squares of the sides equals to the sum of the squares of the diagoals.) x +y y x y x Proof. For every x,y H: x ±y 2 = x 2 +y 2 ±2 Re(x,y). The idetity follows from addig both equatios. A ier product defies a orm. What about a coverse? Suppose we are give the the orm iduced by a ier product. Ca we recover the ier product? The aswer is positive, as show by the followig propositio:
. Defiitios 5 Propositio. Polarizatio idetity (%*7'*9-&5% ;&%'). I a ier-product space (H,(, )), (x,y) = 4 x +y2 x y 2 +ıx +ıy 2 ıx ıy 2. Proof. It is easy to see that Settig y ıy, x +y 2 x y 2 = 4 Re(x,y). x +ıy 2 x ıy 2 = 4 Reı(x,y) = 4 Im(x,y). Multiplyig the secod equatio by ı ad addig it to the first equatio we obtai the desired result. Commet.3 I a real ier-product space: (x,y) = 4 x +y2 x y 2. Defiitio.9 Let (H,(, )) be a ier-product space. x,y H are said to be orthogoal (.*"7*) if (x,y) = ; we deote x y. Propositio.2 Orthogoal vectors i a ier-product space satisfy Pytagoras law: x +y 2 = x 2 +y 2. Proof. Obvious. Exercise.8 Show that a orm over a real vector space V is iduced from a ier-product over V if ad oly if the parallelogram law holds. Hit: set (x,y) = 2 x +y2 x 2 y 2, ad show that it is a ier product ad that the iduced orm is ideed.
6 Hilbert spaces Exercise.9 Show that the `p spaces (the spaces of sequeces with the appropriate orms) ca be tured ito a ier-product space (i.e., the orm ca be iduced from a ier-product) oly for p = 2...4 Hilbert spaces Defiitio. Hilbert space. A complete ier-product space is called a Hilbert space. (Recall: a space is complete (.-:) if every Cauchy sequece coverges.) Commet.4 A ier-product space (H,(, )) is a Hilbert space if it is complete with respect to the metric d(x,y) = (x y,x y) 2. Completeess is a property of metric spaces. A sequece (x ) H is a Cauchy sequece if for all e > there exists a N N such that for every m, > N: x x m < e. Exercise. Let H,...,H be a fiite collectio of ier-product spaces. Defie the space H = H H, alog with coordiate-wise vector space operatios. Defie a product (, ) H H H C: ((x,...,x ),(y,...,y )) H = (x k,y k ) Hk. Show that (, ) H is a ier-product o H. Show that covergece i H is equivalet to compoet-wise covergece i each of the H k. Show that H is complete if ad oly if all the H k are complete. k= We metioed the fact that a ier-product space is also called a pre-hilbert space. The reaso for this omeclature is the followig theorem: ay ier-product space ca be completed caoically ito a Hilbert space. This completio is aalogous to the completio of the field of ratioals ito the field of reals.
. Defiitios 7 Theorem.3 Completio. Let (G,(, ) G ) be a ier-product space. The, there exists a Hilbert space (H,(, ) H ), such that: There exists a liear ijectio T G H, that preserves the ier-product, (x,y) G = (Tx,Ty) H for all x,y G (i.e., elemets i G ca be idetified with elemets i H ). Image(T ) is dese i H (i.e., G is idetified with almost all of" H ). Moreover, the iclusio of G i H is uique: For ay liear ier-product preservig ijectio T G H where H is a Hilbert space ad Image(T ) is dese i H, there is a liear isomorphism S H H, such that T = S T (i.e., H ad H are isomorphic i the category of ier-product spaces). I other words, the completio G is uique modulo isomorphisms. Proof. We start by defiig the space H. Cosider the set of Cauchy sequeces (x ) i G. Two Cauchy sequeces (x ) ad (y ) are defied to be equivalet (deoted (x ) (y )) if lim x y =. It is easy to see that this establishes a equivalece relatio amog all Cauchy sequeces i G. We deote the equivalece class of a Cauchy sequece (x ) by [x ] ad defie H as the set of equivalece classes. We edow H with a vector space structure by defiig a[x ]+b[y ] = [ax +bz ]. (It is easy to see that this defiitio is idepedet of represetig elemets.) Let (x ) ad (y ) be Cauchy sequece i G. Cosider the series This limit exists because lim (x,y ) G. (x,y ) G (x m,y m ) G = (x,y ) G (x,y m ) G +(x,y m ) G (x m,y m ) G (triagle ieq.) (x,y y m ) G +(x x m,y m ) G (Cauchy-Schwarz) x G y y m G +x x m y m G, Sice Cauchy sequeces are bouded (easy!), there exists a M > such that for all m,, (x,y ) G (x m,y m ) G M (y y m G +x x m G ). It follows that (x,y ) G is a Cauchy sequece i F, hece coverges (because F is complete).
8 Hilbert spaces Moreover, if (u ) (x ) ad (v ) (y ) the, (x,y ) G (u,v ) G = (x,y ) G (x,v ) G +(x,v ) G (u,v ) G from which follows that (x,y v ) G +(x u,v ) G x G y v G +x v G v G, lim (x,y ) G = lim (u,v ) G. Thus, we ca defie uambiguously a product (, ) H H F ; ([x ],[y ]) H = lim (x,y ) G. It remais to show that (, ) H is ideed a ier product (do it). The ext step is to defie the iclusio T G H. For x G let Tx= [(x,x,...)], amely, it maps every vector i G ito the equivalece class of a costat sequece. By the defiitio of the liear structure o H, T is liear. It is preserves the ier-product as (Tx,Ty) H = lim ((Tx),(Ty) ) G = lim (x,y) G = (x,y) G. The ext step is to show that Image(T ) is dese i H. Let h H ad let (x ) be a represetative of h. Sice (x ) is a Cauchy sequece i G, lim Tx h H = lim lim x x k G =. k which proves that Tx h, ad therefore Image(T ) is dese i H. The ext step is to show that H is complete. Let (h ) be a Cauchy sequece i H. For every let (x,k ) be a Cauchy sequece i G i the equivalece class of h. Sice Image(T ) is dese i H, there exists for every a y G, such that It follows that Ty h H = lim k y x,k G. y y m G = Ty Ty m H Ty h H +h h m H +h m Ty m H h h m H + + m, i.e., (y ) is a Cauchy sequece i G ad therefore h = [y ] is a elemet of H.
. Defiitios 9 We will show that lim h h H =, which will prove that ay Cauchy sequece i H coverges. By defiitio, Now ad lim h h H = lim lim x,k y k G. k x,k y k G x,k y G +y y k G, lim lim x,k y G lim = ad lim k lim y y k G =. k The last step is to show the uiqueess of the completio modulo isomorphisms. Let h H. Sice Image(T ) is dese i H, there exists a sequece (y ) G, such that lim Ty h H =. It follows that (Ty ) is a Cauchy sequece i H, ad because T preserves the ier-product, (y ) is a Cauchy sequece i G. It follows that (T y ) is a Cauchy sequece i H, ad because the latter is complete (T y ) has a limit i H. This limit is idepedet of the choice of the sequece (y ), hece it is a fuctio of h, which we deote by S(h) = lim T Y. We leave it as a exercise to show that S satisfies the required properties. Exercise. Complete the missig details i the above proof...5 Examples of Hilbert spaces. The space R is a real vector space. The mappig (x,y) is a ier product. The iduced metric i= x i y i 2 d(x,y) = (x i y i ) 2 i= is called the Euclidea metric. It is kow that R is complete with respect to this metric, hece it is a Hilbert space (i fact, ay fiite-dimesioal ormed space is complete, so that the otio of completeess is oly of iterest i ifiite-dimesioal spaces).
2 Hilbert spaces 2. The space C is a complex vector space. The mappig (x,y) i= as a ier product. C edowed with this metric is a Hilbert space. 3. Cosider the space of square summable sequeces: `2 = x C N = x i y i x 2 <. It is a complex vector space with respect to poitwise operatios. We defie (x,y) = i= x i y i. This series coverges absolutely as for every fiite, the Cauchy-Schwarz iequality implies: i= x i y i i= 2 x i 2 i= 2 y i 2, ad the right had side is uiformly bouded. It ca be show that `2 is complete. Moreover, it is easy to show that the subset of ratioal-valued sequeces that have a fiite umber of o-zero terms is dese i `2, i.e., `2 is a separable (*-*952) Hilbert space (has a coutable dese subset). Exercise.2 Prove ( by had") that `2 is complete. 4. Let W be a bouded set i R ad let C(W) be the set of cotiuous complexvalued fuctios o its closure 3. This space is made ito a complex vector space by poitwise additio ad scalar multiplicatio: ( f +g)(x) = f (x)+g(x) ad (a f )(x) = a f (x). We defie o C(W) a ier product with the correspodig orm: ( f,g) = W f (x)g(x)dx, f = f (x) 2 2 dx. W 3 The fact that the domai is compact is crucial; we shall see later i this course that such a structure caot be applied for cotiuous fuctios over ope domais.
. Defiitios 2 This space is ot complete ad hece ot a Hilbert space. To show that, let x W ad r > be such that B(x,2r) W. Defie the sequece of fuctios, x x r f (x) = +(r x x ) r x x r + x x > +, which are defied for sufficietly large. f = f = x The fuctios f are cotiuous ad coverge poitwise to the discotiuous fuctio x x r f (x) = x x > r. The sequece ( f ) is a Cauchy sequece as for > m, f f m (B(x,r +m)b(x,r)) 2, which teds to zero as m,. Suppose that the space was complete. It would imply the existece of a fuctio g C(W), such that lim f g = lim f (x) g(x) 2 2 =. W By Lebesgue s bouded covergece (%/&2(% ;&2,;%% )5:/) theorem lim f g = lim f (x) g(x) 2 2 =, W i.e., g = f a.e., which is a cotradictio. The completio of C(W) with respect to this metric is isomorphic to the Hilbert space L 2 (W) of square itegrable fuctios. Commet.5 The costructio provided by the completio theorem is ot coveiet to work with. We prefer to work with fuctios rather tha with equivalece classes of Cauchy sequeces of fuctios. TA material. Hilbert-Schmidt matrices. Let M be a collectio of all ifiite matrices over C that oly have a fiite umber of o-zero elemets. For A M we deote by (A) the smallest umber for which A ij = for all i, j > (A). (i)
22 Hilbert spaces Show that M is a vector space over C with respect to matrix additio ad scalar multiplicatio. (ii) Defie (A,B) = Tr(AB ) ad show that it is a ier-product. (iii) Show that M is ot complete. (iv) Show that it is possible to idetify the completio H of M with the set alog with the ier-product H = A = (a ij ) i, j= (A,B) = i, j= i, j= A ij B ij. a ij 2 <, This space is kow as the space of Hilbert-Schmidt matrices. Exercise.3 Prove that every fiite-dimesioal ier product space is complete (ad hece a Hilbert space). TA material.2 Sobolev spaces. Edow the space C k (R) with the ierproduct d i f d i g ( f,g) = dx i k R dx i dx i Its completio is deoted H k (R) (or W k,2 (R)). Defie the otio of a weak derivative ad show that if it exists, the it is uique. Show how to idetify H k (R) with the space of fuctios that have square-itegrable k weak derivatives..2 Covexity ad projectio Orthogoality is oe of the cetral cocepts i the theory of Hilbert spaces. Aother cocept, itimately related to orthogoality, is orthogoal projectio (;"7* %-)%). Before gettig to projectios we eed to develop the otio of a covex set. Covexity, is a purely algebraic cocept, but as we will see, it iteracts with the topology iduced by the ier-product..2. Covexity Defiitio. Covex set. Let V be a vector space. A subset C V is called covex (9&/8) if x,y C ad t tx+( t)y C.
.2 Covexity ad projectio 23 (The segmet that coects ay two poits i C is i C ). Differetly stated, for all t [,]: tc +( t)c C. A C B Lemma.4 For ay collectio of sets {C a } ad D a ad every t R: t C a = tc a, a A a A ad C a + D a (C a +D a ). a A a A a A Proof. First, Secod, if t C a = {tx x C a a} = tc a. a A a A x C a + D a, a A a A the there is a c C a for all a ad a d D b for all b, such that x = c + d. Now c+d = C a +D a for all a, hece x (C a +D a ). a A Propositio.5 Covexity is closed uder itersectios. Let V be a vector space. Let {C a V a A} be a collectio of covex sets (ot ecessarily coutable). The C = C a a A is covex.
24 Hilbert spaces Proof. A obscure" proof relies o Lemma.4. For all t [,]: tc +( t)c = t C a +( t) C a a A a A = tc a + ( t)c a a A a A (tc a +( t)c a ) a A C a = C. a A Now for a more trasparet proof: let x,y C. By defiitio: Sice all the C a are covex: ( a A)(x,y C a ). ( a A)( t )(tx+( t)y C a ). Iterchagig the order of the quatifiers, which implies that ( t )( a A)(tx+( t)y C a ), ( t )(tx+( t)y C ). Propositio.6 Covex sets are closed uder covex liear combiatios. Let V be a vector space. Let C V be a covex set. The for every (x,...,x ) C ad every o-egative (t,...,t ) real umbers that sum up to, i= t i x i C. (.) Proof. Equatio (.) holds for = 2 by the very defiitio of covexity. Suppose (.) were true for = k. Give (x,...,x k+ ) C ad (t,...,t k+ ), defie t = k i= t i. The, k+ i= t i =, k+ i= t i x i = k i= k t i x i +t k+ x k+ = t i= t x i +( t)x k+ t i C C
.2 Covexity ad projectio 25 Propositio.7 Let (X, ) be a ormed space ad C X a covex subset. The, The closure C is covex. The iterior C is covex. Commet.6 Iterior ad closure are topological cocepts, whereas covexity is a vector space cocept. The coectio betwee the two stems from the fact that a ormed space has both a topology ad a vector space structure. Proof. Let x,y C. For every e > there are poits x e,y e C with x x e < e ad y y e < e. Let t. The, tx e +( t)y e C ad (tx+( t)y) (tx e +( t)y e ) tx x e +( t)y y e < e, which implies that tx+( t)y C, hece C is covex. Let x,y C. By defiitio of the iterior there exists a r > such that B(x,r) C ad B(y,r) C. Sice C is covex, t [,] tb(x,r)+( t)b(y,r) C, but B(tx+( t)y,r) tb(x,r)+( t)b(y,r), which proves that tx+( t)y C. Hece, C is covex. x tx+( t)y y Examples. Every ope ball B(a,r) i a ormed vector space is covex, for if x,y B(a,r), the for all t : tx+( t)y a = t(x a)+( t)(y a) t x a+( t)y a < r.
26 Hilbert spaces Every liear subspace of a vector space is covex, because it is closed uder ay liear combiatios ad i particular, covex oes. For example, let V = L 2 [,] ad let C be the subset of polyomials. C is a liear subspace of V, hece it is covex. Let W R be a domai ad cosider the Hilbert space L 2 (W). The subset of fuctios that are o-egative (up to a set of measure zero) is covex (but it is ot a liear subspace). Exercise.4 Let V be a vector space ad C V. The covex hull (9W/8) of C is defied by Cov(C) = {x V x is a covex combiatios of elemets i C}. Show that Cov(C) is the smallest covex set that cotais C. Exercise.5 Prove Carathéodory s theorem: let A R ad let x Cov(A). The x is a covex combiatio of + poits i A or less. (Hit: suppose that x is a covex combiatio of x,...,x p A, where p > +. Use the fact that {x i x } p i=2 are liearly depedet to show that x ca be writte as a covex sum of p poits). Show that Carathéodory s theorem may fail if the dimesio of the vector space is ifiite..2.2 Orthogoal projectio Defiitio.2 Let (H,(, )) be a ier-product space, ad let S H (it ca be ay subset; ot ecessarily a vector subspace). We deote by S the set of vectors that are perpedicular to all the elemets i S, S = {x H (x,y) = y S}.
.2 Covexity ad projectio 27 Propositio.8 Let (H,(, )) be a ier-product space. Let S H. The set S is a closed liear subspace of H, ad S S {}, Proof. We start by showig that S is a liear subspace. Let x,y S, i.e., For all a,b F, z S (x,z) = (y,z) =. z S (ax +by,z) = a(x,z)+b(y,z) =, which implies that ax +by S, i.e., S is a liear subspace. We ext show that S is closed. Let (x ) be a sequece i S that coverges to x H. By the cotiuity of the ier product, z S (x,z) = lim (x,z) =, i.e., x S. Suppose x S S. As a elemet i S, x is orthogoal to all the elemets i S, ad i particular to itself, hece (x,x) =, which by the defiig property of the ier-product implies that x =. Exercise.6 Show that S = {x}. x S Exercise.7 Show that if M ad N are closed subspaces of a Hilbert space H, ad N is fiite dimesioal, the M +N is a closed subspace (hit: iductio o the dimesio of N). Show that M +N may ot be closed if N is ifiite dimesioal.
28 Hilbert spaces The followig theorem states that give a closed covex set C i a Hilbert space H, every poit i H has a uique poit i C that is the closest to it amog all poits i C : Theorem.9 Let (H,(, )) be a Hilbert space ad C H closed ad covex. The, x H!y C such that x y = d(x,c ), where d(x,c ) = if x y. y C The mappig x y is called the projectio (%-)%) of x oto the set C ad it is deoted by P C. Commet.7 Note the coditios of this theorem. The space must be complete ad the subset must be covex ad closed. We will see how these coditios are eeded i the proof. A very importat poit is that the space must be a ier-product space. Projectios do ot geerally exist i (complete) ormed spaces. Proof. We start by showig the existece of a distace miimizer. By the defiitio of the ifimum, there exists a sequece (y ) C satisfyig, lim d(x,y ) = d(x,c ). Sice C is covex, 2 (y +y m ) C for all m,, ad therefore, 2 (y +y m ) x d(x,c ). By the parallelogram idetity (which is where the ier-product property eters), a b 2 = 2(a 2 +2b 2 ) a+b 2, ad so y y m 2 = (y x) (y m x) 2 = 2y x 2 +2y m x 2 y +y m 2x 2 2y x 2 +2y m x 2 2d(x,C ) m,. It follows that (y ) is a Cauchy sequece ad hece coverges to a limit y (which is where completeess is essetial). Sice C is closed, y C. Fially, by the cotiuity of the orm, x y = lim x y = d(x,c ), which completes the existece proof of a distace miimizer. Next, we show the uiqueess of the distace miimizer. Suppose that y,z C both satisfy y x = z x = d(x,c ).
.2 Covexity ad projectio 29 By the parallelogram idetity, i.e., y+z 2x 2 +y z 2 = 2y x 2 +2z x 2, y+z 2 x2 = d 2 (x,c ) 4 y z. If y z the (y+z)2, which belogs to C is closer to x tha the distace of x from C, which is a cotradictio. x y C (y+z)/2 z TA material.3 Projectios i Baach spaces. The existece of a uique projectio does ot hold i geeral i complete ormed spaces (i.e., Baach spaces). A distace miimizer does exist i fiite-dimesioal ormed spaces, but it may ot be uique)=. I ifiite-dimesioal Baach spaces distace miimizers may fail to exist. TA material.4 Coditioal expectatios. The followig is a importat applicatio of orthogoal projectios. Let (W,F,P) be a probability space, ad let A F be a sub-s-algebra. Let X W C be a radom variable (i.e., a measurable fuctio) satisfyig X <. The radom variable Y W C is called the coditioal expectatio of X with respect to the s-algebra A if (i) Y is A -measurable, ad (ii) for every A A, A YdP= A XdP. Prove that the coditioal expectatio exists ad is uique i L (W,F,P). (Note that L is ot a Hilbert space, so that the costructio has to start with the subspace L 2 (W,F,P), ad ed up with a desity argumet.) Propositio.2 Let H be a Hilbert space. Let C be a closed covex set. The mappig P C H C is idempotet, P C P C = P C. Proof. Obvious, sice P C = Id o C.
3 Hilbert spaces Exercise.8 Let (H,(, )) be a Hilbert space ad let P A ad P B be orthogoal projectios o closed subspaces A ad B. Show that if P A P B is a orthogoal projectio the it projects o A B. Show that P A P B is a orthogoal projectio if ad oly if P A P B = P B P A. Show that if P A P B is a orthogoal projectio the P A +P B P A P B is a orthogoal projectio o A+B. Fid a example i which P A P B P B P A. The ext propositio has a geometric iterpretatio: the segmet coectig a poit x C with its projectio P C x makes a obtuse agle with ay segmet coectig P C x with aother poit i C. The propositio states that this is i fact a characterizatio of the projectio. Propositio.2 Let C be a closed covex set i a Hilbert space (H,(, )). The for every x H, z = P C x if ad oly if z C ad y C Re(x z,y z). x z = P C x y C Proof. Suppose first that z = P C x. By defiitio z C. Let y C. Sice C is covex the ty+( t)z C for all t [,], ad sice z is the uique distace miimizer from x i C : > x z 2 x (ty+( t)z) 2 = x z 2 (x z) t(y z) 2 = t 2 y z 2 +2t Re(x z,y z). Thus, for all < t, Re(x z,y z) < 2 ty z2.
.2 Covexity ad projectio 3 Lettig t we get that Re(x z,y z). Coversely, suppose that z C ad that for every y C, Re(x z,y z). For every y C, x y 2 x z 2 = (x z)+(z y) 2 x z 2 = y z 2 2Re(x z,y z), which implies that z is the distace miimizer, hece z = P C x. Corollary.22 Projectios are distace reducig. Let C be a closed covex set i a Hilbert space (H,(, )). The for all x,y H, Re(P C x P C y,x y) P C x P C y 2 ad P C x P C y 2 x y 2. y P C y x P C x C Proof. By Propositio.2, with P C y as a arbitrary poit i C, Re(x P C x,p C y P C x). Similarly, with P C x as a arbitrary poit i C Re(y P C y,p C x P C y). Addig up both iequalities: Re((x y) (P C x P C y),p C x P C y), which proves the first assertio. Next, usig the first assertio ad the Cauchy-Schwarz iequality, P C x P C y 2 Re(P C x P C y,x y) (P C x P C y,x y) P C x P C yx y, ad it remais to divide by P C x P C y.
32 Hilbert spaces The ext corollary characterizes the projectio i the case of a closed liear subspace (which is a particular case of a closed covex set). Corollary.23 Let M by a closed liear subspace of a Hilbert space (H,(, )) The, y = P M x if ad oly if y M ad x y M. x y m y = P M x y+m M Proof. Let y M ad suppose that x y M. The, for all m M : hece y = P M x by Propositio.2. (x y, m y M ) =, Coversely, suppose that y = P M x ad let m M. By Propositio.2, Re(y x,y m). We may replace m by y m M, hece for all m M : Re(y x,m). Sice we may replace m by ( m), it follows that for all m M : Re(y x,m) =. Replacig m by ım we obtai Im(y x,m) =. Fiite dimesioal case This last characterizatio of the projectio provides a costructive way to calculate the projectio whe M is a fiite-dimesioal subspace (hece a closed subspace). Let = dimm ad let (e,...,e )
.2 Covexity ad projectio 33 be a set of liearly idepedet vectors i M (i.e., a basis for M ). Let x H. The, x P M x is orthogoal to each of the basis vectors: (x P M x,e`) = for ` =,...,. Expadig P M x with respect to the give basis: we obtai, P M x = a k e k, k= (x,e`) = a k (e k,e`) = k= for ` =,...,. The matrix G whose etries are G ij = (e i,e j ) is kow as the Gram matrix. Because the e i s are liearly idepedet, this matrix is o-sigular, ad a k = G k` (x,e`), i.e., we have a explicit expressio for the projectio of ay vector: `= P M x = k= `= G k` (x,e`)e k. Exercise.9 Let H = L 2 (R) ad set M = { f H f (t) = f ( t) a.e.}. Show that M is a closed subspace. Express the projectio P M explicitly. Fid M. Exercise.2 What is the orthogoal complemet of the followig sets of L 2 [,]? The set of polyomials. The set of polyomials i x 2. The set of polyomials with a =. The set of polyomials with coefficiets summig up to zero.
34 Hilbert spaces Theorem.24 Projectio theorem (%-)%% )5:/). Let M be a closed liear subspace of a Hilbert space (H,(, )). The every vector x H has a uique decompositio x = m+ m M, M. Furthermore, m = P M x. I other words, H = M M. Proof. Let x H. By Corollary.23 x P M x M, hece x = P M x +(x P M x) satisfies the required properties of the decompositio. Next, we show that the decompositio is uique. Assume x = m + = m 2 + 2, where m,m 2 M ad, 2 M. The, M m m 2 = 2 M. Uiqueess follows from the fact that M M = {}. TA material.5 Show that the projectio theorem does ot holds whe the coditios are ot satisfied. Take for example H = `2, with the liear subspace M = {(a ) `2 N > Na = }. This liear subspace it ot closed, ad its orthogoal complemet is {}, i.e., M M = M H. Corollary.25 For every liear subspace M of a Hilbert space H, (M ) = M.
.2 Covexity ad projectio 35 Proof. Let m M. By the defiitio of M : which implies that m (M ), i.e., M (m,) =, M (M ). By Propositio.8 ay orthogoal complemet is closed. If a set is cotaied i a closed set, so is its closure (prove it!), M (M ). Let x (M ). Sice M is a closed liear subspace, there exists a (uique) decompositio x = m+, m M (M ). Takig a ier product with, usig the fact that m : Sice M M, the 2 = (x,). (M ) (M ) (the smaller the set, the larger its orthogoal complemet). Thus (M ), ad therefore x. It follows that =, which meas that x M, i.e., This completes the proof. (M ) M. Corollary.26 Let M be a closed liear subspace of a Hilbert space (H,(, )). The, every x H has a decompositio x = P M x +P M x, ad x 2 = P M x 2 +P M x 2. Proof. As a cosequece of the projectio theorem, usig the fact that both M ad M are closed ad the fact that (M ) = M : x = P M x +(x P M x) M M x = (x P M x) M + P M x. M
36 Hilbert spaces By the uiqueess of the decompositio, both decompositios are idetical, which proves the first part. The secod idetity follows from Pythagoras law. Corollary.27 A projectio is liear, orm-reducig ad idempotet. Let M be a closed liear subspace of a Hilbert space (H,(, )). The the projectio P M is a liear operator satisfyig P 2 M = P M, ad x H P M x x. Proof. We have already see that P M is idempotet. The orm reducig property follows from x 2 = P M x 2 +P M x 2 P M x 2. It remais to show that P M is liear. Let x,y H. It follows from Corollary.26 that x = P M x +P M x hece y = P M y+p M y, x +y = (P M x +P M y) +(P M x +P M y). M M O the other had, it also follows from Corollary.26 that By the uiqueess of the decompositio, x +y = P M (x +y) +P M (x +y). M M P M (x +y) = P M x +P M y. Similarly, but also ax = a P M x +a P M x, ax = P M (ax)+p M (ax), ad from the uiqueess of the decompositio, P M (ax) = a P M x.
.2 Covexity ad projectio 37 The ext theorem shows that the last corollary is i fact a characterizatio of projectios: there is a oe-to-oe correspodece betwee closed subspaces of H ad orthogoal projectios. Theorem.28 Every liear, orm-reducig, idempotet operator is a projectio. Let (H,(, )) be a Hilbert space ad let P H H be a liear, orm reducig, idempotet operator. The P is a projectio o a closed liear subspace of H. Proof. The first step is to idetify the closed subspace of H that P projects oto. Defie M = {x H Px = x} N = {y H Py = }. Both M ad N are liear subspaces of H. If x M N the x = Px =, amely, M N = {}, Both M ad N are closed because for every x,y H : Px Py = P(x y) x y, from which follows that if x M is a sequece with limit x H, the x Px = lim x Px = lim Px Px lim x x =, i.e., Px = x, hece x M. By a similar argumet ee show that N is closed. Let x H. We write By the idempotece of P, x = Px +(Id P)x. Px M ad (Id P)x N. To prove that P = P M it remais to show that N = M (because of the uiqueess of the decompositio). Let Obviously, y N, hece (x,y) =, ad x N ad y = Px x. x 2 Px 2 = x +y 2 = x 2 +y 2 x 2, i.e., y =, i.e., x = Px, i.e., x M. We have just show that N M.
38 Hilbert spaces Take ow x M. By the projectio theorem, there exists a uique decompositio, x = u + v. N M N But sice N M, it follows that u is both i M ad i N, i.e., it is zero ad x N, amely M N. Thus N = M ad further N = (N ) = M. This cocludes the proof..3 Liear fuctioals Amog all liear maps betwee ormed spaces stad out the liear maps ito the field of scalars. The study of liear fuctioals is a cetral theme i fuctioal aalysis..3. Boudedess ad cotiuity Defiitio.3 Let (X, ) ad (X 2, 2 ) be ormed spaces ad let D X be a liear subspace. A liear trasformatio T D X 2 is said to be cotiuous if x D limty Tx 2. y x It is said to be bouded if there exists a costat C > such that x D Tx 2 Cx. If T is bouded, the the lowest boud C is called the orm of T : Tx 2 T = sup = sup T x D x x D x x 2 = sup Tx 2. x = (Recall that if X = R the we call T a liear fuctioal.) Commets. We are dealig here with ormed spaces; o ier-product is eeded. As for ow, we call T a orm, but we eed to show that it is ideed a orm o a vector space. If T is bouded the x D Tx T x. All liear operators betwee fiite-dimesioal ormed spaces are bouded. This otio is therefore oly relevat to ifiite-dimesioal cases.
.3 Liear fuctioals 39 Propositio.29 Boudedess ad cotiuity are equivalet. Let (X, ) ad (X 2, 2 ) be ormed spaces. A liear operator T D X X 2 is bouded if ad oly if it is cotiuous. Proof. Suppose T is bouded. The, Tx Ty 2 = T (x y) 2 T x y, ad y x implies Ty Tx. Suppose that T is cotiuous at D. The there exists a d > such that y B(,d) Ty 2. Usig the homogeeity of the orm ad the liearity of T : x D Tx 2 = 2 d x T d x 2 2 x 2 d x. which implies that T is bouded, T 2d. Commet.8 We have oly used the cotiuity of T at zero. This meas that if T is cotiuous at zero, the it is bouded, ad hece cotiuous everywhere. Examples.2. A orthogoal projectio i a Hilbert space is bouded, sice P M x x, i.e., P M. Sice P M x = x for x M, it follows that P M = sup P M x x= sup x M x= P M x = sup x M x= x =, P M =. 2. Let H = L 2 [,] ad let D = C[,] L 2 [,]. Defie the liear fuctioal T D R, Tf= f (). This operator is ubouded (ad hece ot-cotiuous), for take the sequece of fuctios, x x f (x) = otherwise. The f, whereas T f = f () =, i.e., T f lim f =.
4 Hilbert spaces 3. Cosider the Hilbert space H = L 2 [,] with D the subspace of differetiable fuctios with derivatives i H. Defie the liear operator T D H, (T f)(x) = f (x). This operator is ubouded. Take for example the sequece of fuctios, f (x) = 2 e x22. 2p The f = ad lim Tf =. 4. Importat example! Let H be a Hilbert space. Set y H ad defie the fuctioal: T y = (,y). This fuctioal is liear, ad it is bouded as hece y = (y,y) y (x,y) sup x x T y (x) T = sup = y. x x yx sup = y, x x I other words, to every y H correspods a bouded liear fuctioal T y..3.2 Extesio of bouded liear fuctioals Lemma.3 Give a bouded liear fuctioal T defied o a dese liear subspace D of a Hilbert space (H,(, )), it has a uique extesio T over H. Moreover, T = T. Proof. We start by defiig T. For x H, take a sequece (x ) D that coverges to x. Cosider the sequece Tx. Sice T is liear ad bouded, Tx Tx m T (x x m ) T x x m, which implies that (Tx ) is a scalar-valued Cauchy sequece. The limit does ot deped o the chose sequece: if (y ) D coverges to x as well, the Thus, we ca defie uambiguously lim Tx Ty lim T x y =. Tx= lim Tx.
.3 Liear fuctioals 4 For x D we ca take the costat sequece x = x, hece Tx= lim Tx = Tx, which shows that T is ideed a extesio of T : T D = T. Next, we show that T is liear. Let x,y D. Let x,y D coverge to x,y, respectively, the: T (ax +by) = lim T (ax +by ) = a lim Tx +b lim Ty = a Tx+b Ty. It remais to calculate the orm of T : Tx T = sup x x = sup x lim Tx x T lim x sup = T, x x ad sice T is a extesio of T : T = T. The followig theorem is a istace of the Hah-Baach theorem, which we will meet whe we study Baach spaces: Theorem.3 Extesio theorem. Give a bouded liear fuctioal T defied o a liear subspace D of a Hilbert space (H,(, )), it ca be exteded ito a liear fuctioal over all H, without chagig its orm. That is, there exists a liear fuctioal T o H, such that T D = T ad T = T. Proof. By the previous lemma we may assume without loss of geerality that D is a closed liear subspace of H. We defie T = T P D. Sice T is a compositio of two liear operators, it is liear. Also, T D = T. Fially, T = T P D T P D = T. Sice T is a extesio of T it follows that T = T. TA material.6 Hamel basis. Defiitio.4 Let V be a vector space. A set of vectors {v a a A} V is called a Hamel basis (or algebraic basis) if every v V has a uique represetatio as a liear combiatio of a fiite umber of vectors from {v a a A}.
42 Hilbert spaces Propositio.32 Every vector space V has a Hamel basis. Exercise.2 Prove it. Hit: use the axiom of choice. Propositio.33 Let (X, ) be a ifiite-dimesioal ormed space. The, there exists a ubouded liear fuctioal o X. Proof. Let {x a a A} be a Hamel basis. Sice X is ifiite-dimesioal there is a sequece (x a ) such that {x a N} is liearly idepedet. For x = a A t a x a defie F(x) = x a t a. = It is easy to see that this is a liear fuctioal. However, it is ot cotiuous. Defie y = x a x a, The, y by F(y ) = for all..3.3 The Riesz represetatio theorem The ext (very importat) theorem asserts that all bouded liear fuctioals o a Hilbert space ca be represeted as a ier-product with a fixed elemet of H : Theorem.34 Riesz represetatio theorem, 97. Let H be a Hilbert space ad T a bouded liear fuctioal o H. The, ad moreover T = y T. (!y T H ) T = (,y T ), Commet.9 The represetatio theorem was proved by Frigyes Riesz (88 956), a Hugaria mathematicia, ad brother of the mathematicia Marcel Riesz (886 969). Proof. We start by provig the uiqueess of the represetatio. If y ad z satisfy T = (,y) = (,z), the y z 2 = (y z,y z) = (y z,y) (y z,z) = T (y z) T(y z) =,
.3 Liear fuctioals 43 which implies that y = z. Next, we show the existece of y T. Sice T is liear it follows that kert is a liear subspace of H. Sice T is moreover cotiuous it follows that kert is closed, as (x ) kert with limit x H, implies that Tx= lim Tx =, i.e., x kert. If kert = H, the x H Tx= = (x,), ad the theorem is proved with y T =. If kert H, the we will show that dim(kert ) =. Let y,y 2 (kert ), ad set y = T (y 2 )y T(y )y 2 (kert ). By the liearity of T, T (y) =, i.e., y kert, but sice kert (kert ) = {}, it follows that T (y 2 )y = T (y )y 2, i.e., every two vectors (kert ) are co-liear. Take y (kert ) with y =. The, (kert ) = Spa{y }. The, set y T = T (y )y. By the projectio theorem, for every x H, x = (x,y )y +[x (x,y )y ], (kert ) kert ad applyig T, T (x) = (x,y )T (y ) = (x,y T ). Fially, we have already see that T = y T. The space dual to a Hilbert space Cosider the set of all bouded liear fuctioals o a Hilbert space. These form a vector space by the poitwise operatios, (at +bs)(x) = a T (x)+b S(x).
44 Hilbert spaces We deote this vector space by H ; it is called the space dual (*-!&$ "(9/) to H. The Riesz represetatio theorem states that there is a bijectio H H, T y T. H is made ito a Hilbert space by defiig the ier-product ad the correspodig orm over H is (T,S) = (y T,y S ), T = y T coicides with the previously-defied orm" (we ever showed it was ideed a orm) 4. The followig theorem (the Rado-Nikodym theorem restricted to fiite measure spaces) is a applicatio of the Riesz represetatio theorem: Theorem.35 Rado-Nikodym. Let (W,B, µ) be a fiite measure space. If is a fiite measure o (W,B) that is absolutely cotiuous with respect to µ (i.e., every zero set of µ is also a zero set of ), the there exists a o-egative fuctio f L (W), such that B B (B) = B fdµ. (The fuctio f is called the desity of with respect to µ.) Proof. Let l = µ +. Every zero set of µ is also a zero set of l. Defie the fuctioal F L 2 (l) C: F is a liear fuctioal; it is bouded as F(g) = W gd. 2 2 F(g) gd g 2 d W W d W g L 2 (l) ((W)) 2. (This is where the fiiteess of the measure is crucial; otherwise a L 2 fuctios is ot ecessarily i L.) By the Riesz represetatio theorem there is a uique fuctio h L 2 (l), such that W gd = W hgdl. (.2) We ow show that h satisfies the followig properties: 4 Our defiitio of a geuie orm over H is somewhat awkward. Whe we get to Baach spaces it will be made clear that we do ot eed ier-products ad represetatio theorems i order to show that the operator orm is ideed a orm.
.3 Liear fuctioals 45. h l-a.e: Because the measures are fiite, idicator fuctios are itegrable. The, settig g = I B, (B) = B hdl, which implies that h l-a.e. 2. h < l-a.e: Settig, B = {x W h(x) }, we get (B) = B hdl l(b) = (B)+ µ(b), which implies that µ(b) = (B) = l(b) =, i.e., h < l-a.e. Sice h <, we may represet h as where f is l-a.e. o-egative. Back to Eq. (.2), h = f + f, gd = f W W + f gd + f W + f gdµ, hece W + f gd = W f + f gdµ. Let ow k W R be o-egative ad bouded. Defie ad g = k(+ f )I Bm. The, B m = {x W k(x)(+ f (x)) m}, Bm kd = Bm kfdµ, Sice k is o-egative we ca let m ad get for all measurable ad bouded k: 5 W kd = W kfdµ. Settig k we get that f is i L (µ). Settig k = I B we get B d = B fdµ, which completes the proof. 5 Here we apply Lebesgue s Mootoe Covergece Theorem, whereby the sequece of itegrads is mootoe ad has a poitwise limit.
46 Hilbert spaces.3.4 Biliear ad quadratic forms Defiitio.5 Let (X, ) be a ormed space. A fuctio B X X C is called a biliear form (;*9!**- *" ;*";) if it is liear i its first etry ad ati-liear i its secod etry, amely, B(x,ay+bz) = ab(x,y)+bb(x,z). It is said to be bouded if there exists a costat C such that x,y X B(x,y) Cxy. The smallest such costat C is called the orm of B, i.e., B(x,y) B = sup x,y xy or B = sup B(x,y). x=y= Defiitio.6 Let (X, ) be a ormed space. A mappig Q X C is called a quadratic form (;*3&"*9 ;*";) if Q(x) = B(x,x), where B is a biliear form. It is said to be bouded if there exists a costat C such that x X Q(x) Cx 2. The smallest such C is called the orm of Q, i.e., Q = sup x Q(x), or Q = sup Q(x). x2 x= Propositio.36 Polarizatio idetity. Let (X, ) be a ormed space. For every biliear form B(x,y) with Q(x) = B(x,x), B(x,y) = {Q(x +y) Q(x y)+ı[q(x +ıy) Q(x ıy)]}. 4 Proof. Just follow the same steps as for the relatio betwee the ier-product ad the orm (which is a particular case). By the properties of B, Q(x ±y) = B(x ±y,x ±y) = Q(x)+Q(y)±B(x,y)±B(y,x). Hece Lettig y ıy, Q(x +y) Q(x y) = 2(B(x,y)+B(y,x)). Q(x +ıy) Q(x ıy) = 2ı( B(x,y)+B(y,x)).
.3 Liear fuctioals 47 Multiplyig the secod equatio by ı ad addig to the first we recover the desired result. Propositio.37 Let (X, ) be a complex ormed space. Let Q(x) = B(x,x) be a quadratic form over X. The Q is bouded if ad oly if B is bouded, i which case Q B 2Q. If, moreover, B(x,y) = B(y,x) for all x,y X, the Q = B. Proof. First, suppose that B is bouded. The, i.e., Q is bouded ad Q B. Q(x) = B(x,x) Bx 2, Secod, suppose that Q is bouded. Takig the polarizatio idetity, B(x,y) = {Q(x +y) Q(x y)+ı[q(x +ıy) Q(x ıy)]}, 4 ad usig the defiitio of Q: B(x,y) 4 Qx +y2 +x y 2 +x +ıy 2 +x ıy 2. Usig (twice) the parallelogram law: B(x,y) Qx 2 +y 2, hece B = sup x=y= B(x,y) 2Q. Remais the last part of the theorem. Usig agai the polarizatio idetity, ad otig that Q(y+ıx) = Q(ı(x ıy)) = Q(x ıy) we get Q(y ıx) = Q( ı(x +ıy)) = Q(x +ıy) B(x,y)+B(y,x) = {Q(x +y) Q(x y)+ı[q(x +ıy) Q(x ıy)]} 4 + {Q(x +y) Q(y x)+ı[q(y+ıx) Q(y ıx)]} 4 = {Q(x +y) Q(x y)}. 2
48 Hilbert spaces Thus, usig oce agai the parallelogram law, B(x,y)+B(y,x) 2 Qx +y2 +x y 2 = Qx 2 +y 2, (.3) If B(x,y) = B(y,x), the there exists a phase a = a(x,y) such that B(y,x) = e ıa(x,y) B(x,y). Note that that is B(y,e ıb x) = e ıa(x,y) ıb B(x,y) = e ıa(x,y) 2ıb B(e ıb x,y) B(e ıb y,x) = e ıa(x,y)+ıb B(x,y) = e ıa(x,y)+2ıb B(x,e ıb y), a(e ıb x,y) = a(x,y) 2b a(x,e ıb y) = a(x,y)+2b. For all x = y =, +e ıa(x,y) B(x,y) 2Q. Ad lettig x e ıb x, we get for all b: +e ı(a(x,y) 2b) B(x,y) 2Q. I particular, for b = 2 a(x,y): B(x,y) Q, i.e., B Q. Example. Cosider the real ormed space X = R 2 edowed with the Euclidea ier-product. Let Clearly B >, however B(x,y) = x y 2 x 2 y. Q(x) = B(x,x) =, hece Q =, i cotradictio to B 2Q. What goig o? The above proof was based o the assumptio that F = C. The propositio does ot hold whe F = R. Example.2 Let (H,(, )) be a ier-product space, ad let T be a bouded liear operator o H. Set B(x,y) = (Tx,y). B is a biliear form. By the Cauchy-Schwarz iequality, B(x,Tx) T = sup Tx = sup sup x= x= Tx x= It follows that B = sup x= y= sup B(x,y) sup sup (Tx,y) T. y= sup B(x,y) = T. x= y=
.3 Liear fuctioals 49 The followig theorem states that, i fact, every bouded biliear form o a Hilbert space ca be represeted by a bouded liear operator (this theorem is very similar to the Riesz represetatio theorem). Please ote that like for the Riesz represetatio theorem, completeess is essetial. Theorem.38 To every bouded biliear form B over a Hilbert space (H,(, )) correspods a uique bouded liear operator o H, such that x,y H B(x,y) = (Tx,y). Furthermore, B = T. Commet. The Riesz represetatio theorem asserts that This theorem asserts that H H. H H H H. Proof. We start by costructig T. Fix x H ad defie the fuctioal: F x is liear ad bouded, as F x = B(x, ). F x (y) Bxy, i.e., F x Bx. It follows from the Riesz represetatio theorem that there exists a uique z x H, such that F x = (,z x ) = B(x, ). Deote the mappig x z x by T, thus for every x H : (,Tx) = B(x, ) i.e., for every x,y H : B(x,y) = (Tx,y). Next, we show that T is liear. By defiitio of T ad by the biliearity of both B ad the ier-product: (T (a x +a 2 x 2 ),y) = B(a x +a 2 x 2,y) = a B(x,y)+a 2 B(x 2,y) = a (Tx,y)+a 2 (Tx 2,y) = (a Tx +a 2 Tx 2,y).
5 Hilbert spaces Sice this holds for all y H it follows that T (a x +a 2 x 2 ) = a Tx +a 2 Tx 2. T is bouded, as we have already proved that T = B. It remais to prove that T is uique. Suppose T ad S are both bouded liear operators satisfyig ( x,y H ) B(x,y) = (Tx,y) = (Sx,y). The, hece ( x,y H ) ((T S)x,y) =, ( x H ) (T S)x 2 =, ad hece T = S..3.5 The Lax-Milgram theorem Defiitio.7 Let (X, ) be a ormed space. A biliear form B o X is called coercive (39-/ %/&2() if there exists a costat d > such that x H B(x,x) dx 2. The followig theorem is a cetral pillar i the theory of partial differetial equatios: Theorem.39 Lax-Milgram, 954. Let B be a bouded ad coercive biliear form o a Hilbert space H. The, there exists a uique bouded liear operator S o H such that x,y H (x,y) = B(Sx,y). Furthermore, S exists, it is bouded, ad S d where d is the coercivity parameter. ad S = B, Commet. This theorem is amed after Peter Lax (926 ) ad Arthur Milgram (92 96). Proof. By Theorem.38 there is a uique bouded liear operator T such that x,y H B(x,y) = (Tx,y),
.3 Liear fuctioals 5 ad T = B. If we succeed to show that T is ivertible with bouded iverse S = T ad S d the we are doe, as ad we kow that T = B. (x,y) = (TSx,y) = B(Sx,y), Settig x = y ad usig the coercivity of B ad the Cauchy-Schwarz iequality, we get dx 2 B(x,x) = (Tx,x) Txx, i.e., x H Tx dx. (.4) It follows that kert = {}, hece T is ijective. We ext show that Image(T ) is a closed liear subspace. If Image(T ) x = Tu x H, the u u m d Tu Tu m = d x x m, which implies that (u ) is a Cauchy sequece with limit which we deote by u. By the cotiuity of T, we have x = Tu. Next, we show that T is surjective. Let z (ImageT ), the d z 2 B(z,z) = (Tz,z) =, which implies that z =, i.e., (Image(T )) = {}, ad sice Image(T ) is closed it is equal to H. We coclude that T is a bijectio. Hece, T exists. It is liear because the iverse of a liear operator is always liear. It is bouded because by Eq. (.4) T x d T (T x) d x, from which we coclude that T d. This completes the proof. Example.3 We will study a typical (relatively simple) applicatio of the Lax- Milgram theorem. Let k C[,] satisfy x [,] < c k(x) c 2, ad cosider the boudary value problem d dx du k = f u() = u() =, (.5) dx
52 Hilbert spaces for some f C[,]. Does a solutio exist? 6 We will slightly modify the problem. Let v C [,] satisfy v() = v() = (we deote this space by C [,]). Multiplyig the equatio by v, itegratig over [,] ad itegratig by parts, we get v C [,] ku v dx = fvdx. (.6) Equatio (.6) is called the weak formulatio of (.5). Does a solutio exist to the weak formulatio? First, edow the space C [,] with the ier-product (u,v) = uvdx + u v dx. I fact, this space is ot complete. Its completio is kow as the Sobolev space W,2 [,] = H [,]. So. Eq..6 really takes the followig form: fid u H [,] such that v H [,] where D is the weak derivative. Defie the biliear form kdudvdx = B(u,v) = ku v dx. B is bouded, for by the Cauchy-Schwarz iequality: B(u,v) c 2 Coercivity is more tricky. First, u v dx c 2 u 2 dx c 2 uv. 2 fvdx, v 2 dx B(u,u) = ku 2 dx c u 2 dx. This is ot good eough because the orm also has a part that depeds o the itegral of u 2. To solve this difficulty we will derive a very importat iequality the Poicaré iequality. Sice u() =, u(x) = u (t)dt, x 6 Recall the existece ad uiqueess theorem applies to iitial value problems but ot to boudary value problems. 2
.3 Liear fuctioals 53 hece u 2 (x) = u (t)dt Itegratig over [,], Thus, x 2 x (u (t)) 2 dt u2 dx (u ) 2 dx. x dt (u ) 2 dt. (u ) 2 dx 2 u2 dx+ 2 (u ) 2 dx = 2 u2, which implies that B is coercive with Next, we cosider the mappig B(u,u) c 2 u2. v fvdx, which is a bouded liear fuctioal. By the Riesz represetatio theorem, there exists a vector g H [,] such that (g,v) = fvdx. By the Lax-Milgram theorem, there exists a bouded liear operator S o H [,], such that v H [,] B(Sg,v) = (g,v), i.e., v H [,] i.e., Sg satisfies the weak equatio. 7 k(sg) v dx = fvdx, Theorem.4 Töplitz-Haussdorff. Let Q be a quadratic form o a ierproduct space. The the umerical rage (*9/&% (&&)%), W(Q) = {Q(x) x = } = Q(x) x, x2 is a covex subset of the complex plae. 7 Note that this is a existece proof; we have t solved the equatio.
54 Hilbert spaces Commet.2 I the fiite-dimesioal case, a biliear form is represeted by a matrix Q, ad the correspodig umerical rage is W(Q) = x Qx x x x. It ca be show, for example, that all the eigevalues of Q are withi its umerical rage. Proof. We eed to show that if a ad b belog to the umerical rage of Q, the so does ay ta +( t)b for eve t. I other words, we eed to show that: ( x,y ) Q(x) = ax 2 ad Q(y) = by 2 implies ( t )( z ) Q(z) = (ta +( t)b)z 2. We ca formulate it differetly: we eed to show that ( x,y ) Q(x) ax 2 b a x 2 = ad Q(y) ay 2 b a y 2 =, implies ( t )( z ) Q(z) az 2 b a z 2 = t. Thus it suffices to show that if ad are i the umerical rage of a quadratic form Q, so is ay umber o the uit segmet. Suppose the that Q(x) = ad Q(y) =, x = y =. Let B be the biliear form that defies Q, ad cosider B(x,y)+B(y,x). We may assume that it is real-valued for we ca always replace y by e ıs y. Now, ad Q(( t)x +ty) = ( t) 2 Q(x) +t 2 Q(y) +t( t)(b(x,y)+b(y,x)) R, F(t) = Q(( t)x +ty) ( t)x +ty 2 = t2 +t( t)(b(x,y)+b(y,x)) ( t)x +ty 2. This is a cotiuous fuctio equal to zero at zero ad to oe at oe, hece it assumes all itermediate values.
.4 Orthoormal systems 55.4 Orthoormal systems Defiitio.8 Let A be some idex set (ot ecessarily coutable), ad let {u a a A} be a set of vectors i a ier-product space (H,(, )). The set {u a } is called a orthoormal system (;*-/9&&;9&! ;,93/) if a,b A (u a,u b ) = d ab. Defiitio.9 Let {u a a A} be a orthoormal system ad let x H. The set of scalars { ˆx(a) = (x,u a )a A) are called the Fourier compoets (%**9&5 *"*,9) of x with respect to the orthoormal system. Theorem.4 Gram-Schmidt orthoormalizatio. Let (x ) be either a fiite or a coutable sequece of liearly idepedet vectors i a ier-product space H. The it is possible to costruct a orthoormal sequece (y ) that has the same cardiality as the sequece (x ), such that N Spa{y k k } = Spa{x k k }. Proof. You leared it i liear algebra for spaces of fiite dimesio. The same recursive costructio holds for a coutable sequece. Propositio.42 Let (u,...,u ) be vectors i a ier-product space (H,(, )). The, where x H i= M = max i (x,u i ) 2 Mx 2, (u i,u j ). j= Proof. For every set of scalars (c,...,c ): 2 x c i u i = x 2 2Re[c i (x,u i )]+ i= i= i, j= c i c j (u i,u j ).
56 Hilbert spaces Usig the iequality 2ab a 2 +b 2 : x 2 2Rec i (x,u i )+ i= 2 = x 2 2Rec i (x,u i )+ i= i, j= i= x 2 2Rec i (x,u i )+M c i 2. i= Choose c i = (x,u i )M, the x 2 2 M which yields the desired result. i= c i 2 +c j 2 (u i,u j ) c i 2 (u i,u j ) j= i= (x,u i ) 2 + M i= (x,u i ) 2, Corollary.43 Bessel iequality. Let {u a } be a orthoormal system i a ier-product space (H,(, )). For every coutable subset {u ak } k=, x H k= ˆx(a k ) 2 = (x,u ak ) 2 x 2. k= Proof. Immediate from the previous propositio. Corollary.44 Let {u a a A} be a orthoormal system i a ier-product space (H,(, )). For every x H there is at most a coutable set of o-vaishig Fourier compoets. Proof. Fix x H, ad cosider the sets From Bessel s iequality, B k = {a A ˆx(a) 2 k}. k B k a B k ˆx(a) 2 x 2, which implies that each set B k is fiite. Hece is at most coutable. {a A (ˆx(a) > } = B k k=
.4 Orthoormal systems 57 Propositio.45 Riesz-Fischer. Let (u ) be a orthoormal sequece i a Hilbert space (H,(, )). Let (c ) be a sequece of scalars. The, coverges as if ad oly if s = c k u k k= k= c k 2 <. Proof. Look at the differece betwee s ad s m : s s m 2 = k=m+ c k u k 2 = k=m+ c k 2, where we used the orthoormality of the (u k ). Thus the series of c k 2 is a Cauchy sequece if ad oly if (s ) is a Cauchy sequece. Note that the completeess of H is crucial. Fourier coefficiets as optimizers Let (H,(, )) be a Hilbert space. Let (u,...,u ) be a fiite sequece of orthoormal vectors ad let M = Spa{u,...,u }, which is a closed subspace of H. For every x H : P M x = (x,u k )u k. k= Why? Because x P M x M as for every u j : (x P M x,u j ) = (x,u j ) (x,u j ) =. By the defiitio of the projectio as a distace miimizer, it follows that the Fourier coefficiets ˆx(k) satisfy: x ˆx(k)u j x l k u j, k= for ay set of scalars (l,...,l ). k=
58 Hilbert spaces Defiitio.2 A orthoormal system {u a a A} is said to be complete if it is ot cotaied (i the strict sese) i ay other orthoormal system. That is, it is complete if the oly vector orthogoal to all {u a } is zero: (Spa{u a a A}) = {}. A complete orthoormal system is also called a orthoormal basis. Propositio.46 Every separable Hilbert space (H,(, )) cotais a coutable complete orthoormal system. Proof. Recall that H is separable if it cotais a coutable dese subset. Now, Let (z ) be a dese coutable set. I particular: Spa{z N} = H. We ca costruct iductively a subset (x ) of idepedet vectors such that that is, N < N such that Spa{x k k N} = Spa{z k k }, Spa{x N} = H. By applyig Gram-Schmidt orthoormalizatio we obtai a orthoormal system (u ) such that Spa{u N} = H. We will show that this orthoormal system is complete. Suppose that v were orthogoal to all (u ). Sice every x H is a limit x = lim c,k u k, k= it follows by the cotiuity of the ier-product that (x,v) = lim c,k (u k,v) =, k= i.e., v is orthogoal to all vectors i H, hece it is zero, from which follows that the (u ) form a complete orthoormal system.
.4 Orthoormal systems 59 Theorem.47 Let (H,(, )) be a (o-trivial) Hilbert space. The, it cotais a complete orthoormal system. Moreover, every orthoormal system i H is cotaied i a complete orthoormal system. Proof. The proof replies o the axiom of choice. Let P deotes the set of all orthoormal systems i H. P is ot empty because every ormalized vector i H costitutes a orthoormal system. Let S P ad cosider P S = {T P T S} be the collectio of all the orthoormal systems that cotai S. P S is a partially ordered set with respect to set iclusio. Let P S P S be fully ordered (a chai). The, T = T T P S is a elemet of P S which is a upper boud to all the elemets i P S. It follows from Zor s lemma that P S cotais at least oe maximal elemet 8, which we deote by S. S is a orthoormal system that cotais S. Sice it is maximal, it is by defiitio complete. The importace of complete orthoormal systems We have see that if {u a a A} is a orthoormal set, the for every x H there is at most a coutable umber of Fourier compoets { ˆx(a) a A} that are ot zero. Let x H be give, ad let (a ) be the idexes for which ˆx does ot vaish. It follows from Bessel s iequality that k= ˆx(a k ) 2 x. It follows the from the Riesz-Fischer Theorem that ˆx(a k )u ak k= exists. As for all idexes ot i (a ) the Fourier coefficiets vaish, there is o harm i writig for all x H : ˆx(a) 2 x. a A ad ˆx(a)u a exists. a A 8 Zor s lemma: suppose a partially ordered set P has the property that every totally ordered subset has a upper boud i P. The the set P cotais at least oe maximal elemet.
6 Hilbert spaces Theorem.48 Characterizatio of completeess. Let {u a a A} be a orthoormal system i H. All the followig coditios are equivalet: {u a a A} is complete. For all x H : a A ˆx(a)u a = x. Geeralized Parseval idetity. For all x,y H : (x,y) = a A ˆx(a)ŷ(a). Parseval idetity. For all x H : x 2 = a A ˆx(a) 2. Proof. Suppose that holds, i.e., the orthoormal system, is complete. Give x let (a ) be a sequece of idexes that cotais all idexes for which the Fourier coefficiets of x do ot vaish. For every idex a, I fact, for all a A x ˆx(a k )u ak,u a =. k= x ˆx(a k )u ak,u a =. k= If follows that x k= ˆx(a k)u ak is orthogoal all vectors {u a } but sice we assumed that the orthoormal system is complete, it follows that it is zero, i.e., x = ˆx(a k )u ak, k= ad oce agai we may exted the sum over all a A. Suppose that holds: x H ˆx(a)u a = x. a A Give x,y H let (a ) be a sequece of idexes that cotais all the idexes for which at least oe of the Fourier compoets of either x ad y does ot vaish. By the cotiuity of the ier-product: (x,y) = k= ˆx(a k )u ak, ŷ(a k )u ak = ˆx(a k )ŷ(a k ). Suppose that holds. Settig x = y we obtai the Parseval idetity. Suppose that holds. Let x H be orthogoal to all the {u a }, the k= k= a A ˆx(a) = (x,u a ) =. It follows from the Parseval idetity that x =, i.e., the orthoormal system is complete.
.4 Orthoormal systems 6 Example.4 Cosider the Hilbert space `2 ad the sequece of vectors u = (,,...,,,,...). Clearly, the (u ) form a orthoormal set. Let x `2. x u implies that the -th etry of x is zero, hece if x is orthogoal to all (u ) the it is zero. This implies that the orthoormal system (u ) is complete. Theorem.49 All the complete orthoormal systems i a Hilbert space H have the same cardiality. Thus we ca uambiguously defie the dimesio of a Hilbert space as the cardiality of ay complete orthoormal system. Proof. Let {u a a A} ad {v b b B} be two complete orthoormal system. Suppose first that A is a fiite set, A =. The vectors (u,...,u P are a basis i H, i.e., dimh =. Sice the vectors {v b } are orthoormal, they are liearly idepedet, ad hece B A. By symmetry, A = B. Suppose ow that A is a ifiite set. To every a A we attach a set F a = {b B (u a,v b ) }, which we kow to be either fiite or coutable. By the completeess of the system {u a }, if b B, the there exists a a A such that (u a,v b ), i.e., there exits a a such that b F a, from which we coclude that B F a, a A ad hece, B ℵ A = A, ad by symmetry, A = B. Theorem.5 Two Hilbert space are isomorphic if ad oly if they have the same dimesio.
62 Hilbert spaces Proof. Let (H,(, ) H ) ad (G,(, ) G ) be two Hilbert spaces of equal dimesio ad let {u a a A} H ad {v b b B} G be complete orthoormal systems. Sice they have the same cardiality, we ca use the same idex set, ad create a oe-to-oe correspodece, T u a v a. We exted this correspodece o all H ad G T c u a c v a. = (Note that both sides are well defied if ad oly if c 2 <.) It is easy to see this this correspodece preserves the liear structure, ad by the geeralized Parseval idetity also the ier product, as for x = = c u a ad y = = d u a = (x,y) H = x(a )y(a ) = c d = (Tx,Ty) G. = Thus H ad G are isomorphic. Coversely, if H ad G are isomorphic, the we ca map ay complete orthoormal system i H ito a complete orthoormal system i G, hece both have the same dimesio. = Corollary.5 All separable Hilbert spaces are isomorphic ad i particular isomorphic to `2. Proof. We saw that all separable Hilbert spaces have dimesio ℵ. Example.5 The Haar fuctios are a sequece of fuctios i L 2 [,] defied as follows: f (t) = f (t) = [,2) [2,] f 2 (t) = 2( [,4) [4,2) ) f 3 (t) = 2( [2,34) [34,] ), ad i geeral, f 2 +k(t) = 2 2 [2 k,2 (k+2) [2 (k+2),2 (k+), N, k =,2. The Haar fuctios form a orthoormal system: because take f j o the iterval o which it is o-zero ay f i with i < j is costat. Also, the spa of all (f ) is the
.5 Weak covergece 63 same as the spa of all step fuctios with dyadic itervals. It is kow that this spa is dese i L 2 [,], hece the Haar fuctios form a orthoormal basis i L 2 [,]. If therefore follows that for every f L 2 [,]: f = ( f,f )f. = The limit is i L 2 [,]. The questio is whether the sum also coverges poitwise (a.e.). Such questios are usually quite hard. For the specific choice of the Haar basis it is relatively easy, due to the good" orderig of those fuctios. The first observatio is that Spa{f i i 2 } = {step fuctios over dyadic itervals of legth 2 } P, i.e., fuctios of the form where 2 c k y,k, k= y,k = 2 2 [2 k,2 (k+)). Thus, the liear operator S 2 S f = k= ( f,f k )f k, returs the orthogoal projectio of f over the space of step fuctios P, ad by the uiqueess of this projectio, Note that, 2 S f = k= 2 S f = 2 k= 2 (k+) 2 k ( f,y,k )y,k. f (t)dt [2 k,2 (k+)). It follows that (S f )(x) is equal to the average of f i a iterval of size 2 aroud x. It is kow that as this average coverges to f (x) a.e..5 Weak covergece
64 Hilbert spaces Defiitio.2 Let (x ) be a sequece of vectors i a ier-product space (H,(, )). We say that this sequece weakly coverges (:-( ;2,;/) to a vector x H if y H lim (x,y) = (x,y). x is called the weak limit (:-( -&"#) of (x ), ad we write x x. Defiitio.22 A sequece (x ) of vectors i a ier-product space (H,(, )) is said to be weakly Cauchy if y H (x,y) is a Cauchy sequece. Commets.2 Uiqueess of weak limit: If a sequece has a weak limit the the weak limit is uique, for if x ad z are both weak limits of (x ) the y H lim (x,y) = (x,y) = (z,y), from which follows that x = z. Sice there is a oe-to-oe correspodece betwee (,y) ad bouded liear fuctioals we ca reformulate the defiitio of weak covergece as follows: a sequece (x ) of vectors i a Hilbert space is said to weakly coverge to x if f H lim f (x ) = f (x). Propositio.52 Strog covergece implies weak covergece. Let (H,(, )) be a ier-product space. If x x the x x. Proof. Let x x. For all f H, lim f (x ) f (x) lim f x x =, i.e., x x. Propositio.53 I a fiite-dimesioal Hilbert space strog ad weak covergece coicide.
.5 Weak covergece 65 Proof. It oly remais to prove that weak covergece implies strog covergece. Let {u k k N} be a orthoormal basis for H. Let x we a weakly covergig sequece with limit x. Expadig we get, Now, N x = (x,u k )u k ad x = (x,u k )u k. k= N k= N x x = (x x,u k )u k (x x,u k )u k = (x x,u k ). k= Sice the right had side teds to zero as it follows that x x. N k= N k= What about the geeral case? Does weak covergece imply strog covergece. The followig propositio shows that the aswer is o: Propositio.54 Every ifiite orthoormal sequece i a ier-product space (H,(, )) weakly coverges to zero. Commet.3 Obviously, a ifiite orthoormal sequece does ot (strogly) coverge to zero as the correspodig sequece of orms is costat ad equal to oe, ad the orm is cotiuous with respect to (strog) covergece. Proof. This is a immediate cosequece of the Bessel iequality: if (u ) is a orthoormal sequece, the for every x H from which follows that = (x,u ) 2 x 2, i.e., u. x H lim (u,x) =, Example.6 Riema-Lebesgue lemma. Let H = L 2 [,2p], ad cosider the vectors u (x) = e ıx. 2p It is easy to check that they costitute a orthoormal system, hece f H lim 2p 2p f (x)e ıx dx =.
66 Hilbert spaces Propositio.55 The orm is weakly lower-semicotiuous. Let (H,(, )) be a ier-product space. If x x the x limif x. Proof. By defiitio of weak covergece: x 2 = lim (x,x) = lim (x,x) limif x x. The ext propositio shows that for a weakly-covergig sequece to strogly coverge, it is sufficiet for the orm to coverge as well: Propositio.56 Weak + covergece of orm = strog. Let x x. The x x if ad oly if lim x = x. Proof. Suppose x x ad x x. The, x x 2 = x 2 +x 2 (x,x ) (x,x). x 2 x 2 x 2 The other directio has already bee proved. So far we have t made ay use of the weak Cauchy property. Here is comes: Propositio.57 A weak Cauchy sequece i a Hilbert space is bouded. Commet.4 This is a geeralizatio of the statemet that a strog Cauchy sequece is bouded. Obviously, the curret statemet is stroger (ad harder to prove). Commet.5 Baire s category theorem: a set A is called owhere dese (%-*-$ %7&"8) if its closure has a empty iterior. Equivaletly, if there is o ope set B such that A B is dese i B. (A lie is owhere dese i R 2. The ratioal are ot owhere dese i R). A set A that is a coutable uio of owhere dese sets is called of the first category; otherwise it is called of the secod category. Baire s theorem states that i a complete metric space a set of first category has a empty iterior. Equivaletly, a set that has a o-empty iterior is ot of the first
.5 Weak covergece 67 category. It follows that if X is a complete metric space ad X = A, the ot all the A are owhere dese. That is, there exists a m such that (A m ) is ot empty. If the A happe to be closed, the oe of them must cotai a ope ball. Proof. Recall that a sequece is weakly Cauchy if for every y H, (x,y) is a Cauchy sequece, i.e., coverges (but ot ecessarily to the same (x,y)). For every N defie the set V = {y H k N, (x k,y) }. These sets are icreasig V V 2... They are closed (by the cotiuity of the ier product). Sice for every y H the sequece (x k,y) is bouded, i.e., y H N such that y V, = V = H. By Baire s Category theorem there exists a m N for which V m cotais a ball, say, B(y,r). That is, It follows that for every k N, y B(y,r) k N (x k,y) m. x k x k = x k, x k = 2 r x k, r x k 2 x k = 2 r x k,y + r x k 2 x k (x k,y ) 4m r. Why are weak Cauchy sequeces importat? The followig theorem provides the aswer. Theorem.58 Hilbert spaces are weakly complete. Every weak Cauchy sequece i a Hilbert space weakly coverges. Proof. Let (x ) be a weak Cauchy sequece. For every y H, the sequece of scalars (x,y) coverges, Defie the fuctioal F(y) = lim (y,x ).
68 Hilbert spaces It is a liear fuctioal, ad it is bouded, as F(y) limsupx y. By the Riesz represetatio theorem there exists a x, such that F(y) = (y,x), i.e., which completes the proof. y H (y,x) = lim (y,x ), Theorem.59 The uit ball is weakly sequetially compact. Every bouded sequece i a Hilbert space (H,(, )) has a weakly covergig subsequece. (Equivaletly, the uit ball i a Hilbert space is weakly compact.) Commet.6 Note that it is ot true for strog covergece! Take for example H = `2 ad x = e (the -th uit vector). This is a bouded sequece that does ot have ay (strogly) covergig subsequece. O the other had, we saw that it has a weak limit (zero). Proof. We first prove the theorem for the case where H is separable. Let (x ) be a bouded sequece. Let (y ) be a dese sequece. Cosider the sequece (y,x ). Sice it is bouded there exists a subsequece (x () ) of (x ) such that (y,x () ) coverges. Similarly, there exists a sub-subsequece (x (2) ) such that (y 2,x (2) coverges (ad also (y,x (2) ) coverges). We proceed iductively to costruct the subsequece (x (k) ) for which all the (y`,x (k)) ) for ` k coverges. Cosider the diagoal sequece (x () ), which is a subsequece of (x ). For every k, (x () ) has a tail that is a subsequece of (x (k) ), from which follows that for every k, Next, we show that, y H from which follows that x () theorem weakly coverges. `k lim (y k,x () ) exists. lim (y,x () ) exists, ) is a weak Cauchy sequece, ad by the previous Let y H ad let y m y be a sequece i the dese coutable set. Set, (`m) is a Cauchy sequece, as `m = lim (y m,x () ). `m `k = lim (y m y k,x () ) y m y k limsupx ().
.5 Weak covergece 69 Let ` be its limit, the (y,x () ) ` (y y m,x () )+(y m,x () ) ` y y m x () +(y m,x () ) `, ad it remais to take sequetially limsup ad m. Next, cosider the case where H is ot separable. Deote H = Spa{x N}. H is a closed separable subspace of H. Hece there exists a subsequece (x k ) of (x ) that weakly coverges i H, amely, there exists a x H, such that y H lim (y,x ) = (y,x). Take ay y H. From the projectio theorem, hece y = P H y+p H y, lim (y,x ) = lim (P H y,x ) = (P H y,x) = (P H y+p H y,x) = (y,x), which completes the proof. Weak covergece does ot imply strog covergece, ad does ot eve implies a strogly coverget subsequece. The followig theorem establishes aother relatio betwee weak ad strog covergece. Theorem.6 Baach-Saks. Let (H,(, )) be a Hilbert space. If x x the there exists a subsequece (x k ) such that the sequece of ruig averages strogly coverges to x. S k = k k x j j= Proof. Without loss of geerality we may assume that x =, otherwise cosider the sequece (x x). As (x ) weakly coverges it is bouded; deote M = limsup x. We costruct the subsequece (x k ) as follows. Because x, we ca choose x k such that j < k (x k,x j ) k.
7 Hilbert spaces The, k 2 k j = j=x k k 2 x j 2 +2Re i,x j ) j= i j k(x k 2 km2 +2 2 + 2 3 + 3 + +k 4 k M2 +2, k i.e., the ruig average strogly coverges to zero. Corollary.6 Strogly closed + covex implies weakly closed. Let (H,(, )) be a Hilbert space. Every closed ad covex set C H is also closed with respect to sequetial weak covergece. That is, if (x ) is a subsequece i C with weak limit x H, the x C. Commet.7 Closed" meas closed with respect to strog covergece. A closed set is ot ecessarily closed with respect to weak covergece, uless it is covex. Proof. Suppose that x is a sequece i C with weak limit x H. By the Baach- Sack theorem there exists a subsequece whose ruig averages (strogly) coverge to x. Because C is covex, the ruig averages are also i C, ad because C is closed, x C. Defiitio.23 A real-valued fuctioal f H R is called lowersemicotiuous (39-/ %7(/- 4*79) at a poit x if for every sequece x x, f (x) limif f (x ). Defiitio.24 A real-valued fuctioal defied o a covex set C is called covex if t f (tx+( t)y) tf(x)+( t) f (y). Example.7 For every p the real-valued fuctioal f (x) = x p is covex. Theorem.62 Let (H,(, )) be a Hilbert space. Let C H be a closed, bouded, covex set. Let f C R be bouded from below, covex ad lower-
.5 Weak covergece 7 semicotiuous. The f has a miimum i C. Proof. Let m = if{ f (x) x C }, which is fiite because f is bouded from below. Let (x ) be a sequece i C such that lim f (x ) = m. Sice C is bouded there exists a subsequece x k that weakly coverges to x. By the previous corollary, sice C is closed ad covex, x C. From the Baach-Saks theorem follows that there exists a sub-subsequece (heck! o relabelig) whose ruig average, s k = k k x j j= strogly coverges to x. Sice f is covex, it follows iductively that f (s k ) k Sice f is moreover lower-semicotiuous, m f (x) limif k k f (x j ). j= k f (s k) limif f (x k k j ) = m, j= i.e., x is a miimizer of f..5. The weak topology We have see that bouded liear fuctioal are cotiuous with respect to the strog topology. Weak covergece is defied such that bouded liear fuctioals are cotiuous with respect to weak covergece, amely, if x x the T H Tx Tx (this is the defiitio of weak covergece). What is the topology uderlyig the cocept of weak covergece? It is the coarsest topology for which bouded liear fuctioals are cotiuous. Cosider all the topologies o H with respect to which all bouded liear fuctioals are cotiuous; this set is ot empty as it icludes the orm topology. The itersectio of all these topologies is a topology it is the weak topology.
72 Hilbert spaces The ope sets i the weak topology are geerated by the subbase: or equivaletly, {T (U) T H, U ope i C}, {x (x,y) U, y H, U ope i C}, That is, a set is ope with respect to the weak topology if ad oly if it ca be writte as a uio of sets that are fiite itersectios of sets of the form T (U)..6 Approximatio by polyomials This sectio deals with the approximatio of fuctios by polyomials. There are mai two reasos to cosider this: (i) a tool for provig theorems, (ii) a computatioal tool for approximatig solutios to equatios that caot be solved by aalytical meas. I the followig we deote by P [a,b] the set of polyomials over [a,b] of degree or less, ad P[a,b] = P [a,b]. = The followig well-kow theorem is usually taught i the first year of udergraduate studies: Theorem.63 Weierstraß approximatio theorem. Every cotiuous fuctio o a iterval [a,b] ca be uiformly approximated by polyomials. That is, for every f C[a,b] ad every e > there exists a polyomial p P[a,b] for which sup f (x) p(x) < e. a x b Proof. Not i this course. Let K be a compact hausdorff space (i.e., every two distict poits have disjoit eighborhoods). We deote by C(K) the space of cotiuous scalar-valued fuctios o K, ad defie the orm, f = max x K f (x). The maximum exists because K is compact.
.6 Approximatio by polyomials 73 Defiitio.25 A liear subspace of C(K) that is closed also uder multiplicatio is called a algebra (of cotiuous fuctios). Example.8 P(K) C(K) is a algebra of cotiuous fuctios. Defiitio.26 A collectio A of fuctios o K is called poit-separatig (;&$&8 *" %$*95/) if for every x,y K there exists a fuctio f A, such that f (x) f (y). Theorem.64 Stoe-Weierstraß. Let A be a poit-separatig algebra of cotiuous real-valued fuctios o a compact Hausdorff space K. Suppose furthermore that icludes the fuctio. The A is dese i C(K) with respect to the maximum orm. Commet.8 The Weierstraß approximatio theorem is a particular case. Commet.9 This theorem has othig to do with Hilbert spaces but we will use it i the cotext of Hilbert spaces. Proof. The uiform limit of cotiuous fuctios is cotiuous, hece the closure of A (with respect to the ifiity orm) is also a algebra of cotiuous fuctios 9. Thus, we ca assume that A is closed ad prove that A = C(K). We first show that if f A the f A. Let f A be give. By the Weierstraß approximatio theorem, ( e > )( p P[ f, f ]) max t p(t) < e. f t f Every polyomial of f is also i A (because A is a algebra), ad max f (x) p( f (x)) < e, x K i.e., f is i the closure of A, ad sice A is closed, the f A. Next, we show that if f,g A the so are f g ad f g, as f g = 2 ( f +g) 2 f g ad f g = 2 ( f +g)+ 2 f g. Next, we show that ( F C(K), x K, e > )( f A) ( f (x) = F(x)) ( f F +e). 9 If f,g Ā the there exist sequeces f,g A such that f f ad g g. The f g fg, hece fg Ā.
74 Hilbert spaces Because A separates betwee poits ad A, there exists for every x,y A ad every a,b R a elemet g A such that g(x) = a ad g(y) = b (very easy to show). Thus, give F, x, ad e, the ( y K)( g y A) (g y (x) = F(x)) (g y (y) = F(y)). Sice both F ad g y are cotiuous, there exists a ope eighborhood U y of y such that g y Uy F Uy +e. The collectio {U y } is a ope coverig of K, ad sice K is compact, there exists a fiite ope sub-coverig {U yi } i=. The fuctio satisfies the required properties. f = g y g y2...g y Let F C(K) ad e > be give. We have see that ( x K)( f x A) ( f x (x) = F(x)) ( f x F +e). Relyig agai o cotiuity, there exists a ope eighborhood V x of x, such that f x Vx F Vx e. By the compactess of K, K ca be covered by a fiite umber of {V xi } i=. The fuctio f = f x f x2... f x satisfies, which completes the proof. f F e, This theorem, as is, does ot apply to complex-valued fuctios. For this, we eed the followig modificatio: Theorem.65 Stoe-Weierstraß for complex fuctios. Let A be a poitseparatig algebra of cotiuous complex-valued fuctios o a compact Hausdorff space K, that icludes the fuctio ad is self-cojugate (%/73- %$&/7) i the sese that f A implies f A. The A is dese i C(K) with respect to the maximum orm. Proof. Let A R A be the subset of real-valued fuctios. Notig that Re f = f + f 2 ad Im f = f f 2ı are both i A R, it follows that A R is poit-separatig. Thus, we ca uiformly approximate both Re f ad Im f.
.6 Approximatio by polyomials 75 Corollary.66 Let K R be compact, the every fuctio f C(K) ca be uiformly approximated by a polyomial i variables. We retur ow to Hilbert spaces. Cosider the space L 2 [a,b]. Sice the polyomials P[a,b] are dese i L 2 [a,b] (ay uiform e-approximatio is a (b a)e-l 2 approximatio), it follows that a Gram-Schmidt orthoormalizatio of the sequece of vectors,x,x 2,... yields a complete orthoormal system. Example.9 Cosider the followig sequece of polyomials i [,]: d p (x) = p (x) = 2! dx (x2 ), which are called the Legedre polyomials. For example, p (x) = d 2 dx (x2 ) = x, ad p 2 (x) = d 2 8 dx 2 (x2 ) 2 = d 8 dx 2x(x2 ) = 4 (3x2 ), ad it is clear that p P. We claim that the p are orthogoal. For m <, xm p (x)dx = 2! xm d dx (x2 ) dx. Itegratig by parts times we get zero (do you see why the boudary terms vaish?). Thus, What about their orms? p2 (x)dx = 2 2! p m(x)p (x)dx =. d dx (x2 ) d dx (x2 ) dx. We itegrate times by parts (agai the boudary term vaishes) ad get p2 (x)dx = 2 2! ( ) (the last idetity ca be proved iductively). d2 dx 2 (x2 ) (x 2 ) dx = 2 2! ( ) (2!) (x2 ) dx = 2 +
76 Hilbert spaces.7 Fourier series ad trasform.7. Defiitios Cosider the Hilbert space H = L 2 [,2p] ad the (complex) trigoometric fuctios f (x) = 2p e ıx Z. Propositio.67 The trigoometric fuctios {f } form a complete orthoormal system. Proof. It is easy to see that the {f } are orthoormal as for m, (f m,f ) = 2p 2p e ımx e ıx dx = 2p e ı(m )x ı(m ) Why is this system complete? Cosider the liear space of trigoometric polyomials: A = Spa{f N}. It is a self-adjoit algebra of cotiuous fuctios that icludes the fuctio. There is oly oe problem: it does ot separate the poits ad 2p. Thus, we view A as a algebra of cotiuous fuctios o a circle, S (i.e., we idetify the poits ad 2p). A is poit separatig for fuctios o S, hece it is dese i C(S). O the other had, C(S) is dese i L 2 [,2p], which proves the completeess of the system of the trigoometric fuctios. 2p =. Corollary.68 For every f L 2 [,2p], f (x) = ˆf (k)e ıx, 2p = where the series coverges i L 2 [,2p], ad ˆf (k) = 2p 2p f (x)e ıx dx. Moreover, 2p f (x) 2 dx = = ˆf (k) 2.
.7 Fourier series ad trasform 77 Proof. This follows immediately from the properties of a complete orthoormal system. Commet.2 It is customary to defie the Fourier compoets ˆf (k) divided by a additioal factor of 2p, i which case we get where ad, f (x) = = ˆf (k)e ıx, ˆf (k) = 2p 2p f (x)e ıx dx, 2p 2p f (x) 2 dx = = ˆf (k) 2. The trigoometric series equal to f is called its Fourier series (%**9&5 9&)). Real-valued Fourier series A related complete orthoormal system is the system, six, si2x,..., cosx, cos2x,... 2p p p p p (The proof of its completeess is agai based o the Stoe-Weierstraß theorem.) The, every L 2 [,2p] fuctio has the followig expasio: where ad The Parseval idetity is f (x) = a + a cosx + b six, = a = 2p 2p f (x)dx = a = 2p p f (x)cosx dx, b = 2p p f (x)six dx. 2p 2p f (x) 2 dx = a 2 + a 2 +b 2. 2 Clearly, if f is real-valued the so are the Fourier coefficiets. =.7.2 Two applicatios The first applicatio of the trigoometric fuctios uses oly their completeess, but ot the Fourier series.
78 Hilbert spaces Defiitio.27 A sequece of poits (x ) i [,2p] is said to be equidistributed (%&&: %$*/" ;#-&5/) if for every a,b: Note that {k a x k b} lim = b a 2p. {k a x k b} hece a sequece is equi-distributed if lim = [a,b] (x k ), k= [a,b] (x k ) = 2p 2p [a,b] (x)dx. k= (Empirical average coverges to esemble average".) If follows that the sequece is equi-distributed if ad oly if for every step fuctio f, lim k= f (x k ) = 2p 2p f (x)dx. Propositio.69 Herma Weyl s criterio for equi-distributio. A sequece (x ) i [,2p] is equi-distributed if ad oly if lim e ımx k = 2p 2p e ımx dx = d m,. k= Proof. Suppose that the poits are equi-distributed. For every m ad e > there exist step fuctios f, f 2, such that x [,2p] f (x) cos(mx) f 2 (x), ad f f 2 < e. Now, lim cos(mx k ) lim k= k= f 2 (x k ) = 2p 2p f 2 (x)dx, whereas Similarly, 2p 2p cos(mx)dx 2p 2p f 2 (x)dx. lim cos(mx k ) lim k= k= f (x k ) = 2p 2p f (x)dx,
.7 Fourier series ad trasform 79 ad It follows that 2p 2p cos(mx)dx 2p 2p f (x)dx. lim cos(mx k ) 2p 2p cos(mx)dx 2p 2p ( f 2 (x) f (x))dx e. k= We proceed similarly for the imagiary part of e ımx. Coversely, suppose that The lim lim e ımx k = 2p 2p e ımx dx. k= k= f (x k ) = 2p 2p f (x)dx for every trigoometric polyomial f. Let F be a cotiuous fuctio. The for every e > there exists a trigoometric polyomial f such that F f < e (relyig agai o the Stoe-Weierstraß theorem). The, lim F(x k ) lim f (x k ) e, k= k= 2p 2p f (x)dx ad 2p 2p F(x)dx 2p 2p f (x)dx e, which proves that lim k= F(x k) = F(x)dx for every cotiuous F. It follows that this is true also for step fuctios sice we ca uiformly approximate step fuctios by cotiuous fuctios. 2p 2p Corollary.7 Piers Bohl-Wacław Sierpisky theorem, 9. If a2p is irratioal the the sequece is equi-distributed i [,2p]. (a mod 2p)
8 Hilbert spaces Proof. This proof is due to Weyl (the stadard theorem is for the iterval [,]). For every m : k=e ımx k = k= e ımka = eıma e ıma e ıma Because a2p is irratioal the deomiator is ever zero, ad it follows as oce that lim e ımx k =. k= Commet.2 We showed i fact that if a sequece (x ) is equi-distributed i [,2p], the lim f (x ) = 2p 2p f (x)dx k= for every cotiuous fuctio f. Does it apply also for all f L [,2p]? No. Take for example a equi-distributed sequece whose elemets are ratioal ad take f to be the Dirichlet fuctio. Commet.22 I 93 George Birkhoff proved the celebrated ergodic theorem: Let (W,B,µ) be a measure space, ad let T W W be a measure preservig trasformatio, amely, ( A B) µ(a) = µ(t (A)), where T is to be uderstood as a set fuctio. For f L (W) ad x W, cosider the log-time average, F(x) = lim k= f (T k x). The (strog) ergodic theorem states that F(x) exists for almost every x, ad it T ivariat, amely F T = F. T is said to be ergodic if for every E B for which T (E) = E (i.e., for every T -ivariat set), either µ(e) = or µ(e) = (oly trivial sets are T -ivariat). If T is ergodic (ad the measure is fiite) the F is costat ad equal to F(x) = W fdµ µ(w). Back to our example: for W = [,2p] the trasformatio Tx= x +a mod 2p
.7 Fourier series ad trasform 8 with a2p irratioal ca be show to be ergodic, hece lim for every L [,2p] fuctio. k= The secod applicatio uses Fourier series: f (x ) = 2p 2p f (x)dx Theorem.7 Isoperimetric iequality. Let g be a closed simple plaar curve of legth L ad eclosed area S. The S L2 4p. (For those with a icliatio to geometry: we get a equality for the circle.) Proof. The proof here will be for the special case where the curve is piecewise cotiuously differetiable. WLOG we may assume that the curve is parametrized o [,2p], ad that the parametrizatio is proportioal to arclegth. That is, if g [,2p] is of the form g(t) = (x(t),y(t)), the g differetiable everywhere except at most at a fiite umber of poits, ad We expad the curve i Fourier series: where [x (t)] 2 +[y (t)] 2 = L 2p 2. x(t) = (a cost +b sit) ad y(t) = (c cost +d sit), = a = 2p p x(t)cost dt b = 2p p x(t)sit dt c = 2p p y(t)cost dt d = 2p p y(t)sit dt. Sice we assume that x(t) ad y(t) are cotiuously differetiable, their derivative (which is i particular i L 2 [,2p]) ca also be expaded as a Fourier series. Settig: x (t) = (a cost +b sit) ad y (t) = (g cost +d sit), = = =
82 Hilbert spaces we have a = 2p p x (t)cost dt b = 2p p x (t)sit dt g = 2p p y (t)cost dt d = 2p p y (t)sit dt. Itegratig by parts ad usig the periodicity of x(t), y(t), we fid right away that amely a = b b = a g = d ad d = c, x (t) = ( a sit +b cost) ad y (t) = ( c sit +d cost). = By the Parseval idetity: ad therefore = 2p 2p x (t) 2 dt = 2 a 2 +b 2 2 = 2p 2p y (t) 2 dt = 2 c 2 +d 2 2, L 2 = 4p 2 = 2 a 2 +b 2 +c 2 +d 2 2. = The eclosed area, o the other had, is give by S = ad by the geeralized Parseval idetity: Usig the iequalities 2p x(t)y (t)dt = 2p (x,y ), S 2p = (a d b c ). 2 = a d 2 a2 +d 2 2 2 a2 +d 2 ad b c 2 b2 +c 2 2 2 b2 +c 2, we get which completes the proof. S 2p 4 = 2 a 2 +b 2 +c 2 +d 2 = L2 8p 2, Uder what coditios do we get a equality? The iequality 2 is a equality oly for =, which meas that = is the oly o-zero Fourier compoet. The iequalities 2a d a 2 + d 2 ad 2b c b 2 + c 2 are equalities oly for a = d ad b = c, which is a circle.
.7 Fourier series ad trasform 83.7.3 Poitwise covergece of Fourier series Set H = L 2 [,2p]. We defie a sequece of operators S H H by (S f )(x) = k= ˆf (k)e ıkx. The fuctios S f are trigoometric fuctios. We kow that S f f i H. The questio i uder what coditios they is coverge poitwise. Recall the Weierstraß M-test. Sice ˆf (k)e ıkx = ˆf (k) is follows that if ˆf (k) < k= the the sequece S f coverges uiformly, ad i particular coverges poitwise to f (sice uiform covergece implies covergece i L 2 ad the limit is uique). I this case we say that f has a absolutely coverget Fourier series. Recall the followig: Defiitio.28 A fuctio f is said to be absolutely cotiuous ()-(%" %5*79) if for every e > there exists a d >, such that wheever a fiite sequece of disjoit itervals (x k,y k ) satisfies s k= x k y k < d, the s k= f (x k ) f (y k ) < e. Moreover, Theorem.72 For a fuctio f defied o a iterval [a,b], absolute cotiuity is equivalet to beig differetiable a.e., with a Lebesgue itegrable derivative f satisfyig This leads us to the followig: f (x) = f (a)+ a x f (t)dt.
84 Hilbert spaces Theorem.73 Let f be absolutely cotiuous o [,2p] such that f () = f (2p) ad f L 2 [,2p], the the Fourier series of f coverges absolutely, ad moreover, lim S f f =. Proof. Sice f L 2 [,2p], the usig the properties of f : f () = 2p 2p f (x)e ıx dx = ı 2p 2p f (x)e ıx dx = ı ˆf (), hece usig the Parseval idetity, ˆf () = ˆf ()+ = f () 2 2 ˆf ()+ 2 f () 2 = ˆf ()+C f, which proves that the series coverges uiformly. Thus, we foud a class of fuctios (which i particular icludes C fuctios) for which the covergece of the Fourier series is poitwise (ad eve uiform). Fourier series of L [,2p] fuctios Note that i the defiitio of ˆf () we do ot eve eed f to be square itegrable; it oly eeds to be i L [,2p]. Thus, we cosider as of ow Fourier series of L fuctios (which however do ot ecessarily exist, sice the series of ˆf () 2 is ot ecessarily summable). Note that S f (x) = k= = 2p 2p 2p 2p f (t)e ıkt dte ıkx f (t) k= e ık(t x) dt = 2p 2p f (t)d (x t)dt,
.7 Fourier series ad trasform 85 where D (x) k= e ıkx = e ıx e ı(2+)x e ıx = e ı(+)x e ıx e ıx = e 2 ıx e e 2 ıx e 2 ıx e 2 ıx = si(+ 2 )x si 2 x. ı(+ 2 )x e ı(+ 2 )x The fuctios D (x) are called the Dirichlet kerels (%-,*9*$ **39#); D is symmetric i x, is defied o the etire lie, ad is 2p periodic. Also, settig f we get 2p 2p D (x)dx =. Chagig variables, x t = y, ad extedig f o the etire like periodically, we ca get as well: S f (x) = 2p 2p f (x y)d (y)dy. Cosider the graph of D (x) for = 32: 35 3 25 2 D 32 (x) 5 5 5 3 2 2 3 x This graph is cetered" at x =, ad oscillates farther away, isiuatig that for large, S f (x) is some local average" of f i the viciity of the poit x. Propositio.74 Localizatio priciple. Let f,g be L [,2p] fuctios, such that f = g i some eighborhood of a poit x. The, lim (S f (x) S g(x)) =.
86 Hilbert spaces Commet.23 It is ot implied that the limits of S f (x) ad S g(x) exist. Proof. We have S f (x) S g(x) = 2p 2p f (x y) g(x y) si 2 y si+ y 2 dy. It is give that f (x y) g(x y) vaishes i a eighborhood of y =. Hece, the fuctio f (x y) g(x y) si 2 y is itegrable. The limit follows from the Riema-Lebesgue lemma. Theorem.75 Dii criterio. Let f L [,2p] ad x [,2p]. Suppose there exists a scalar ` such that the lim S f (x) = `. p f (x +t)+ f (x t) `dt <, t 2 Proof. We kow that Sice D is symmetric, we also have ad sice D is ormalized, S f (x) = 2p p p f (x t)d (t)dt. S f (x) = 2p p p f (x +t)d (t)dt, S f (x) ` = p 2p p f (x t)+ f (x t) 2 The Riema-Lebesgue lemma states that for every f L [,2p]: 2p lim f (x)e ıx dx =. The proof is simple. A direct calculatio shows that for every iterval I, lim I eıx dx =, `D (t)dt. hece this is true for every step fuctio, ad by domiated covergece for every positive itegrable fuctios, ad fially for every itegrable fuctio.
.7 Fourier series ad trasform 87 Now, S f (x) ` = p 2p p f (x t)+ f (x t) t ` t 2 si 2 t si+ bouded i L t dt, 2 ad this expressio teds to zero by the Riema-Lebesgue lemma. Commet.24 This criterio holds, for example, if f (x +t)+ f (x t) 2 ` = O(t a ) for some a >, i.e., the fuctios eeds oly to be a little more tha differetiable" at x. The geeral questio The basic questio remais: uder what coditios does the Fourier series of a fuctio (if it exists) coverge poitwise to the fuctio? The questio was asked already by Fourier himself i the begiig of the 9th cetury. Dirichlet proved that the Fourier series of cotiuously differetiable fuctios coverges everywhere. It turs out, however, that cotiuity is ot eough for poitwise covergece everywhere (Paul du Bois-Reymod, showed i 876 that there is a cotiuous fuctio whose Fourier series diverges at oe poit). I 966, Leart Carleso proved that for every f L 2 [,2p], lim S f (x) = f (x) almost everywhere. (This was cojectured i 95 by Nikolai Lusi; the proof is by o meas easy.) Two years later Richard Hut exteded the proof for every L p [,2p] fuctio for p >. O the other had, Adrey Kolmogorov showed back i the 92s that there exist L [,2p] fuctios whose Fourier series owhere coverges. Fejér sums Rather tha lookig at the partial sums S f, we ca cosider their ruig average: s f = S f. + Such sums are called after Lipót Fejér. It turs out that Fejér sums are much better behaved tha the Fourier partial sums. Note that it is trivial that if S f (x) coverges as, the so does s f (x), ad to the same limit. k=
88 Hilbert spaces Cosider the Fejér sums i explicit form: where, s f (x) = + k= 2p 2p = 2p 2p f (x y) + f (x y)d k (y)dy D k (y) dy k= 2p 2p f (x y)k (y)dy, D k (y) = + + k= k= sik + 2 y si 2 y = + si 2 y Im e ı(k+ 2 )y k= = + si 2 y Ime 2 ıy e ıy e ıy 2 ıy 2 y = + si 2 y Ime 2 ıy e si e 2 ıy si 2 y = si 2 2 y + si 2 2 y. The fuctio K (x) is kow as the Fejér kerel. 6 4 2 K 6 (x) 8 6 4 2 3 2 2 3 x The Fejér kerel is also ormalized, as foud by settig f : = s f (x) = 2p 2p K (y)dy. It differs a lot from the Dirichlet kerel i that it is o-egative.
.7 Fourier series ad trasform 89 Like the Dirichlet kerel it is cetered" at zero. That is, for every e >, 2p e lim e 2p e K (x)dx = lim + si 2 2 x e si 2 2 x dx 2p lim + dx si 2 2 d =. Theorem.76 Fejér. For every f C[,2p] such that f () = f (2p): lim s f f =. Moreover, if f L [,2p] is cotiuous at x the lim s f (x) = f (x). Proof. Let f C[,2p]. Sice cotiuous fuctios o compact itervals are uiformly cotiuous, ( e > )( d > ) ( x,y x y < d)( f (x) f (y) < e). Usig the ormalizatio of the Fejér kerel: s f (x) f (x) = 2p p p ( f (x y) f (x))k (y)dy = 2p x d ( f (x y) f (x))k (y)dy+ 2p x<d ( f (x y) f (x))k (y)dy 2 f 2p K (y)dy+ e 2p x d 2p K (y)dy, ad we used the fact that the Fejér kerel is o-egative. For sufficietly large the right had side ca be made smaller tha a costat (idepedet of x) times e, which proves the first part. The proof of the secod part is similar. Cotiuity at x meas local boudedess. By the localizatio priciple we may therefore replace f by a bouded fuctio that coicides with f i some eighborhood of x..7.4 The Fourier trasform The Fourier series represets a fuctio o [,2p] (or equivaletly a 2p-periodic fuctio o R) as a sum of trigoometric fuctios e ıx. We ow wish to exted the treatmet to the Hilbert space L 2 (R). We still wat to expad the fuctios as a sum of trigoometric fuctios e ıx x, but this time we eed to cosider x R (otherwise we ll oly represet 2p-periodic fuctios).
9 Hilbert spaces Ca this be doe? Note that these basis" fuctios are ot i L 2 (R). Moreover, they form a ucoutable set of fuctios, whereas we kow that L 2 (R) is separable, ad hece every orthoormal basis is coutable. Defiitio.29 Let f L (R). The fuctio ˆf R C defied by ˆf (x ) = 2p R f (x)e ıx x dx is called the Fourier trasform (%**9&5.9&529)) of f. Propositio.77 For f L (R), ˆf is cotiuous. Proof. This is a immediate cosequece of domiated covergece. Let x x. The, ˆf (x ) = 2p R f (x)e ıxx g (x) dx. The sequece g (x) coverges poitwise to f (x)e ıx x, ad g (x) = f (x), ad f L (R). It follows that ˆf (x ) ˆf (x ). Propositio.78 The mappig f ˆf is a bouded liear operator L (R) L (R), ad ˆf L (R) f L 2p (R). Proof. Obvious. Lemma.79 Riema-Lebesgue. For every f L (R): lim ˆf (x ) =. x Proof. Let f = [a,b], the ˆf (x ) = b 2p a e ıx x dx = 2p ıx e ıx b e ıx a 2, 2p x
.7 Fourier series ad trasform 9 which satisfies the desired property. Thus, the lemma holds for every step fuctio. Let f L (R) ad let e >. There exists a step fuctio g such that The, by the previous propositio f (x) g(x) L (R) < e. ˆf ĝ L (R) f g L 2p (R) e. 2p Sice g is a step fuctio, there exists a L such that The for every x > L, ( x > L) ĝ(x ) e. ˆf (x ) ĝ(x )+ e + e. 2p 2p Propositio.8 Let f L (R). For R l ad t R set g(x) = f (lx+t). The ĝ(x ) = l eıxtl ˆf x l. If Id f L (R) the ˆf is differetiable ad ˆf (x ) = ı Id f (x ). Proof. For the first part, just follow the defiitio. g is clearly i L (R), ad ĝ(x ) = 2p R g(x)e ıx x dx = 2p R f (lx+t)e ıx x dx = l eıxtl 2p R f (lx+t)e ıx (lx+t)l ldx = l eıxtl ˆf (x l).
92 Hilbert spaces Next, Observig that ˆf (x +h) ˆf (x ) = f (x)e ıx x e ıhx dx h 2p R h `h(x) e ıhx x ad lim = f (x)e h h `h(x) ıx x ( ıx), we apply Lebesgue s domiated covergece theorem to coclude that lim h ˆf (x +h) ˆf (x ) = ıid h f. Commet.25 If the opposite true? Does f L (R) such that ˆf L (R) is differetiable almost everywhere imply that ıx f L (R)? The Fourier trasform i L 2 (R) So far, we defied the Fourier trasform for fuctios i L (R). Recall that ulike for bouded itervals, L 2 (R) L (R), hece the Fourier trasform is as of ow ot defied for fuctios i L 2 (R). Lemma.8 Let f L 2 (R) have compact support (i.e., it vaishes outside a segmet [a,b]), the ˆf (x ) exists ad R f (x) 2 dx = R ˆf (x ) 2 dx. Proof. Sice f [a,b] L 2 [a,b] the it is itegrable, ad Thus, ˆf (x ) = 2p ˆf (x ) 2 = b 2p a Itegratig from L to L: L ˆf (x ) 2 dx = b L 2p a = b 2p a b a b a b a a b f (x)e ıx x dx. f (x) f (y)e ıx (x y) dxdy. L f (x) f (y) L e ıx (x y) dx dxdy 2siL(x y) f (x) f (y) dxdy. x y
.7 Fourier series ad trasform 93 We are defiitely stuck... although physicists would say that the ratio teds as L to the delta-fuctio" d(x y). Let s be serious... first we ll show that we may well assume that the support of f is [,2p]. Defie g(x) = f 2p b a x 2pa b a. The fuctio g has support i [,2p]. By the previous propositio: where l = 2p(b a), hece, Now, ad ĝ(x ) = l eıstuff ˆf (x l), ĝ(x ) 2 = l 2 ˆf (x l) 2. R ĝ(x ) 2 dx = l 2 R ˆf (x l) 2 dx = l R ˆf (x ) 2 dx, R g(x) 2 dx = R f (lx+stuff) 2 dx = l R f (x)2 dx, which meas that we ca assume WLOG that the support of the fuctio is i [,2p]. We are i the realm of fuctios i L 2 [,2p]. Note first that for every t R: e ıtx g() = 2p 2p e ıtx g(x)e ıx dx = 2p ĝ(+t). Applyig the Parseval idetity we have for every t R: 2p 2p g(x) 2 dx = 2p 2p Itegratig over t [,] we get e ıxt g(x) 2 dx = 2p R g(x) 2 dx = R ĝ(x ) 2 dx. ĝ(+t) 2. =
94 Hilbert spaces Theorem.82 Placherel. There exists a uique bouded liear operator F i L 2 (R), such that The operator F is a isometry, amely, ( f L 2 (R) L (R)) F( f ) = ˆf. F( f ) = f. Proof. Let f L 2 (R). By the previous lemma: exists, ad f (x ) f [,] (x ) = f 2p (x)e ıx x dx f (x ) 2 dx = f R (x)2 dx. I fact, the same Lemma implies that for > m: f (x ) f m (x ) 2 dx = f (x) f m (x) 2 dx = f R R (x)2 dx f m (x)2 dx, which meas that f is a Cauchy sequece i L 2 (R), ad hece coverges. Defie: F( f ) = lim f, where the limit is i L 2 (R). By the cotiuity of the orm F( f ) = f. Also, for f L 2 (R) L (R), F( f ) = ˆf. Uiqueess is immediate because F( f ) is specified o a dese subset ad the operator F is bouded. Commet.26 We have thus show that the Fourier trasform ca be aturally" exteded to a operator o L 2 (R). From ow o ˆf will deote the extesio F( f ). m Hermite fuctios Cosider the sequece of elemets i L 2 (R): {x e x2 2 N}. If we perform a Gram-Schmidt orthoormalizatio o this sequece, we get the followig orthoormal system: h (x) = 2! p e x2 2 H (x),
.7 Fourier series ad trasform 95 where h (x) are kow as the Hermite fuctios ad H (x) = ( ) e x2 d dx e x2 are kow as the Hermite polyomials (H is of degree ). Also, the H are alterately odd/eve. Commet.27 The details of the Hermite fuctios are ot importat for our purposes. We oly care that they are obtaied from the fuctios {x e x2 2 by Gram-Schmidt orthoormalizatio. Propositio.83 The Hermite fuctios h form a basis i L 2 (R). Proof. The orthoormality ca be checked by a direct calculatio. Suppose that f L 2 (R) was orthogoal to all Hermite fuctios, amely This amouts to havig ( N) R f (x)h (x)dx =. ( N) R f (x)x e x2 2 dx =. Cosider the fuctio j(z) = 2p R e ıtz e t2 2 f (t)dt. This fuctio is aalytic o the whole complex plae. Note that j () (z) = 2p R (ıt) e ıtz e t2 2 f (t)dt =, from which we coclude that j. I particular, for z = x, e t2 2 f (x ) =. Because the Fourier trasform is a isometry, e t22 f, which completes the proof. Cosider ext the Fourier trasform of the fuctio e x2 2 : F[e x2 2 ](x ) = 2p R e x2 2 e ıx x dx = 2p e x 2 2 R e (x+ıx )2 2 dx.
96 Hilbert spaces Usig Cauchy s theorem for the fuctio e z2 2, for rectagles z = [ R,R] ad z = [ R+ıx,R+ıx ] ad lettig R, F[e x2 2 ](x ) = 2p e x 2 2 R e x2 2 dx = e x 2 2. That is, the fuctio e x2 2 is ivariat uder the Fourier trasform. Next, recallig that F[ f ] = F[ ıx f ](if the latter exists), it follows that ad hece d dx e x 2 2 = F ( ıx) e x2 2 (x ), F[h ](x ) = F a,k x k e x22 (x ) k= = a,k ı k F ( ıx) e x22 (x ) k= = k= a,k ı k d dx e x 2 2. The right had side is a polyomial times e x 22, hece it ca be expaded i the form: F[h ](x ) = b,k h k (x ). k= Moreover, sice F is a isometry, the fuctios F[h ] form a orthoormal system (a isometry is also a isomorphism). It follows that both the h ad the F[h ] are orthoormalizatios of the same set, however, the Gram-Schmidt orthoormalizatio is uique up to prefactors of modulus, amely, F[h ] = l h, where l =. To fid l it is oly eeded to look at the prefactor of x, ad it is easy to see that F[h ] = ( ı) h. For ay f L 2 (R), f = ( f,h )h, = F[x e x2 2 ] = ı d 2 dx e x 2 = ı ( x ) e x 2 2 +...
.7 Fourier series ad trasform 97 hece ad F[ f ] = ( f,h )F[h ] = ( ı) ( f,h )h, = = F[F[ f ]] = ( ) ( f,h )h. = Sice h ( x) = ( ) h (x), it follows that F[F[ f ]](x) = ( f,h )h ( x) = f ( x). = That is, the Fourier trasform is almost its ow iverse. I particular, if ˆf L (R), the f ( x) = ˆf (x )e ıx x dx. 2p R To summarize: Theorem.84 Iverse Fourier trasform. Let f L 2 (R) such that ˆf L (R), the f (x) = ˆf (x )e ıx x dx. 2p R