Measure, Integral, and Conditional Expectation

Transcription

1 Jan van Neerven Measure, Integral, and Conditional Expectation Handout May 2, 2012

2

3 1 Measure and Integral In this lecture we review elements of the Lebesgue theory of integration. 1.1 σ-algebras Let be a set. A σ-algebra in is a collection F of subsets of with the following properties: (1) F ; (2) A F implies A F ; (3) A 1, A 2, F implies A n F. Here, A = \ A is the complement of A. A measurable space is a pair (, F ), where is a set and F is a σ-algebra in. The above three properties express that F is non-empty, closed under taking complements, and closed under taking countable unions. From ( A n = A n ) it follows that F is closed under taking countable intersections. Clearly, F is closed under finite unions and intersections as well. Example 1.1. In every set there is a minimal σ-algebra, {, }, and a maximal σ-algebra, the set P() of all subsets of. Example 1.2. If A is any collection of subsets of, then there is a smallest σ-algebra, denoted σ(a ), containing A. This σ-algebra is obtained as the intersection of all σ-algebras containing A and is called the σ-algebra generated by A.

4 2 1 Measure and Integral Example 1.3. Let ( 1, F 1 ), ( 2, F 2 ),... be a sequence of measurable spaces. The product ( n, F n ) is defined by n = (the cartesian product) and F n = F 1 F 2... is defined as the σ-algebra generated by all sets of the form A 1 A N N+1 N+2... with N = 1, 2,... and A n F n for n = 1,..., N. Finite products N ( n, F n ) are defined similarly; here one takes the σ-algebra in 1 N generated by all sets of the form A 1 A N with A n F n for n = 1,..., N. Example 1.4. The Borel σ-algebra of R d, notation B(R d ), is the σ-algebra generated by all open sets in R d. The sets in this σ-algebra are called the Borel sets of R d. As an exercise, check that (R d, B(R d )) = d (R, B(R)). Hint: Every open set in R d is a countable union of rectangles of the form [a 1, b 1 ) [a d, b d ). Example 1.5. Let f : R be any function. For Borel sets B B(R) we define {f B} := {ω : f(ω) B}. The collection σ(f) = { {f B} : B B(R) } is a σ-algebra in, the σ-algebra generated by f. 1.2 Measurable functions Let ( 1, F 1 ) and ( 2, F 2 ) be measurable spaces. A function f : 1 2 is said to be measurable if {f A} F 1 for all A F 2. Clearly, compositions of measurable functions are measurable. Proposition 1.6. If A 2 is a subset of F 2 with the property that σ(a 2 ) = F 2, then a function f : 1 2 is measurable if and only if {f A} F 1 for all A A 2. Proof. Just notice that {A F 2 : {f A} F 1 } is a sub-σ-algebra of F 2 containing A 2.

5 1.3 Measures 3 In most applications we are concerned with functions f from a measurable space (, F ) to (R, B(R)). Such functions are said to be Borel measurable. In what follows we summarise some measurability properties of Borel measurable functions (the adjective Borel will be omitted when no confusion is likely to arise). Proposition 1.6 implies that a function f : R is Borel measurable if and only if {f > a} F for all a R. From this it follows that linear combinations, products, and quotients (if defined) of measurable functions are measurable. For example, if f and g are measurable, then f + g is measurable since {f + g > a} = q Q{f > q} {g > a q}. This, in turn, implies that fg is measurable, as we have fg = 1 2 [(f + g)2 (f 2 + g 2 )]. If f = sup n 1 f n pointwise and each f n is measurable, then f is measurable since {f > a} = {f n > a}. It follows from inf n 1 f n = sup n 1 ( f n ) that the pointwise infimum of a sequence of measurable functions is measurable as well. From this we get that the pointwise limes superior and limes inferior of measurable functions is measurable, since ( ) ( ) lim sup f n = lim f k = inf f k n 1 sup k n sup k n and lim inf f n = lim sup ( f n ). In particular, the pointwise limit lim f n of a sequence of measurable functions is measurable. In these results, it is implicitly understood that the suprema and infima exist and are finite pointwise. This restriction can be lifted by considering functions f : [, ]. Such a function is said to be measurable if the sets f B} as well as the sets {f = ± } are in F. This extension will also be useful later on. 1.3 Measures Let (, F ) be a measurable space. A measure is a mapping µ : F [0, ] with the following properties: (1) µ( ) = 0; (2) For all disjoint set A 1, A 2,... in F we have µ ( A n) = µ(a n).

6 4 1 Measure and Integral A triple (, F, µ), with µ a measure on a measurable space (, F ), is called measure space. A measure space (, F, P) is called finite if µ is a finite measure, that is, if µ() <. If µ() = 1, then µ is called a probability measure and (, F, µ) is called a probability space. In probability theory, it is customary to use the symbol P for a probability measure. The following properties of measures are easily checked: (a) if A 1 A 2 in F, then µ(a 1 ) µ(a 2 ); (b) if A 1, A 2,... in F, then ( µ A n ) µ(a n ); Hint: Consider the disjoint sets A n+1 \ (A 1 A n ). (c) if A 1 A 2... in F, then ( µ A n ) = lim µ(a n); Hint: Consider the disjoint sets A n+1 \ A n. (d) if A 1 A 2... in F and µ(a 1 ) <, then Hint: Take complements. ( µ A n ) = lim µ(a n). Note that in c) and d) the limits (in [0, ]) exist by monotonicity. Example 1.7. Let (, F ) and (, F ) be measurable spaces and let µ be a measure on (, F ). If f : is measurable, then f (µ)(ã) := µ{f Ã}, Ã F, defines a measure f (µ) on (, F ). This measure is called the image of µ under f. In the context where (, F, P) is a probability space, the image of P under f is called the distribution of f. Concerning existence and uniqueness of measures we have the following two fundamental results. Theorem 1.8 (Caratheodory). Let A be a non-empty collection of subsets of a set that is closed under taking complements and finite unions. If µ : A [0, ] is a mapping such that properties 1.2(1) and 1.2(2) hold, then there exists a unique measure µ : σ(a ) [0, ] such that µ A = µ.

7 1.3 Measures 5 The proof is lengthy and rather tedious. The typical situation is that measures are given, and therefore the proof is omitted. The uniqueness part is a consequence of the next result, which is useful in many other situations as well. Theorem 1.9 (Dynkin). Let µ 1 and µ 2 be two finite measures defined on a measurable space (, F ). Let A F be a collection of sets with the following properties: (1) A is closed under finite intersections; (2) σ(a ), the σ-algebra generated by A, equals F. If µ 1 (A) = µ 2 (A) for all A A, then µ 1 = µ 2. Proof. Let D denote the collection of all sets D F with µ 1 (D) = µ 2 (D). Then A D and D is a Dynkin system, that is, D; if D 1 D 2 with D 1, D 2 D, then also D 2 \ D 1 D; if D 1 D 2... with all D n D, then also D n D. The finiteness of µ 1 and µ 2 is used to prove the second property. By assumption we have D F = σ(a ); we will show that σ(a ) D. To this end let D 0 denote the smallest Dynkin system in F containing A. We will show that σ(a ) D 0. In view of D 0 D, this will prove the lemma. Let C = {D 0 D 0 : D 0 A D 0 for all A A }. Then C is a Dynkin system and A C since A is closed under taking finite intersections. It follows that D 0 C, since D 0 is the smallest Dynkin system containing A. But obviously, C D 0, and therefore C = D 0. Now let C = {D 0 D 0 : D 0 D D 0 for all D D 0 }. Then C is a Dynkin system and the fact that C = D 0 implies that A C. Hence D 0 C, since D 0 is the smallest Dynkin system containing A. But obviously, C D 0, and therefore C = D 0. It follows that D 0 is closed under taking finite intersections. But a Dynkin system with this property is a σ-algebra. Thus, D 0 is a σ-algebra, and now A D 0 implies that also σ(a ) D 0. Example 1.10 (Lebesgue measure). In R d, let A be the collection of all finite unions of rectangles [a 1, b 1 ) [a d, b d ). Define µ : A [0, ] by µ([a 1, b 1 ) [a d, b d )) := d (b n a n ) and extend this definition to finite disjoint unions by additivity. It can be verified that A and µ satisfy the assumptions of Caratheodory s theorem. The resulting measure µ on σ(a ) = B(R d ), the Borel σ-algebra of R d, is called the Lebesgue measure.

8 6 1 Measure and Integral Example Let ( 1, F 1, P 1 ), ( 2, F 2, P 2 ),... be a sequence of probability spaces. There exists a unique probability measure P = P n on F = F n which satisfies P(A 1 A N N+1 N+2... ) = N P n (A n ) for all N 1 and A n F n for n = 1,..., N. Finite products of arbitrary measures µ 1,..., µ N can be defined similarly. For example, the Lebesgue measure on R d is the product measure of d copies of the Lebesgue measure on R. As an illustration of the use of Dynkin s lemma we conclude this section with an elementary result on independence. In probability theory, a measurable function f : R, where (, F, P) is a probability space, is called a random variable. Random variables f 1,..., f N : R are said to be independent if for all B 1,..., B N B(R) we have P{f 1 B 1,..., f N B N } = N P{f 1 B 1 }. Proposition The random variables f 1,..., f N : R are independent if and only if ν = N ν n, where ν is the distribution of (f 1,..., f N ) and ν n is the distribution of f n, n = 1,..., N. Proof. By definition, f 1,..., f N are independent if and only if ν(b) = N ν n(b n ) for all B = B 1 B N with B n B(R) for all n = 1,..., N. Since these sets generate B(R N ), by Dynkin s lemma the latter implies ν = N ν n. 1.4 The Lebesgue integral Let (, F ) be a measurable space. A simple function is a function f : R that can be represented in the form f = N c n 1 An with coefficients c n R and disjoint sets A n F for all n = 1,..., N. Proposition A function f : [, ] is measurable if and only if it is the pointwise limit of a sequence of simple functions. If in addition f is non-negative, we may arrange that 0 f n f pointwise. Proof. Take, for instance, f n = 2 2n m= 2 2n m 2 1 n {f [ m 2 n, m+1 2n )}.

9 1.4 The Lebesgue integral Integration of simple functions Let (, F, µ) be a measure space. For a non-negative simple function f = N c n1 An we define f dµ := N c n µ(a n ). We allow µ(a n ) to be infinite; this causes no problems because the coefficients c n are non-negative (we use the convention 0 = 0). It is easy to check that this integral is well-defined, in the sense that it does not depend on the particular representation of f as a simple function. Also, the integral is linear with respect to addition and multiplication with non-negative scalars, af 1 + bf 2 dµ = a f 1 dµ + b f 2 dµ, and monotone in the sense that if 0 f 1 f 2 pointwise, then f 1 dµ f 2 dµ Integration of non-negative functions In what follows, a non-negative function is a function with values in [0, ]. For a non-negative measurable function f we choose a sequence of simple functions 0 f n f and define f dµ := lim f n dµ. The following lemma implies that this definition does not depend on the approximating sequence. Lemma For a non-negative measurable function f and non-negative simple functions f n and g such that 0 f n f and g f pointwise we have g dµ lim f n dµ. Proof. First consider the case g = 1 A. Fix ε > 0 arbitrary and let A n = {1 A f n 1 ε}. Then A 1 A 2... A and therefore µ(a n ) µ(a n ). Since 1 A f n (1 ε)1 An, lim 1 A f n dµ (1 ε) lim µ(a n) = (1 ε)µ(a) = (1 ε) This proves the lemma for g = 1 A. The general case follows by linearity. g dµ.

10 8 1 Measure and Integral The integral is linear and monotone on the set of non-negative measurable functions. Indeed, if f and g are such functions and 0 f n f and 0 g n g, then for a, b 0 we have 0 af n + bg n af + bg and therefore af + bg dµ = lim af n + bg n dµ = a lim f n dµ + b lim g n dµ = a f dµ + b g dµ. If such f and g satisfy f g pointwise, then by noting that 0 f n max{f n, g n } g we see that f dµ = lim f n dµ lim max{f n, g n } dµ g dµ. The next theorem is the cornerstone of Integration Theory. Theorem 1.15 (Monotone Convergence Theorem). Let 0 f 1 f 2... be a sequence of non-negative measurable functions converging pointwise to a function f. Then, lim f n dµ = f dµ. Proof. First note that f is non-negative and measurable. For each n 1 choose a sequence of simple functions 0 f nk k f n. Set g nk := max{f 1k,..., f nk }. For m n we have g mk g nk. Also, for k l we have f mk f ml, m = 1,..., n, and therefore g nk g nl. We conclude that the functions g nk are monotone in both indices. From f mk f m f n, 1 m n, we see that f nk g nk f n, and we conclude that 0 g nk k f n. From f n = lim k g nk lim k g kk f we deduce that 0 g kk f. Recalling that g kk f k it follows that f dµ = lim g kk dµ lim f k dµ f dµ. k k Example In the situation of Example 1.7 we have the following substitution formula. For any measurable f : 1 2 and non-negative measurable φ : 2 R,

11 1.4 The Lebesgue integral 9 φ f dµ = φ df (µ). 1 2 To prove this, note the this is trivial for simple functions φ = 1 A with A F 2. By linearity, the identity extends to non-negative simple functions φ, and by monotone convergence (using Proposition 1.13) to non-negative measurable functions φ. From the monotone convergence theorem we deduce the following useful corollary. Theorem 1.17 (Fatou Lemma). Let (f n ) be a sequence of non-negative measurable functions on (, F, µ). Then lim inf f n dµ lim inf f n dµ. Proof. From inf k n f k f m, m n, we infer inf f k dµ inf k n m n Hence, by the monotone convergence theorem, lim inf f n dµ = lim inf f k dµ k n = lim inf f k dµ lim k n inf m n Integration of real-valued functions f m dµ. A measurable function f : [, ] is called integrable if f dµ <. f m dµ = lim inf f n dµ. Clearly, if f and g are measurable and g f pointwise, then g is integrable if f is integrable. In particular, if f is integrable, then the non-negative functions f + and f are integrable, and we define f dµ := f + dµ f dµ. For a set A F we write A f dµ := 1 A f dµ, noting that 1 A f is integrable. The monotonicity and additivity properties of this integral carry over to this more general situation, provided we assume that the functions we integrate are integrable. The next result is among the most useful in analysis.

12 10 1 Measure and Integral Theorem 1.18 (Dominated Convergence Theorem). Let (f n ) be a sequence of integrable functions such that lim f n = f pointwise. If there exists an integrable function g such that f n g for all n 1, then In particular, lim f n f dµ = 0. lim f n dµ = f dµ. Proof. We make the preliminary observation that if (h n ) is a sequence of non-negative measurable functions such that lim h n = 0 pointwise and h is a non-negative integrable function such that h n h for all n 1, then by the Fatou lemma h dµ = lim inf lim inf (h h n) dµ h h n dµ = Since h dµ is finite, it follows that 0 lim sup therefore h n dµ = 0. lim h dµ lim sup h n dµ. h n dµ 0 and The theorem follows by applying this to h n = f n f and h = 2 g. Let (, F, P) be a probability space. The integral of an integrable random variable f : R is called the expectation of f, notation Ef = f dp. Proposition If f : R and g : R are independent integrable random variables, then fg : R is integrable and Efg = Ef Eg. Proof. To prove this we first observe that for any non-negative Borel measurable φ : R R, the random variables φ f and φ g are independent. Taking φ(x) = x + and φ(x) = x, it follows that we may consider the positive and negative parts of f and g separately. In doing so, we may assume that f and g are non-negative. Denoting the distributions of f, g, and (f, g) by ν f, ν g, and ν (f,g) respectively, by the substitution formula (with φ(x, y) = xy1 [0, ) 2(x, y)) and the fact that ν (f,g) = ν f ν g we obtain

13 1.5 L p -spaces 11 E(fg) = xy dν (f,g) (x, y) = xy d(ν f dν g )(x, y) [0, ) 2 [0, ) 2 = xy dν f (x)dν g (y) = Ef Eg. [0, ) [0, ) This proves the integrability of f g along with the desired identity The role of null sets We begin with a simple observation. Proposition Let f be a non-negative measurable function. 1. If f dµ <, then µ{f = } = If f dµ = 0, then µ{f 0} = 0. Proof. For all c > 0 we have 0 c1 {f= } f and therefore 0 cµ{f = } f dµ. The first result follows from this by letting c. For the second, note that for all n 1 we have 1 n 1 {f 1 n } f and therefore 1 n µ{ f 1 } f dµ = 0. n It follows that µ{f 1 n } = 0. Now note that {f > 0} = {f 1 n }. Proposition If f is integrable and µ{f 0} = 0, then f dµ = 0. Proof. By considering f + and f separately we may assume f is non-negative. Choose simple functions 0 f n f. Then µ{f n > 0} µ{f > 0} = 0 and therefore f n dµ = 0 for all n 1. The result follows from this. What these results show is that in everything that has been said so far, we may replace pointwise convergence by pointwise convergence µ-almost everywhere, where the latter means that we allow an exceptional set of µ-measure zero. This applies, in particular, to the monotonicity assumption in the monotone convergence theorem and the convergence and domination assumptions in the dominated convergence theorem. 1.5 L p -spaces Let µ be a measure on (, F ) and fix 1 p <. We define L p (µ) as the set of all measurable functions f : R such that

14 12 1 Measure and Integral For such functions we define ( f p := f p dµ <. f p dµ ) 1 p The next result shows that L p (µ) is a vector space: Proposition 1.22 (Minkowski s inequality). For all f, g L p (µ) we have f + g L p (µ) and f + g p f p + g p. Proof. By elementary calculus it is checked that for all non-negative real numbers a and b one has (a + b) p = inf t (0,1) t1 p a p + (1 t) 1 p b p. Applying this identity to f(x) and g(x) and integrating with respect to µ, for all fixed t (0, 1) we obtain f(x) + g(x) p dµ(x) ( f(x) + g(x) ) p dµ(x) t 1 p f(x) p dµ(x) + (1 t) 1 p g(x) p dµ(x). Stated differently, this says that f + g p p t 1 p f p p + (1 t) 1 p g p p.. Taking the infimum over all t (0, 1) gives the result. In spite of this result, p is not a norm on L p (µ), because f p = 0 only implies that f = 0 µ-almost everywhere. In order to get around this imperfection, we define an equivalence relation on L p (µ) by f g f = g µ-almost everywhere. The equivalence class of a function f modulo is denoted by [f]. On the quotient space L p (µ) := L p (µ)/ we define a scalar multiplication and addition in the natural way: c[f] := [cf], [f] + [g] := [f + g]. We leave it as an exercise to check that both operators are well defined. With these operations, L p (µ) is a normed vector space with respect to the norm

15 1.5 L p -spaces 13 [f] p := f p. Following common practice we shall make no distinction between a function in L p (µ) and its equivalence class in L p (µ). The next result states that L p (µ) is a Banach space (which, by definition, is a complete normed vector space): Theorem The normed space L p (µ) is complete. Proof. Let (f n ) be a Cauchy sequence with respect to the norm p of L p (µ). By passing to a subsequence we may assume that f n+1 f n p 1, n = 1, 2,... 2n Define the non-negative measurable functions g N := N 1 n=0 f n+1 f n, g := f n+1 f n, with the convention that f 0 = 0. By the monotone convergence theorem, g p dµ = lim g p N dµ. N n=0 Taking p-th roots and using Minkowski s inequality we obtain N 1 g p = lim g N p lim f n+1 f n p = N N n=0 f n+1 f n p 1+ f 1 p. If follows that g is finitely-valued µ-almost everywhere, which means that the sum defining g converges absolutely µ-almost everywhere. As a result, the sum f := (f n+1 f n ) n=0 n=0 converges on the set {g < }. On this set we have N 1 f = lim (f n+1 f n ) = lim f N. N N n=0 Defining f to be zero on the null set {g = }, the resulting function f is measurable. From N 1 f N p = (f n+1 f n ) p n=0 ( N 1 n=0 ) p f n+1 f n g p and the dominated convergence theorem we conclude that

16 14 1 Measure and Integral lim f f N p = 0. N We have proved that a subsequence of the original Cauchy sequence converges to f in L p (µ). As is easily verified, this implies that the original Cauchy sequence converges to f as well. In the course of the proof we obtained the following result: Corollary Every convergence sequence in L p (µ) has a µ-almost everywhere convergent subsequence. For p = the above definitions have to be modified a little. We define L (µ) to be the vector space of all measurable functions f : R such that sup f(ω) < ω and define, for such functions, The space f := sup f(ω). ω L (µ) := L (µ)/ is again a normed vector space in a natural way. The simple proof that L (µ) is a Banach space is left as an exercise. We continue with an approximation result, the proof of which is an easy application of Proposition 1.13 together with the dominated convergence theorem: Proposition For 1 p, the simple functions in L p (µ) are dense in L p (µ). We conclude with an important inequality, known as Hölder s inequality. For p = q = 2, it is known as the Cauchy-Schwarz inequality. Theorem Let 1 p, q satisfy 1 p + 1 q g L q (µ), then fg L 1 (µ) and = 1. If f Lp (µ) and fg 1 f p g q. Proof. For p = 1, q = and for p =, q = 1, this follows by a direct estimate. Thus we may assume from now on that 1 < p, q <. The inequality is then proved in the same way as Minkowski s inequality, this time using the identity ( t p a p ) ab = inf t>0 p + bq qt q.

17 1.7 Notes Fubini s theorem A measure space (, F, µ) is called σ-finite if we can write = (n) with sets (n) F satisfying µ( (n) ) <. Theorem 1.27 (Fubini). Let ( 1, F 1, µ 1 ) and ( 1, F 1, µ 1 ) be σ-finite measure spaces. For any non-negative measurable function f : 1 2 R the following assertions hold: (1) The non-negative function ω 1 2 f(ω 1, ω 2 ) dµ 2 (ω 2 ) is measurable; (2) The non-negative function ω 2 1 f(ω 1, ω 2 ) dµ 1 (ω 1 ) is measurable; (3) We have f d(µ 1 µ 2 ) = 1 2 f dµ 1 dµ 2 = 1 f dµ 2 dµ We shall not give a detailed proof, but sketch the main steps. First, for f(ω 1, ω 2 ) = 1 A1 (ω 1 )1 A2 (ω 2 ) with A 1 F 1, A 2 F 2, the result is clear. Dynkin s lemma is used to extend the result to functions f = 1 A with A F = F 1 F 2. One has to assume first that µ 1 and µ 2 are finite; the σ-finite case then follows by approximation. By taking linear combinations, the result extends to simple functions. The result for arbitrary non-negative measurable functions then follows by monotone convergence. A variation of the Fubini theorem holds if we replace non-negative measurable by integrable. In that case, for µ 1 -almost all ω 1 1 the function ω 2 f(ω 1, ω 2 ) is integrable and for µ 2 -almost all ω 2 2 the function ω 1 f(ω 1, ω 2 ) is integrable. Moreover, the functions ω 1 2 f(ω 1, ω 2 ) dµ 2 (ω 2 ) and ω 2 1 f(ω 1, ω 2 ) dµ 1 (ω 1 ) are integrable and the above identities hold Notes The proof of the monotone convergence theorem is taken from Kallenberg [1]. The proofs of the Minkowski inequality and Hölder inequality, although not the ones that are usually presented in introductory courses, belong to the folklore of the subject.

18

19 2 Conditional expectations In this lecture we introduce conditional expectations and study some of their basic properties. We begin by recalling some terminology. A random variable is a measurable function on a probability space (, F, P). In this context, is often called the sample space and the sets in F the events. For an integrable random variable f : R we write Ef := f dp for its expectation. Below we fix a sub-σ-algebra G of F. For 1 p we denote by L p (, G ) the subspace of all f L p () having a G -measurable representative. With this notation, L p () = L p (, F ). We will be interested in the problem of finding good approximations of a random variable f L p () by random variables in L p (; G ). Lemma 2.1. L p (, G ) is a closed subspace of L p (). Proof. Suppose that (ξ n ) is a sequence in L p (, G ) such that lim f n = f in L p (). We may assume that the f n are pointwise defined and G - measurable. After passing to a subsequence (when 1 p < ) we may furthermore assume that lim f n = f almost surely. The set C of all ω where the sequence (f n (ω)) converges is G -measurable. Put f := lim 1 C f n, where the limit exists pointwise. The random variable f is G -measurable and agrees almost surely with f. This shows that f defines an element of L p (, G ). Our aim is to show that L p (, G ) is the range of a contractive projection in L p (). For p = 2 this is clear: we have the orthogonal decomposition L 2 () = L 2 (, G ) L 2 (, G )

20 18 2 Conditional expectations and the projection we have in mind is the orthogonal projection, denoted by P G, onto L 2 (, G ) along this decomposition. Following common usage we write E(f G ) := P G (f), f L 2 (), and call E(f G ) the conditional expectation of f with respect to G. Let us emphasise that E(f G ) is defined as an element of L 2 (, G ), that is, as an equivalence class of random variables. Lemma 2.2. For all f L 2 () and G G we have E(f G ) dp = f dp. G As a consequence, if f 0 almost surely, then E(f G ) 0 almost surely. Proof. By definition we have f E(f G ) L 2 (, G ). If G G, then 1 G L 2 (, G ) and therefore 1 G (f E(f G )) dp = 0. This gives the desired identity. For the second assertion, choose a G -measurable representative of g := E(f G ) and apply the identity to the G -measurable set {g < 0}. Taking G = we obtain the identity E(E(f G )) = Ef. This will be used in the lemma, which asserts that the mapping f E(f G ) is L 1 -bounded. Lemma 2.3. For all f L 2 () we have E E(f G ) E f. Proof. It suffices to check that E(f G ) E( f G ), since then the lemma follows from E E(f G ) EE( f G ) = E f. Splitting f into positive and negative parts, almost surely we have E(f G ) = E(f + G ) E(f G ) E(f + G ) + E(f G ) = E(f + G ) + E(f G ) = E( f G ). Since L 2 () is dense in L 1 () this lemma shows that the condition expectation operator has a unique extension to a contractive linear operator on L 1 (), which we also denote by E( G ). This operator is a projection (this follows by noting that it is a projection in L 2 () and then using that L 2 () is dense in L 1 ()) and it is positive in the sense that it maps positive random variables to positive random variables; this follows from Lemma 2.2 by approximation. Lemma 2.4 (Conditional Jensen inequality). If φ : R R is convex, then for all f L 1 () such that φ f L 1 () we have, almost surely, G φ E(f G ) E(φ f G ).

21 2 Conditional expectations 19 Proof. If a, b R are such that at + b φ(t) for all t R, then the positivity of the conditional expectation operator gives ae(f G ) + b = E(af + b G ) E(φ f G ) almost surely. Since φ is convex we can find real sequences (a n ) and (b n ) such that φ(t) = sup n 1 (a n t + b n ) for all t R; we leave the proof of this fact as an exercise. Hence almost surely, φ E(f G ) = sup a n E(f G ) + b n E(φ f G ). n 1 Theorem 2.5 (L p -contractivity). For all 1 p the conditional expectation operator extends to a contractive positive projection on L p () with range L p (, G ). For f L p (), the random variable E(f G ) is the unique element of L p (, G ) with the property that for all G G, E(f G ) dp = f dp. (2.1) G Proof. For 1 p < the L p -contractivity follows from Lemma 2.4 applied to the convex function φ(t) = t p. For p = we argue as follows. If f L (), then 0 f f 1 and therefore 0 E( f G ) f 1 almost surely. Hence, E( f G ) L () and E( f G ) f. For 2 p, (2.1) follows from Lemma 2.2. For f L p () with 1 p < 2 we choose a sequence (f n ) in L 2 () such that lim f n = f in L p (). Then lim E(f n G ) = E(f G ) in L p () and therefore, for any G G, G E(f G ) dp = lim E(f n G ) dp = lim f n dp = f dp. G G G If g L p (, G ) satisfies G g dp = G f dp for all G G, then G g dp = E(f G ) dp for all G G. Since both g and E(f G ) are G -measurable, as G in the proof of the second part of Lemma 2.2 this implies that g = E(f G ) almost surely. In particular, E(E(f G ) G ) = E(f G ) for all f L p () and E(f G ) = f for all f L p (, G ). This shows that E( G ) is a projection onto L p (, G ). The next two results develop some properties of conditional expectations. Proposition 2.6. (1) If f L 1 () and H is a sub-σ-algebra of G, then almost surely G E(E(f G ) H ) = E(f H ).

22 20 2 Conditional expectations (2) If f L 1 () is independent of G (that is, f is independent of 1 G for all G G ), then almost surely E(f G ) = Ef. (3) If f L p () and g L q (, G ) with 1 p, q, 1 p + 1 q = 1, then almost surely E(gf G ) = ge(f G ). Proof. (1): For all H H we have H E(E(f G ) H ) dp = E(f G ) dp = H f dp by Theorem 2.5, first applied to H and then to G (observe that H H G ). Now the result follows from the uniqueness part of the theorem. (2): For all G G we have G Ef dp = E1 GEf = E1 G f = f dp, and G the result follows from the uniqueness part of Theorem 2.5. (3): For all G, G G we have G 1 G E(f G ) dp = E(f G ) dp = G G f dp = G G G f1 G dp. Hence E(f1 G G ) = 1 G E(f G ) by the uniqueness part of Theorem 2.5. By linearity, this gives the result for simple functions g, and the general case follows by approximation. Example 2.7. Let f : ( r, r) R be integrable and let G be the σ-algebra of all symmetric Borel sets in ( r, r) (we call a set B symmetric if B = B). Then, E(f G ) = 1 2 (f + ˇf) almost surely, where ˇf(x) = f( x). This is verified by checking the condition (2.1). Example 2.8. Let f : [0, 1) R be integrable and let G be the σ-algebra generated by the sets I n := [ n 1 N, n N ), n = 1,..., N. Then, E(f G ) = N m n 1 [ n 1 N, n N ) almost surely, where m n = 1 f(x) dx. I n I n Again this is verified by checking the condition (2.1). If f and g are random variables on (, F, P), we write E(g f) := E(g σ(f)), where σ(f) is the σ-algebra generated by f. Example 2.9. Let f 1,..., f N be independent integrable random variables satisfying E(f 1 ) = = E(f N ) = 0, and put S N := f f N. Then, for 1 n N, E(S N f n ) = f n almost surely. This follows from the following almost sure identities: E(S N f n ) = E(f n f n ) + m n E(f m f n ) = f n + m n E(f m ) = f n.

23 2.1 Notes 21 Example Let f 1,..., f N be independent and identically distributed integrable random variables and put S N := f f N. Then and therefore E(f 1 S N ) = = E(f N S N ) almost surely E(f n S N ) = S N /N almost surely, n = 1,..., N. It is clear that the second identity follows from the first. To prove the first, let µ denote the (common) distribution of the random variables f n. By independence, the distribution of the R N -valued random variable (f 1,..., f N ) equals the N-fold product measure µ N := µ µ. Then, for any Borel set B B(R) we have, using the substitution formula and Fubini s theorem, E(f n S N ) dp {S N B} = = = {S N B} f n dp {x 1+ +x N B} R N 1 which is independent of n. x n dµ N (x) {y B z 1 z N 1 } y dµ(y) dµ N 1 (z 1,..., z N 1 ), 2.1 Notes We recommend Williams [2] for an introduction to the subject. The bible for modern probabilists is Kallenberg [1]. The definition of conditional expectations starting from orthogonal projections in L 2 () follows Kallenberg [1]. Many textbooks prefer the shorter (but less elementary and transparent) route via the Radon-Nikodým theorem. In this approach, one notes that if f is a random variable on (, F, P) and G is a sub-σ-algebra of F, then Q(G) := f dp, G G, G defines a probability measure Q on (, G, P G ) which is absolutely continuous with respect to P G. Hence by the Radon-Nikodým, Q has an integrable density with respect to P G. This density equals the conditional expectation E(f G ) almost surely.

24

25 References 1. O. Kallenberg, Foundations of Modern Probability, second ed., Probability and its Applications (New York), Springer-Verlag, New York, D. Williams, Probability with Martingales, Cambridge Mathematical Textbooks, Cambridge University Press, Cambridge, 1991.