MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 13 10/22/2008 PRODUCT MEASURE AND FUBINI S THEOREM

Transcription

1 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 13 10/22/2008 PRODUCT MEASURE AND FUBINI S THEOREM Contents 1. Product measure 2. Fubini s theorem In elementary math and calculus, we often interchange the order of summation and integration. The discussion here is concerned with conditions under which this is legitimate. 1 PRODUCT MEASURE Consider two probabilistic experiments described by probability spaces (Ω 1, F 1, P 1 ) and (Ω 2, F 2, P 2 ), respectively. We are interested in forming a probabilistic model of a joint experiment in which the original two experiments are carried out independently. 1.1 The sample space of the joint experiment If the first experiment has an outcome ω 1, and the second has an outcome ω 2, then the outcome of the joint experiment is the pair (ω 1, ω 2 ). This leads us to define a new sample space Ω = Ω 1 Ω The σ-field of the joint experiment Next, we need a σ-field on Ω. If A 1 F 1, we certainly want to be able to talk about the event {ω 1 A 1 } and its probability. In terms of the joint experiment, this would be the same as the event A 1 Ω 1 = {(ω 1, ω 2 ) ω 1 A 1, ω 2 Ω 2 }. 1

2 Thus, we would like our σ-field on Ω to include all sets of the form A 1 Ω 2, (with A 1 F 1 ) and by symmetry, all sets of the form Ω 1 A 2 (with (A 2 F 2 ). This leads us to the following definition. Definition 1. We define F 1 F 2 as the smallest σ-field of subsets of Ω 1 Ω 2 that contains all sets of the form A 1 Ω 2 and Ω 1 A 2, where A 1 F 1 and A 2 F 2. Note that the notation F 1 F 2 is misleading: this is not the Cartesian product of F 1 and F 2! Since σ-fields are closed under intersection, we observe that if A i F i, then A 1 A 2 = (A 1 Ω 2 ) (Ω 1 A 2 ) F 1 F 2. It turns out (and is not hard to show) that F 1 F 2 can also be defined as the smallest σ-field containing all sets of the form A 1 A 2, where A i F i. 1.3 The product measure We now define a measure, to be denoted by P 1 P 2 (or just P, for short) on the measurable space (Ω 1 Ω 2, F 1 F 2 ). To capture the notion of independence, we require that P(A 1 A 2 ) = P 1 (A 1 )P 2 (A 2 ), A 1 F 1, A 2 F 2. (1) Theorem 1. There exists a unique measure P on (Ω 1 Ω 2, F 1 F 2 ) that has property (1). Theorem 1 has the flavor of Carathéodory s extension theorem: we define a measure on certain subsets that generate the σ-field F 1 F 2, and then extend it to the entire σ-field. However, Caratheodory s extension theorem involves certain conditions, and checking them does take some nontrivial work. Various proofs can be found in most measure-theoretic probability texts. 1.4 Beyond probability measures Everything in these notes extends to the case where instead of probability measures P i, we are dealing with general measures µ i, under the assumptions that the measures µ i are σ-finite. (A measure µ is called σ-finite if the set Ω can be partitioned into a countable union of sets, each of which has finite measure.) 2

3 The most relevant example of a σ-finite measure is the Lebesgue measure on the real line. Indeed, the real line can be broken into a countable sequence of intervals (n, n + 1], each of which has finite Lebesgue measure. 1.5 The product measure on R 2 The two-dimensional plane R 2 is the Cartesian product of R with itself. We endow each copy of R with the Borel σ-field B and one-dimensional Lebesgue measure. The resulting σ-field B B is called the Borel σ-field on R 2. The resulting product measure on R 2 is called two-dimensional Lebesgue measure, to be denoted here by λ 2. The measure λ 2 corresponds to the natural notion of area. For example, λ 2 ([a, b] [c, d]) = λ([a, b]) λ([c, d]) = (b a) (d c). More generally, for any nice set of the form encountered in calculus, e.g., sets of the form A = {(x, y) f(x, y) c}, where f is a continuous function, λ 2 (A) coincides with the usual notion of the area of A. Remark for those of you who know a little bit of topology otherwise ignore it. We could define the Borel σ-field on R 2 as the σ-field generated by the collection of open subsets of R 2. (This is the standard way of defining Borel sets in topological spaces.) It turns out that this definition results in the same σ-field as the method of Section FUBINI S THEOREM Fubini s theorem is a powerful tool that provides conditions for interchanging the order of integration in a double integral. Given that sums are essentially special cases of integrals (with respect to discrete measures), it also gives conditions for interchanging the order of summations, or the order of a summation and an integration. In this respect, it subsumes results such as Corollary 1 at the end of the notes for Lecture 12. In the sequel, we will assume that g : Ω 1 Ω 2 R is a measurable function. This means that for any Borel set A R, the set {(ω 1, ω 2 ) g(ω 1, ω 2 ) A} belongs to the σ-field F 1 F 2. As a practical matter, it is enough to verify that for any scalar c, the set {(ω 1, ω 2 ) g(ω 1, ω 2 ) c} is measurable. Other than using this definition directly, how else can we verify that such a function g is measurable? The basic tools at hand are the following: (a) continuous functions from R 2 to R are measurable; 3

4 (b) indicator functions of measurable sets are measurable; (c) combining measurable functions in the usual ways (e.g., adding them, multiplying them, taking limits, etc.) results in measurable functions. Fubini s theorem holds under two different sets of conditions: (a) nonnegative functions g (compare with the MCT); (b) functions g whose absolute value has a finite integral (compare with the DCT). We state the two versions separately, because of some subtle differences. The two statements below are taken verbatim from the text by Adams & Guillemin, with minor changes to conform to our notation. Theorem 2. Let g : Ω 1 Ω 2 R be a nonnegative measurable function. Let P = P 1 P 2 be a product measure. Then, (a) For every ω 1 Ω 1, g(ω 1, ω 2 ) is a measurable function of ω 2. (b) For every ω 2 Ω 2, g(ω 1, ω 2 ) is a measurable function of ω 1. (c) Ω 2 g(ω 1, ω 2 ) dp 2 is a measurable function of ω 1. (d) Ω 1 g(ω 1, ω 2 ) dp 1 is a measurable function of ω 2. (e) We have [ ] [ ] g(ω 1, ω 2 ) dp 2 dp 1 = g(ω 1, ω 2 ) dp 1 dp 2 Ω 1 Ω 2 Ω 2 Ω 1 = g(ω 1, ω 2 ) dp. Ω 1 Ω 2 Note that some of the integrals above may be infinite, but this is not a problem; since everything is nonnegative, expressions of the form do not arise. Recall now that a function is said to be integrable if it is measurable and the integral of its absolute value is finite. 4

5 Theorem 3. Let g : Ω 1 Ω 2 R be a measurable function such that g(ω 1, ω 2 ) dp <, where P = P 1 P 2. Ω 1 Ω 2 (a) For almost all ω 1 Ω 1, g(ω 1, ω 2 ) is an integrable function of ω 2. (b) For almost all ω 2 Ω 2, g(ω 1, ω 2 ) is an integrable function of ω 1. (c) There exists an integrable function h : Ω 1 R such that Ω2 g(ω 1, ω 2 ) dp 2 = h(ω 1 ), a.s. (i.e., except for a set of ω 1 of zero P 1 -measure for which Ω 2 g(ω 1, ω 2 ) dp 2 is undefined or infinite). (d) There exists an integrable function h : Ω 2 R such that Ω1 g(ω 1, ω 2 ) dp 1 = h(ω 2 ), a.s. (i.e., except for a set of ω 2 of zero P 2 -measure for which g(ω 1, ω 2 ) dp 1 is undefined or infinite). Ω 1 (e) We have [ ] [ ] g(ω 1, ω 2 ) dp 2 dp 1 = g(ω 1, ω 2 ) dp 1 dp 2 Ω 1 Ω 2 Ω 2 Ω 1 = g(ω 1, ω 2 ) dp. Ω 1 Ω 2 We repeat that all of these results remain valid when dealing with σ-finite measures, such as the Lebesgue measure on R 2. This provides us with conditions for the familiar calculus formula g(x, y) dx dy = g(x, y) dy dx. In order to apply Theorem 3, we need a practical method for checking the integrability condition g(ω 1, ω 2 ) dp <. Ω 1 Ω 2 in Theorem 3. Here, Theorem 2 comes to the rescue. Indeed, by Theorem 2, we have g(ω 1, ω 2 ) dp = g(ω 1, ω 2 ) dp 2 dp 1, Ω 1 Ω 2 Ω 1 Ω 2 5

6 so all we need is to work with the right hand side, and integrate one variable at a time, possibly also using some bounds on the way. Finally, let us note that all the hard work goes into proving Theorem 2. Theorem 3 is relatively easy to derive once Theorem 2 is available: Given a function g, decompose it into its positive and negative parts, apply Theorem 2 to each part, and in the process make sure that you do not encounter expressions of the form. 3 Some cautionary examples We give a few examples where Fubini s theorem does not apply. 3.1 Nonnegative and Integrability Suppose both of our sample spaces are the nonnegative integers: Ω 1 = Ω 2 = {1, 2,..., }. The σ-fields F 1 and F 2 will be all subsets of Ω 1 and Ω 2, respectively. Then, σ(f 1 F 2 ) will be composed of all subsets of {1, 2,..., } 2. Both P 1 and P 2 will be the counting measure, i.e. P (A) = A. This means that gdp 1 = f(a), hdp 2 = h(b), fdp 1 P 2 = f(c). A a A B b B C c C Consider the function f defined by f(m, m) = 1, f(m, m + 1) = 1, and f = 0 elsewhere. It is easier to visualize f with a picture: So fdp 1 dp 2 = f(n, m) = 0 = 1 = f(n, m) = fdp 2 dp 1 Ω 1 Ω 2 n m m n Ω 2 Ω 1 The problem is that the function we are integrating is neither nonnegative nor integrable. 6

7 3.2 σ-finiteness Let Ω 1 = (0, 1), and let F 1 be the Borel sets, and P 1 be the Lebesgue measure. Let Ω 2 = (0, 1) and F 2 be the set of all subsets of (0, 1), and let P 2 be the counting measure. Define f(x, y) = 1 if x = y and 0 otherwise. Then, f(x, y)dp 2 (y)dp 1 (x) = 1dP 1 (y) = 1, Ω 1 Ω 2 Ω 1 but f(x, y)dp 1 (x)dp 2 (y) = 0dP 2 (y) = 0. Ω 2 Ω 1 Ω 2 The problem is that the counting measure on (0, 1) is not σ-finite. 4 An application Let s apply Fubini s theorem to prove a generalization of a familiar relation from a beginning probability course. Let X be a nonnegative integer-valued random variable. Then, E[X] = This is usually proved as follows: i=1 i = p(i) P (X i). E[X] = ip(i) i=1 i=1 k=1 = p(i) k=1 i=k = P (X k) k=1 where the sum exchange is typically justified by an appeal to nonnegativity. Let s rigourously prove a justification of this relation in the most general case. We will show that if X is a nonnegative random variable, then E[X] = 0 P (X x)dx. 7

8 Proof: Define A = {(w, x) 0 x X(w)}. Intuitively, if Ω = R, then A would be the region under the curve X(w). We argue that E[X] = X(w)dP = 1 A (w, x)dxdp, Ω Ω 0 and now let s postpone the technical issues for a moment and interchange the integrals to get E[X] = 1 A (w, x)dp dx 0 Ω = P (X x)dx. 0 Now let s consider the technical details necessary to make the above argument work. The integral interchange can be justified on account of the funciton 1 A being nonnegative, so we just need to show that all the functions we deal with are measurable. In particular we need to show that: 1. For fixed x, 1 A (w, x) is a measurable functions of w. 2. For fixed w, 1 A (w, x) is a measurable function of x. 3. X(ω) is a measurable function of ω. 4. P (X x) is a measurable function of x A (w, x) is a measurable function of w and x. and we do this as follows: 1. For fixed x, 1 A (w, x) is the indicator function of the set X x, so it must be measurable. 2. For fixed w, 1 A (w, x) is the indicator function of the interval [0, X(w)], so it is lebesgue measurable. 3. X is measurable since its a random variable. 4. Using the notation Z(x) = P (X x), observe that if a {Z z}, then so is every number below a. It follows that the set {Z z} is always an interval, so it is Lebesgue measurable. 8

9 5. To show that 1 A is measurable, we argue that A is measurable.indeed, the function g : Ω R R defined by g(w, x) = X(w) is measurable, since for any Borel set B, g 1 (B) = X 1 (B) (, + ). Similarly, h : Ω R R defined as h(w, x) = x is measurable for the same reason. Since A = {g h} {h 0}, it follows that A is measurable. 9

10 MIT OpenCourseWare J / J Fundamentals of Probability Fall 2008 For information about citing these materials or our Terms of Use, visit: