Extension of measure

Transcription

1 1 Extension of measure Sayan Mukherjee Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard. It is easier to define the measure on a much smaller collection of the power set and state consistency conditions that allow us to extend the measure to the the power set. This is what Dynkin s π λ theorem was designed to do. Definition π-system) Given a set Ω a π system is a collection of subsets P that are closed under finite intersections. 1) P is non-empty; 2) A B P whenever A, B P. Definition λ-system) Given a set Ω a λ system is a collection of subsets L that contains Ω and is closed under complementation and disjoint countable unions. 1) Ω L ; 2) A L A c L ; 3) A n L, n 1 with A i A j = i j n=1 A n L. Definition σ-algebra) Let F be a collection of subsets of Ω. F is called a field algebra) if Ω F and F is closed under complementation and countable unions, 1) Ω F; 2) A F A c F; 3) A 1,..., A n F n j=1 A j F; 4) A 1,..., A n,... F j=1 A j F. A σ-algebra is a π system. A σ-algebra is a λ system but a λ system need not be a σ algebra, a λ system is a weaker system. The difference is between unions and disjoint unions. Example Ω = {a, b, c, d} and L = {Ω,, {a, b}, {b, c}, {c, d}, {a, d}, {b, d}, {a, c}} L is closed under disjoint unions but not unions. However: Lemma A class that is both a π system and a λ system is a σ-algebra. Proof. We need to show that the class is closed under countable union. Consider A 1, A 2,..., A n L note that these A i are not disjoint. We disjointify them, B 1 = A 1 and B n = A n \ n 1 A n 1 i = A n Ac i. B n L by π system and complement property of λ system. A i = B i but the {B i } are disjoint so A i = B i L. We will start constructing probability measures on infinite sets and we will need to push the idea of extension of measure. In this the following theorem is central. Theorem Dynkin) 1) If P is a π system and L is a λ system, then P L implies σp) L. b) If P is a π system σp) = L P). Proof of a). Let L 0 be the λ system generated by P, the intersection of all λ systems containing P. This is a λ system and contains P and is contained in every λ system that contains P. So P L 0 L. If L 0 is also a π system, it is a λ system and so is a σ algebra. By minimality of σp) it follows σp) L 0 so P σp) L 0 L. So we need to show that L 0 is a π system. Set A σp) and define G A = {B σp) : A B L P)}.

2 2 1) We show if A L P) then G A is a λ system. i) Ω G A since A Ω = A L P). ii) Suppose B G A. B c A = A\A B) but B G A means A B L P) and since A L P) it holds A \ A B) = B c A L P) by complementarity. Since B c A L P) it holds that B c G A. iii) {B j } are mutually disjoint and B j G A then 2) We show if A P then L P) G A. A B j ) = A B j ) L P). j=1 Since A P L P) from 1) G A is a λ system, j=1 For B P we have A B P since A P and P is a π system. So if B P then A B P L P) implies B G A so P G A and since G A is a λ system L P) G A. 3) We rephrase 2) to state if A P and B L P) then A B L P) if we can extend A to L P) we are done). 4) We show that if A L P) then L P) G A. If B P and A L P) then A B L P) then from 3) it holds A B L P). So when A L P) it holds P G A so from 1) G A is a π system and L P) G A. 5) If A L P) then for any B L P), B G A then A B L P). The following lemma is a result of Dynkin s theorem and highlights uniqueness. Lemma If P i and P 2 are two probability measures on σp) where P is a π system. If P 1 and P 2 agree on P, then they agree on σp). Proof is a λ system and P L so σp) L. L = {A P : P 1 A) = P 2 A)}, We will need to define a probability measure, µ, for all sets in a σ algebra. It will be convenient and almost necessary to define a pre)-measure, µ 0, on a smaller collection F 0 that extends to µ on σf 0 ) and µ is unique, µ 1 F ) = µ 2 F ), F F 0 implies µ 1 F ) = µ 2 F ), F σf 0 ). Restatement of Dynkin s theorem in the above context is that the smallest σ algebra generated from a π system is the λ system generated from that π system. Theorem Let F 0 be a π system then λf 0 ) = σf 0 ). We now define the term extension and develop this idea of extension of measure. The main working example we will use in the development is. Ω = R, S = {a, b] : a b }, Pa, b]) = F b) F a). Remember, that the Borel sets BR) = σs). Goal: we have a probability measure on S we want to extend it to BR). Extension: If G 1 and G 2 are collections of subsets of Ω with G 1 G 2 and P 1 : G 1 [0, 1], P 2 : G 2 [0, 1]. P 2 is an extension of P 1 if P 2 G1 = P 1, P 2 A 1 ) = P 1 A i ) A i G 1.

3 3 Few terms: A measure is P additive if for disjoint sets A 1,..., A n G n A i n n A i G P A i ) = A measure is σ additive if for disjoint sets A 1,..., A n,... G n PA i ). A i G P A i ) = PA i ). Definition Semialgebra) A class S of subsets of Ω is a semialgebra if 1), Ω S 2) S is a π system 3) If A S there exists disjoint sets C 1,..., C n with each C i S such that A c = n C i. Example of Semialgebra: Intervals Ω = R, S = {a, b] : a b }. If I 1, I 2 S then I 1 I 2 S and I c is a union of disjoint intervals. The idea/plan: 1. Start with a semialgebra S 2. Show a unique extension to an algebra AS) generated by S 3. Show a unique extension to σas)) = σs). If S is a semialgebra of Ω then AS) = { S i : I is finite, {S i } is disjoint, S i S}, i I a system of finite collections of mutually disjoint sets. An example of extension of measure for a countable set. Ω = {ω 1,..., ω n,...} i p i p i 0, i = 1 p 1 = 1. F = PΩ) the power set of Ω. We define for all A F the following extension of measure PA) = p i. ω i A For this to be a proper measure the follow need to hold they do) 1. PA) 0 for all A F 2. PΩ) = p i = 1 3. If {A j } with j 1 are disjoint then P Aj ) = j=1 p i = p i = PA j ). w i ja j j ω i A j j Is the power set PΩ) countable? What was the semialgebra above? What would we do if Ω = R? As we consider the following theorems and proofs or proof sketches a good example to keep in mind is the case where Ω = R and S == {a, b] : a < b }.

4 4 Theorem First extension) S is a semialgebra of Ω and P : S [0, 1] is σ-additive on S and PΩ) = 1. There is a unique extension P of P to AS) defined by P S i ) = i PS i ), P Ω) = 1 and P is σ additive on AS). Proof Page 46, Probability Path by Resnick) We need to show that P S i ) = i PS i ), uniquely defines P. Suppose that A AS) has representations A = i I S i = j J S j, we need to verify Since S i A the following hold i I PS i ) = j J PS j). i I PS i ) = i I PS i A) = i I PS i j J = i I P j J S i S j), S j) and S i = j J S i S j S and P is additive on S so and by the same argument PS i ) = PS i S j) = i I i I j J i I PS j) = PS i S j) = j J i I j J i I We now check that P is σ additive on AS). Suppose for i 1, PS i S j), j J PS i S j). j J A i = j J i S ij AS), S ij S, and {A i, i 1} are mutually disjoint and A = A i AS). Since A AS), A can be represented as A = S k, k K S k S, k K, where K is a finite index set. So by the definition of P P A) = k K PS k ). We can write S k = S k A = S k A i = S k S ij. j J i

5 5 Since S k S ij S and j J i S k S ij = S k S and P is σ additive on S PS k ) = PS k S ij ) = PS k S ij ). k K k K j J i k K j J i Note that Sk S ij = A S ij = S ij S, k K and by additivity of P on S we show σ additivity PS k S ij ) = j J i k K = PS ij ) j J i P S ij ) = P A i ). j J i The last step is to check the extension is unique. Assume P 1 and P 2 aretwo additive extensions, then for any A = i I S i AS) we have P 1A) = i I PS i ) = P 2A). We now show how to extend from an algebra to a sigma algebra. Theorem Second extension, Caratheodory) A probability measure P defined on an algebra A has a unique extension to a probability measure P on σa). Proof sketch Page 4, Probability Theory by Varadhan) There are 5 steps to the proof. 1. Define the extension: {A j } are the set of all countable collections of A that are disjoint and we define the outer measure P as P A) = inf A ja j j PA j ). P will be the extension. 2. Show P satisfies properties of a measure i) Countable subattitivity P j A j ) j P A j ). ii) For all A A, P A) = PA). This is done in two steps by showing 1) P A) P A) and 2) P A) PA). 3. Checking measurability of the extension are all of the extra sets in going from the algebra to the σ algebra adding correctly). A set E is measurable if P A) P A E) + P A E c ), for all A A. The class M is the collection of measurable sets. We need to show that M is a σ algebra and P is σ additive on M.

6 6 4. Show A M. This implies that σa) M and P extends from A to σa). 5. Uniqueness. Let P 1 and P 2 be σ additive measures on A then show if for all A A P 1 A) = P 2 A), then P 1 B) = P 2 B), B σa). We now look at two examples. The first is the extension of the Lebesgue measure to Borel sets. The second example illustrates sets that are not measurable. Lebesgue measure: Ω = [0, 1], F = B0, 1]), S = {a, b] : 0 a b 1}. S is a semialgebra and we define the following measure on it λ : S a, b], λ ) = 0, λa, b]) = b a. This measure λ extends to the Borel set B0, 1]). To show that λ extends to B0, 1]) we need to show that it is σ additive on B0, 1]). We first show finite additivity. Set a, b] = K a i, b i ], where a = a 1, b K = b, b i = a i+1. We can see K λa i, b i ] = K b i a i ) = b 1 a 1 + b 2 a 2 + = b K a 1 = b a. We now note a few asymptotic properties of intervals a, b) = [a, b] = {a} = n=1 n=1 n=1 a, b 1 ], a < b n a 1n ], b, a < b a 1n, a ], a < b. The Borel σ algebra contains all singletons a as well as the forms a, b), [a, b], [a, b), a, b]. We now have to address σ additivity. First note that the interval [0, 1] is compact, this means that there exists a finite ε cover of the interval for any ε > 0. Let A n be a sequence of sets from S such that A n. We can define a set B n S such that its closure [B n ] A n and λa n ) λb n ) ε 2 n. Remember A n = and since the sets [B n ] are closed there exists a finite n 0 ε) such that n0 [B n] =. Remember that A n0 A n0 1 A 1 so ) n 0 n0 ) λa n0 ) = λ A n0 \ B k + λ B k n 0 ) = λ A n0 \ B k λ n 0 n0 ) A k \ B k n 0 λa k \ B k ) ε 2 k ε.

7 7 The reason this shows σ additivity is that if for all A n S, A n implies λa n ) 0 then σ additivity holds on σs). Not a measurable set: Let Ω = N be the natural numbers and E be the even numbers and E c the odd numbers and set F = {2 2k + 1,..., 2 2k+1 } = {2,. 5,..., 8, 17,..., 32, 65,..., 128,...}. k=0 1. For n = 2 2k the ratio P n F ) = #[F {1,..., n}]/n = n 1)/3n 1/3 and for n = 2 2k+1, P n F )2n 1)/3n 2/3 so P n cannot converge as n. 2. The even and odd portions of F and F c, A F E and B F c E c, have relative frequencies that do not converge. 3. C = A B does have a relative frequency that converges: P 2n C) = 1/2 + 1/2n so lim n P n = 1/2. 4. Both E and C have well defined asymptotic frequences but A = E C does not and the collection of sets M for which P n converges is not an algebra.