Solutions to In-Class Problems Week 15, Mon

Transcription

1 Massachusetts Istitute of Techology 6.042J/8.062J, Fall 02: Mathematics for Computer Sciece Prof. Albert Meyer ad Dr. Radhika Nagpal Solutios to I-Class Problems Week 5, Mo Problem. Prove that the Cetral Limit Theorem implies the Weak Law of Large Numbers. Hit: The oly properties of N (y) eeded i the proof are that lim y N (y) = ad lim y N ( y) = 0. Solutio. Note first that µ S = µ, Var [S ] = σ 2, ad so σ S = σ. Now, S µ > ɛ iff S µ > ɛ iff S µ > ɛ σ S σ S ɛ iff S >. σ But for ay real umber β > 0, ɛ > β σ will hold for all large. Hece, for ay β > 0 ad all large, { } Pr S µ ɛ > ɛ = Pr S > Pr { S σ > β}. () So S µ > ɛ { S > β} (by ()) = {S > β} + Pr {S < β} = N (β) + N ( β), (by the Cetral Limit Thm (8)) for all real umbers β > 0. By choosig β large eough, we ca esure that N (β) is arbitrarily close to ad N ( β) is arbitrarily close to 0, so that fial term above is arbitrarily close to -+0 = 0. Hece, S µ > ɛ = 0, which is the Weak Law of Large Numbers. Copyright 2002, Prof. Albert R. Meyer.

2 Solutios to I-Class Problems Week 5, Mo 2 NOTE: We did t get to the followig problem i class. Problem 2. To clarify the somewhat subtle differece betwee the Weak ad Strog Laws of Large Numbers, we will costruct a example of a sequece X, X 2,... of mutually idepedet radom variables that satisfies the Weak Law of Large Numbers, but ot the Strog Law. The distributio of X i will have to deped o i, because otherwise both laws would be satisfied. I particular, let X, X 2,... be a sequece of mutually idepedet radom variables such that X = 0, ad for each iteger i >, Pr {X i = i} = 2i log i, Pr {X i = i} = 2i log i, Pr {X i = 0} = i log i. Note that µ = E [X i ] = 0 for all i. (a) Show that Var [S ] = Θ( 2 / log ). Hit: / log > i/ log i for 2 i. Solutio. Var [S ] = Var [X i ] (idepedet variace additivity) i= [ ] 2 = Var [X ] + E X i E 2 [X i ] ( ) = 0 + i 2 0 i log i i =. log i = Θ( 2 / log ). (see below) (2) To justify (2), ote that x/ log x is icreasig for x > e sice its derivative (/ log x)( / log x) is This problem is adapted from Gristead & Sell, Itro. to Probability, Ch.8, exercise 6, pp34 35, where is credited to David Masle.

3 Solutios to I-Class Problems Week 5, Mo 3 positive. So / log i/ log i for 2 i. Therefore, 2 = log log i= i log i i log i i= /2 /2 log i= /2 + /2 2 log 2. 4 log (b) Show that the sequece X, X 2,... satisfies the Weak Law of Large Numbers, i.e., prove that for ay ɛ > 0 S ɛ = 0. Solutio. Pr S ɛ = Pr S 0 ɛ [ S ] Var ɛ 2 (Chebychev Boud) Var [S ] = 2 ɛ 2 = Θ( ), (by (2)) ɛ 2 log which goes to zero as goes to. We ow show that the sequece X, X 2,... does ot satisfy the Strog Law of Large Numbers. (c) (The first Borel-Catelli lemma.) Let A, A 2,... be ay ifiite sequece of mutually idepedet evets such that Pr {A i } =. (3) i=

4 Solutios to I-Class Problems Week 5, Mo 4 Prove that Pr {ifiitely may A i occur} =. Hit: We kow that the probability that o A i with i r occurs is e E[Tr, ] (4) where T r, ::= i=r I A i is the umber of evets A i with i r that occur. What happes as? Solutio. Let K r be the evet that o A i with i r occurs. Also, let K r, be the evet that o A i with i r occurs. Fially, let K be the evet that oly fiitely may A i s occur. We must prove that Pr {K} = 0. We begi by computig lim e E[Tr,] : E [T r, ] = E [I Ai ] (liearity of expectatio) i=r = Pr {A i } (expectatio of idicator variable) i=r If we take the the limit as we have: lim E [T r, ] = {A i } i=r = (by (3)). Sice e x 0 as x, we coclude that: lim e E[Tr,] = 0 Note that K r K r, for ay r,. Hece Pr {K r } Pr {K r, } e E[Tr,]. We have just proved that the right had side teds to zero as goes to. Sice the left had side does ot deped o, ad the iequality holds for all s.t. r, we coclude that Pr {K r } must be zero. Now ote that K = r K r, so by Boole s law, Pr {K} r Pr {K r }, ad sice we ve just proved that Pr {K r } = 0 for all r, it follows that Pr {K} = 0. Hece the probability that ifiitely may A i s occur is. (d) Show that i= Pr { X i i} diverges. Hit: /(xlogx) dx = log log x.

5 Solutios to I-Class Problems Week 5, Mo 5 Solutio. Pr { X i i} = /(i log i), so Pr { X i i} = 0 + i log i + dx 2 x log x = log log( + ) log log 2, i= ad this last term approaches ifiity ad approaches ifiity. (e) Coclude that Pr lim S = µ = 0. (5) ad hece that the Strog Law of Large Numbers completely fails for the sequece X, X 2,.... Hit: X S S =, so if lim S / = 0, the also lim X / = 0. Solutio. By parts (c) ad (d), the probability that X i i for ifiitely may i is. But if X i i for ifiitely may i, the by defiitio of the limit, lim X / 0. Hece, Pr lim X / 0 =, which meas But the hit implies that Pr lim X / = 0 = 0, (6) Pr lim S X = 0 Pr lim = 0. (7) Now (6) ad (7) immediately imply (5). A Appedix The probability desity fuctio (pdf) for a radom variable, R, is the fuctio f R : rage (R) [0, ] defied by: f R (x) ::= Pr {R = x}.

6 Solutios to I-Class Problems Week 5, Mo 6 Radom variables R, R 2,... are mutually idepedet iff Pr [R i = x i ] = Pr {R i = x i }, i for all x, x 2, R. They are k-wise idepedet iff {R i i J } are mutually idepedet for all subsets J N with J = k. Theorem (Weak Law of Large Numbers). Let S ::= i= X i, where X,..., X,... are pairwise idepedet variables with the same expectatio, µ, ad stadard deviatio, σ. For ay ɛ > 0, S µ ɛ = 0. Theorem (The Strog Law of Large Numbers). Let S ::= i= X i where X,..., X i,... are mutually idepedet, idetically distributed radom variables with fiite expectatio, µ. The Pr lim S = µ =. Defiitio. For ay radom variable, R, with fiite mea, µ R, ad deviatio, σ R, let R be the radom variable R is called the ormalized versio of R. R ::= R µ R. σ R Defiitio. The ormal desity fuctio is the fuctio ad the ormal distributio fuctio is its itegral η(x) = e x2 /2, 2π i y y N (y) = η(x)dx = e x2 /2 dx. 2π The fuctio η(x) defies the stadard Bell curve, cetered about the origi with height / 2π ad about two-thirds of its area withi uit distace of the origi. The ormal distributio fuctio N (y) approaches 0 as y. As y approaches zero from below, N (y) grows rapidly towards /2. The as y cotiues to icrease beyod zero, N (y) rapidly approaches. Theorem (Cetral Limit Theorem). Let S = i= X i where X,..., X i,... are mutually idepedet variables with the same mea, µ, ad deviatio, σ, ad let S be the ormalized versio of S. The for ay real umber β. {S β} = N (β) (8)