Section 8.2 Markov and Chebyshev Inequalities and the Weak Law of Large Numbers

Transcription

1 Sectio 8. Markov ad Chebyshev Iequalities ad the Weak Law of Large Numbers THEOREM (Markov s Iequality): Suppose that X is a radom variable takig oly o-egative values. The, for ay a > 0 we have X a} E[X] a roof: Cosider the radom variable Note that because X 0, X a} = if X a 0 else X a} X a Therefore, E[X] E [ ] X a} = X a} = } = X a} a Rearragig terms give s the result. EXAMLE: Let X be uiform(0, ). The for ay a > 0 Markov s iequality gives whereas It is easy to show that a (0, ) at a = with X a} = X a} a a dx = a, 0 < a < > a for ay a > 0. Moreover, f(a) = + a attais its miimum o a ( ) f = = 0... EXAMLE: A post office hadles, o average, 0,000 letters a day. What ca be said about the probability that it will hadle at least 5,000 letters tomorrow.

2 EXAMLE: A post office hadles, o average, 0,000 letters a day. What ca be said about the probability that it will hadle at least 5,000 letters tomorrow. Solutio: Let X be the umber of letters hadled tomorrow. The E(X) = 0, 000. Further, by Markov s iequality we get (X 5, 000) E(X) 0, 000 = 5, 000 5, 000 = 3 THEOREM (Chebyshev s Iequality): Let X be a radom variable with fiite mea µ ad variace σ, the for ay k > 0 we have X µ k} σ k

3 THEOREM (Chebyshev s Iequality): Let X be a radom variable with fiite mea µ ad variace σ, the for ay k > 0 we have X µ k} σ k roof: Note that (X µ) 0, so we may use Markov s iequality. Apply with a = k to fid that But (X µ) k if ad oly if X µ k, so (X µ) k } E[(X µ) ] k = σ k X µ k} σ k REMARK: Note that if k = aσ i Chebyshev s iequality, the ( X µ aσ) a This gives that probability of beig a deviates from the expected value drops like /a. Of course, we also have ( X µ < aσ) a EXAMLE: Let X be uiform(0, ). The for ay a > 0 Chebyshev s Iequality gives ( X ) < aσ a whereas ( X ) < aσ = ( aσ < X ) < aσ = = ( aσ + < X < aσ + ) aσ+/ aσ+/ dx = aσ = a = a 3, 0 < a 3 a Oe ca show that > 3 a for ay a > 0. Moreover, f(a) = a + attais its miimum o 3 a (0, ) at a = 6 = with ( ) 6 f = ( REMARK: Note that X ) < aσ = if a 3 ad as a. a EXAMLE: A post-office hadles 0,000 letters per day with a variace of,000 letters. What ca be said about the probability that this post office hadles betwee 8,000 ad,000 letters tomorrow? What about the probability that more tha 5,000 letters come i? 3

4 EXAMLE: A post-office hadles 0,000 letters per day with a variace of,000 letters. What ca be said about the probability that this post office hadles betwee 8,000 ad,000 letters tomorrow? What about the probability that more tha 5,000 letters come i? Solutio: We wat By Chebyshev s iequality we have Thus, (8, 000 < X <, 000) = (, 000 < X 0, 000 <, 000) = ( X 0, 000 <, 000) = ( X 0, 000, 000) ( X 0, 000, 000) σ (, 000) =, 000 = (8, 000 < X <, 000) = Now to boud the probability that more tha 5,000 letters come i. We have X 5, 000} = X 0, 000 5, 000} (X 0, 000 5, 000) (X 0, 000 5, 000)} = X 0, 000 5, 000}, 000 5, 000 = 500 = Recall, that the old boud (o iformatio about variace) was /3. EXAMLE: Let X be the outcome of a roll of a die. We kow that E[X] = /6 = 3.5 ad Var(X) = 35/ Thus, by Markov, Chebyshev gives 0.67 /6 = (X 6) E[X] 6 = /6 6 = / ( X or X 5) = /3 = ( X 3.5.5) 35/.5.96 THEOREM: If Var(X) = 0, the X = E[X]} =

5 THEOREM: If Var(X) = 0, the X = E[X]} = roof. We will use Chebyshev s iequality. Let E[X] = µ. For ay we have X µ } = 0 Let E = X µ }. Note that so this is a icreasig sequece of sets. Therefore, E E... E E = lim X µ } = lim X µ }} = } E = X µ} = Hece, X = µ} = X µ} = THEOREM (The Weak Law of Large Numbers): Let X, X,... be a sequece of idepedet ad idetically distributed radom variables, each havig the fiite mea E[X i ] = µ. The for ay ǫ > 0 we have } X X µ ǫ 0 as 5

6 THEOREM (The Weak Law of Large Numbers): Let X, X,... be a sequece of idepedet ad idetically distributed radom variables, each havig the fiite mea E[X i ] = µ. The for ay ǫ > 0 we have } X X µ ǫ 0 as roof: We will also assume that σ is fiite to make the proof easier. We kow that [ ] ( ) X X X X E = µ ad Var = σ Therefore, a direct applicatio of Chebyshev s iequality shows that } X X µ ǫ σ 0 as ǫ Statistics ad Sample Sizes Suppose that we wat to estimate the mea µ of a radom variable usig a fiite umber of idepedet samples from a populatio (IQ score, average icome, average surface water temperature, etc.). Major Questio: How may samples do we eed to be 95% sure our value is withi.0 of the correct value? Let µ be the mea ad σ be the variace ad let be the sample mea. We kow that E[X] = µ, By Chebyshev we have Now, we wat that for some X = X X ( ) X X Var(X) = Var = σ X µ ǫ} σ ǫ Thus, it is sufficiet to guaratee that X µ <.0}.95 = X µ.0}.05 σ (.0) =.05 = σ = 00, (.0) 000σ (.05) To be 95% sure we are withi. of the mea we eed to satisfy σ (.) =.05 = σ =, (.) 000σ (.05) 6

7 Sectio 8.3 The Cetral Limit Theorem EXAMLE: Cosider rollig a die 00 times (each X i is the output from oe roll) ad addig outcomes. We will get aroud 350, plus or mius some. We do this experimet 0,000 times ad plot the umber of times you get each outcome. The graph looks like a bell curve. EXAMLE: Go to a library ad go to the stacks. Each row of books is divided ito pieces. Let X i be the umber of books o piece i. The X i is the umber of books o a give row. Do this for all rows of the same legth. You will get a plot that looks like a bell curve. Recall, if X, X,... are idepedet ad idetically distributed radom variables with mea µ, the by the weak law of large umbers we have that [X X ] µ or [X X µ] 0 I a sese, this says that scalig by was too much. Why? Note that the radom variable X + X X µ has mea zero ad variace Therefore, has mea zero ad variace oe. Var(X X ) = Var(X) X X µ σ THEOREM (Cetral Limit Theorem): Let X, X,... be idepedet ad idetically distributed radom variables with mea µ ad variace σ. The the distributio of X X µ σ teds to the stadard ormal as. That is, for < a < we have } X X µ σ a a e x / dx as π To prove the theorem, we eed the followig: LEMMA: Let Z, Z,... be a sequece of radom variables with distributio fuctios F Z ad momet geeratig fuctios M Z, ; ad let Z be a radom variable with a distributio fuctio F Z ad a momet geeratig fuctio M Z. If M Z (t) M Z (t) for all t, the F Z (t) F Z (t) for all t at which F Z (t) is cotiuous. 7

8 REMARK: Note, that if Z is a stadard ormal, the M Z (t) = e t / which is cotiuous everywhere, so we ca igore that coditio i our case. Thus, we ow see that if M Z (t) e t / as for each t, the F Z (t) Φ(t) roof: Assume that µ = 0 ad σ =. Let M(t) deote the momet geeratig fuctio of the commo distributio. The momet geeratig fuctio of X i / is ( ) E[e tx i/ t ] = M Therefore, the momet geeratig fuctio of the sum [ ( )] t M e t / l [ ( )] X i t is M. We eed ([ M ( t )] ) t Let Note that ad uttig pieces together, we get Fially, we obtai L(t) = l M(t) L(0) = l M(0) = 0, L (0) = M (0) M(0) = µ = 0 L (0) = M (0)M(0) M (0) [M(0)] = E[X ] (E[X]) = l ([ M ( )] ) ( ) ( ) t t t = l M = L L(t/ ) L (t/ ) 3/ t lim = lim (by L Hopital s rule) = lim L (t/ )t / = lim L (t/ )t 3/ (/) 3/ (by L Hopital s rule) = lim L (t/ )t = t 8

9 So, we are doe if µ = 0 ad σ =. For the geeral case, just cosider X i = X i µ σ which has mea zero ad variace oe. The, } X X t Φ(t) = X X µ σ which is the geeral result. } t Φ(t) EXAMLE: A astroomer is measurig the distace to a star. Because of differet errors, each measuremet will ot be precisely correct, but merely a estimate. He will therefore make a series of measuremets ad use the average as his estimate of the distace. He believes his measuremets are idepedet ad idetically distributed with mea d ad variace. How may measuremets does he eed to make to be 95% sure his estimate is accurate to withi ±.5 light-years? Solutio: We let the observatios be deoted by X i. From the cetral limit theorem we kow that Z = X i d has approximately a ormal distributio. Therefore, for a fixed we have }.5 } X i d.5 =.5 Z.5 To be 95% certai we wat to fid so that ( ) Φ =.95 ( ) ( ) = Φ Φ ( ) = Φ ( ) or Φ =.975 Usig the chart we see that =.96 or = (7.8) = 6.7 Thus, he should make 6 observatios. It s tricky to kow how good this approximatio is. However, we could also use Chebyshev s iequality if we wat a sharp boud. Sice [ ] ( ) E X i = d ad Var X i = Chebyshev s iequality yields } X i d >.5 9 (.5) = 6

10 Therefore, to guaratee the above probability is less tha or equal to.05, we eed to solve 6 =.05 = = 6.05 = 30 That is, if he makes 30 observatios, he ca be 95% sure his value is withi ±.5 light-years of the true value. EXAMLE: Let X, X,... be idepedet ad idetically distributed radom variables with mea µ ad stadard deviatio σ. Set S = X +...+X. For a large, what is the approximate probability that S is betwee E[S ] kσ S ad E[S ] + kσ S (the probability of beig k deviates from the mea)? Solutio: We have E[S ] = E[X X ] = µ ad σ S = Var(S ) = Var(X X ) = σ Thus, lettig Z N(0, ), we get E[S ] kσ S S E[S ] + kσ S } = µ kσ S µ kσ } Compare: Chebyshev s iequality gives = kσ S µ kσ } = k S µ σ k} ( k Z k) = π k k e x / dx.686 k =.955 k = =.997 k = k = S µ < kσ } = S µ kσ } σ k σ = k 0 k =.75 k = =.8889 k = k = 0

11 Sectio 8. The Strog Law of Large Numbers THEOREM (The Strog Law of Large Numbers): Let X, X,... be a sequece of idepedet ad idetically distributed radom variables, each havig a fiite mea µ = E[X i ] ad fiite fourth momet E[X i ] = K. The, with probability oe, X X µ as Compare: By the Weak Law we have } X X µ ǫ 0 as Imagie you are rollig a die ad you are trackig X = X X The Weak Law says that for large, the probability that X is more tha ǫ away from µ goes to zero. However, we could still have X 00 µ > ǫ, X 000 µ > ǫ, X 0000 µ > ǫ, X µ > ǫ, etc., it s just that it happes very rarely. The Strog Law says that with probability oe for a give ǫ > 0, there is a N such that for all > N X µ ǫ roof: As we did i the proof of the cetral limit theorem, we begi by assumig that the mea of the radom variables is zero. That is E[X i ] = 0 Now we let ad cosider Expadig the quartic results i terms of the form S = X X E[S ] = E[(X X ) ] X i, X 3 i X j, X i X j, X i X j X k, X i X k X j X l However, we kow that they are idepedet ad have a mea of zero. Therefore, lots of the terms are zero E[X 3 i X j ] = E[X 3 i ]E[X j ] = 0 E[X i X jx k ] = E[X i ]E[X i]e[x j ] = 0 E[X i X k X j X l ] = E[X i ]E[X k ]E[X j ]E[X l ] = 0 Therefore, the oly terms that remai are those of the form X i, X i X j

12 Obviously, there are exactly terms of the form Xi. As for the other, we ote that for each pair of i ad j, ( ) = 6 ways to choose i two times from the four. Thus, each X i Xj will appear 6 times. Also, there are ( ) ways to choose objects from, so the expasio is ( ) E[S ] = E[X i ] + 6 E[Xi ]E[X j] = K + 3( )(E[Xi ]) () We have ad so Therefore, by () we get or Hece, Could E [ = S 0 Var(X i ) = E[X i ] (E[X i ]) ] (E[X i ]) E[X i ] = K E[S ] K + 3( )K = E [ ] S K + 3K 3 E [ ] S = = S = ( K + 3K ) < 3 with probability p > 0? If so, the the expected value would be ifiite, which we kow it is t. Therefore, with a probability of oe we have that S < But the, with a probability of oe, = which is precisely what we wated to prove. S 0 as or S 0 as I the case whe µ 0, we apply the precedig argumet to X i µ to show that with a probability of oe we have X i µ lim = 0 or, equivaletly, which proves the result. lim X i = µ