Steve R. Dubar Departmet o Mathematics 203 Avery Hall Uiversity o Nebrasa-Licol Licol, NE 68588-0130 http://www.math.ul.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics i Probability Theory ad Stochastic Processes Steve R. Dubar The Wea Law o Large Numbers Ratig Mathematicias Oly: prologed scees o itese rigor. 1
Questio o the Day Cosider a air (p = 1/2 = q) coi tossig game carried out or 1000 tosses. Explai i a setece what the law o averages says about the outcomes o this game. Be as precise as possible. Key Cocepts 1. Marov s Iequality: Let X be a radom variable taig oly oegative values. The or each a > 0 P [X > a] E [X] /a; 2. Chebyshev s Iequality: Let X be a radom variable. The or a > 0 P [ X E [X] a] Var [X] a 2 3. Wea Law o Large Numbers: For ɛ > 0 [ ] S P p > ɛ 0 as ad the covergece is uiorm i p. 4. Let be a real uctio that is deied ad cotiuous o the iterval [0, 1]. The ( ) ( ) sup (x) x (1 x) 0 as. =0 2
Vocabulary 1. The Wea Law o Large Numbers says that or ɛ > 0 [ ] S P p > ɛ 0 as ad the covergece is uiorm i p. 2. The polyomials B, (t) = are called the Berstei polyomials. ( ) x (1 x) Mathematical Ideas Proo o the Wea Law Usig Chebyshev s Iequality Propositio 1 (Marov s Iequality). Let X be a radom variable taig oly o-egative values. The or each a > 0 Proo. P [X a] E [X] /a. P [X a] = E [I X a ] = dp [] X a x dp [] a 1 a E [X] 3
Propositio 2 (Chebyshev s Iequality). Let X be a radom variable. The or a > 0 Var [X] P [ X E [X] a]. a 2 Proo. This immediately ollows rom Marov s iequality applied to the oegative radom variable (X E [X]) 2. Theorem 3 (Wea Law o Large Numbers). For ɛ > 0 [ ] S P p > ɛ 0 as ad the covergece is uiorm i p. Remar. The otatio P [] idicates that we are cosiderig a amily o probability measures o the sample space Ω. The Wea Law establishes the covergece o the sequece o measures i a particular way. Proo. The variace o the radom variable S is p(1 p). Rewrite the probability as the equivalet evet: [ ] S P p > ɛ = P [ S p > ɛ]. By Chebyshev s iequality P [ S p > ɛ] Var [S ] (ɛ) 2 = Sice p(1 p) 1/4, the proo is complete. Remar. This iequality demostrates that [ ] S P p > ɛ = O(1/) uiormly i p. p(1 p) ɛ 2 1. Remar. Jacob Beroulli origially proved the Wea Law o Large Numbers i 1713 or the special case whe the X i are biomial radom variables. Beroulli had to create a igeious proo to establish the result, sice Chebyshev s iequality was ot ow at the time. The theorem the 4
became ow as Beroulli s Theorem. Simeo Poisso proved a geeralizatio o Beroulli s biomial Wea Law ad irst called it the Law o Large Numbers. I 1929 the Russia mathematicia Alesadr Khichi proved the geeral orm o the Wea Law o Large Numbers preseted here. May other versios o the Wea Law are ow, with hypotheses that do ot require such striget requiremets as beig idetically distributed, ad havig iite variace. Remar. Aother proo o the Wea Law o Large Numbers usig momet geeratig uctios is i Mathematical Fiace/Cetral Limit Theorem Berstei s Proo o the Weierstrass Approximatio Theorem Theorem 4. Let be a real uctio deied ad cotiuous o the iterval [0, 1]. The ( ) ( ) sup (x) x (1 x) 0 as. =0 Proo. 1. Fix ɛ > 0. Sice cotiuous o the compact iterval [0, 1] it is uiormly cotiuous o [0, 1]. Thereore there is a η > 0 such that (x) (y) < ɛ i x y < η. 2. The expectatio E [(S /)] ca be expressed as a polyomial i p: [ E ( )] S = =0 ( ) P [S = ] = =0 ( ) ( ) p (1 p). 3. By the Wea Law o Large Numbers, or the give ɛ > 0, there is a 0 such that [ ] S P p > η < ɛ. 4. E [ ( ) S (p)] = =0 5 ( ( ) ) (p) P [S = ].
5. Apply the triagle iequality to the right had side ad express i terms o two summatios: ( ( ) ) (p) P [S = ] + p η ( ( ) ) + (p) P [S = ] p >η Note the secod applicatio o the triagle iequality o the secod summatio. 6. Now estimate the terms: ɛp [S = ] + p η p >η 2 sup (x) P [S = ] 7. Fially, do the additio over the idividual values o the probabilities over sigle values to re-write them as probabilities over evets: [ ] [ ] S = ɛp p η S + 2 sup (x) P p > η 8. Now apply the Wea Law to the secod term to see that: ( ) [ E S (p)] < ɛ + 2ɛ sup (x). This shows that E [ ( S ) (p) ] ca be made arbitrarily small, uiormly with respect to p, by picig suicietly large. Remar. The polyomials B, (t) = ( ) x (1 x) are called the Berstei polyomials. The Berstei polyomials have several useul properties: 6
1. B i, (t) = B i, (1 t) 2. B i, (t) 0 3. i=0 B i,(t) = 1 or 0 t 1. Corollary 1. A polyomial o degree uiormly approximatig the cotiuous uctio (x) o the iterval [a, b] is Sources =0 ( a + (b a) ) ( ) ( x a b a ) ( ) b x b a This sectio is adapted rom: Heads or Tails, by Emmauel Lesige, Studet Mathematical Library Volume 28, America Mathematical Society, Providece, 2005, Chapter 5, [3]. Problems to Wor or Uderstadig 1. Suppose X is a cotiuous radom variable with mea ad variace both equal to 20. What ca be said about P [0 X 40]? 2. Suppose X is a expoetially distributed radom variable with mea E [X] = 1. For x = 0.5, 1, ad 2, compare P [X x] with the Marov iequality boud. 3. Suppose X is a Beroulli radom variable with P [X = 1] = p ad P [X = 0] = 1 p = q. Compare P [X 1] with the Marov iequality boud. 4. Let X 1, X 2,..., X 10 be idepedet Poisso radom variables with mea 1. First use the Marov Iequality to get a boud o P [X 1 + + X 10 > 15]. Next id the exact probability that P [X 1 + + X 10 > 15] usig that the act that the sum o idepedet Poisso radom variables with parameters λ 1, λ 2 is agai Poisso with parameter λ 1 + λ 2. 7
5. Cosider a air (p = 1/2 = q) coi tossig game carried out or = 100 tosses. Calculate the exact value [ ] S P p > 1/10 ad compare it to the estimates i the proo o the Wea Law o Large Numbers. 6. Calculate the Berstei polyomial approximatio o si(πx) o degree 1, 2, ad 3 ad plot the graphs o si(πx) ad the approximatios. 7. Calculate the Berstei polyomial approximatio o cos(πx) o degree 1, 2, ad 3 ad plot the graphs o cos(πx) ad the approximatios. 8. Calculate the Berstei polyomial approximatio o exp(πx) o degree 1, 2, ad 3 ad plot the graphs o exp(πx) ad the approximatios. Readig Suggestio: Reereces [1] Leo Breima. Probability. SIAM, 1992. [2] William Feller. A Itroductio to Probability Theory ad Its Applicatios, Volume I, volume I. Joh Wiley ad Sos, third editio, 1973. QA 273 F3712. [3] Emmauel Lesige. Heads or Tails: A Itroductio to Limit Theorems i Probability, volume 28 o Studet Mathematical Library. America Mathematical Society, 2005. 8
Outside Readigs ad Lis: 1. Virtual Laboratories i Probability ad Statistics / Biomial 2. Weisstei, Eric W. Wea Law o Large Numbers. From MathWorld A Wolram Web Resource. Wea Law o Large Numbers. 3. Wiipedia, Wea Law o Large Numbers I chec all the iormatio o each page or correctess ad typographical errors. Nevertheless, some errors may occur ad I would be grateul i you would alert me to such errors. I mae every reasoable eort to preset curret ad accurate iormatio or public use, however I do ot guaratee the accuracy or timeliess o iormatio o this website. Your use o the iormatio rom this website is strictly volutary ad at your ris. I have checed the lis to exteral sites or useuless. Lis to exteral websites are provided as a coveiece. I do ot edorse, cotrol, moitor, or guaratee the iormatio cotaied i ay exteral website. I do t guaratee that the lis are active at all times. Use the lis here with the same cautio as you would all iormatio o the Iteret. This website relects the thoughts, iterests ad opiios o its author. They do ot explicitly represet oicial positios or policies o my employer. Iormatio o this website is subject to chage without otice. Steve Dubar s Home Page, http://www.math.ul.edu/~sdubar1 Email to Steve Dubar, sdubar1 at ul dot edu Last modiied: Processed rom L A TEX source o May 25, 2011 9