Cotiuous Radom Variables Math 394 1 (Almost bullet-proof) Defiitio of Expectatio Assume we have a sample space Ω, with a σ algebra of subsets F, ad a probability P, satisfyig our axioms. Defie a radom variable as a a fuctio X : Ω R, such that all subsets of Ω of the form {ω a < X(ω) b}, for ay real a b are evets (belog to F). Assume at first that the rage of X is bouded, say it is cotaied i the iterval A,B]. We work with X by approximatig it with a sequece of discrete radom variables X (), defied as {X () = } = { X < +1 } where A = x < x 1 <... < x = B is a partitio of our iterval with, for example, x j+1 x j = B A. We ca ow defie EX] = lim E X ()] = lim P X < +1 ] if the limit exists. We limit ourselves to absolute cotiuous radom variables, so that P X < +1 ] = ˆ xk+1 f X (u)du wheref X is a piecewise cotiuous o-egative fuctio, such that f X(x)dx = B A f X(x)dx = 1. It is ow straightforward to prove thatex] = xf (x)dx. Ideed xk+1 ˆ k k ˆ xk+1 ˆ B f(u)du A xf (x)dx k ˆ xk+1 ( x)f(x)dx x f(x)dx B A ˆ B If, f(x)dx = B A A repeatedly usig the triagle iequality. 1
1 (Almost bullet-proof) Defiitio of Expectatio 2 If the rage of X is ubouded, we proceed as i the defiitio of improper itegrals over the real lie, by cosiderig a icreasig sequece of itervals A,B ], with =1 A,B ] = R, ad defie EX] = ˆ B xf(x)dx = lim xf(x)dx A (1) if the limit exists i the sese of improper itegrals (for example by computig separatelylim B xf(x)dx, adlim A xf(x)dx ad defiig the sum as (1), as log they do t diverge both). Similarly, we will defie Eg(X)] = g(x)f X(x)dx if the limit exists, usig the same argumet. Note that most of the results about expectatios that we saw i previous chapters exted to the cotiuous case, thaks to its ature as a limit. Thus, for example, ] E a k X k +b = a k EX k ]+b ] Var a k X k +b = a 2 k VarX k]+2 1=k<j a k a j CovX k X j ] with momets, variace, covariace, ad so o defied as previously. Also, the proof of Markov s Iequality, ad Chebyshev s Iequality go through, as does the proof of the Weak Law of Large Numbers. Remark Note that we have as a cosequece that EX Y] = EX] EY], but VarX Y] = VarX]+VarY] 2CovX i X j ]. I other words, we ca reduce the variace of a sum oly by choosig the covariace with care. A famous applicatio of this priciple is i fiacial mathematics, where the variace of prices is iterpreted as volatility, ad a measure of risk. The suggestio is the to look for ivestmets that have egative correlatio, i order to reduce hteir combied risk.
2 Notable Cotiuous Distributios as Limits of Notable Discrete Oes 3 2 Notable Cotiuous Distributios as Limits of Notable Discrete Oes 2.1 Uiform Distributio Take a iterval, for example, 1]. Cosider the sequece of discrete uiform radom variables with distributios P X () = k ] = 1, k = 1,2,... Of course, ad ( E X ()) ] 2 = 1 E X ()] = 1 k = +1 2 k 2 = 1 2 (+1)(2+1) = (+1)(2+1) 3 6 6 2 Var X ()] = (+1)(2+1) (+1)2 = 2(+1)(2+1) 3(+1)2 = 6 2 4 2 12 2 (+1)( 1) = 2 1 12 2 12 2 The limit of this distributio as has clearly cotiuous desity of the form { 1 x 1 u(x) = elsewhere ad such a radom variable has EX] = E X 2] = ˆ 1 ˆ 1 xdx = 1 2 x 2 dx = 1 3 VarX] = 1 3 1 4 = 1 12 which are, as expected, the limits of the correspodig quatities for X ().
2 Notable Cotiuous Distributios as Limits of Notable Discrete Oes 4 2.2 From the Geometric to the Expoetial Distributio We saw how to obtai a Poisso distributio as a limit of biomials (the Law of Rare Evets ). I a sequece of Beroulli trials, cosider the time of first success : P S = ] = p(1 p) 1 ES] = p(1 p) 1 = 1 p =1 VarS] = 1 p p 2 as we saw whe itroducig this distributio. Also, P S > ] = k=+1 p(1 p) k 1 = p(1 p) (1 p) k = (1 p) As i the Law of Rare Evets, take a time spa,t], divide the time axis i itervals of legth 1, ad cosider a sequece of Beroulli trials with probability of success p = λ. As we kow, the limit of the umber of wis will have a Poisso distributio, with parameter λt. We ca determie the probability distributio of the first success (or arrival ) T from k= P T > t] = P N t = ] = e λt(λt)! = e λt This is the survival fuctio of a expoetial distributio, as ca be see immediately. Cosistet with this, we ca show that the limit of the discrete aalog, the geometric distributio, teds to the expoetial oe. Ideed, we would have that ( P S > t] = 1 λ ) t e λt
2 Notable Cotiuous Distributios as Limits of Notable Discrete Oes 5 We ca prove the same fact by ivokig a much more powerful theorem, that we metioed whe discussig the Momet Geeratig Fuctio ad its sibligs. The theorems says that if a sequece of momet geeratig fuctios coverges to a momet geeratig fuctio, the correspodig distributios coverge as well (for example, i the sese that the cumulative distributio fuctios coverge 1 ) Ideed, for the geometric distributio, M G (w) = E e wg] = e wk p(1 p) k 1 = e w p (e w (1 p)) k 1 = e w p 1 e w (1 p) = We have the, takig p = λ, that e w 1 e w +pe w ( w ) M G = e w λ 1 e w + λ e w ad sice 1 e w = w +o( ) 1 w, ad e 1, M G ( w ) λ λ w = M(w) Computig the stadard quatities for the expoetial distributio, we have (with repeated itegratio by parts, based o xe x dx = xe x + e x dx+c) EX] = λ xe λx dx = 1 λ E X 2] = λ (λx)e λx d(λx) = x 2 e λx dx = 2 λ 2 M X (w) = λ VarX] = 1 λ 2 e wx e λx dx = λ λ w (ote that the itegral coverges oly for w < λ), which are the limits of the correspodig quatities for geometric distributios, i the settig of the Law of Rare Evets. 1 To be precise, F (x) F(x) at every poit x where F is cotiuous. I practically all our examples, F will be cotiuous everywhere, so the caveat is ot relevat there.
3 From the Biomial (ad may others) to the Normal Distributio 6 3 From the Biomial (ad may others) to the Normal Distributio De Moivre proved by brute calculatio that a sequece of biomial distributios with parameters (,p) (ote that p does ot chage), as, looks more ad more like a Gaussia (Normal) distributio. I the more moder form, as stated by Laplace P a X p p(1 p) b ] Φ(b) Φ(a) 2 where Φ(x) = 2π 1 x e u 2 du is the cumulative distributio fuctio of the stadard ormal distributio, which caot be expressed i terms of elemetary fuctios. The origial proof is based o takig explicitly the limit of the biomial distributio, ad applyig Stirlig s Approximatio! + 1 2e 2π 1 as. De Moivre s result, i somewhat moderized otatio is that, for sufficietly large } 1 (k p)2 P X = k] exp { 2πp(1 p) 2p(1 p) A biomial radom variable ca be thought as a sum of idepedet Beroulli radom variables, ad, i fact, the covergece to a ormal distributio holds i a much more geeral settig, as was soo recogized by Laplace. The Cetral Limit Theorem states that, for a sequece of idepedet, idetically distributed, radom variables X, with fiite variace σ 2, ad mea µ 1 P X ] k µ x Φ(x) σ 2 This ca be read i a umber of ways, oe of which allows a sharper estimate for the discrepacy betwee 1 X k ad its expectatio µ, tha what provided by Chebyshev s Iequality (which, however, has broader applicability). This ca also be stated i this form: as, addig up idepedet idetically distributed radom variables, with mea µad variace σ 2, if their size is scaled by (the umber of terms i the sum), X k µ, a costat (this is the verbal statemet of the Law of Large Numbers). Equivaletly, 1 (X k µ), that is addig may small mea zero effects results i cacellatio. If, however, we scale by their differece from µ, Z, where Z is a ormal radom variable with mea ad variace X k µ σ 2. A cosequece is that addig may small (but ot too small) idepedet mea zero effects results i a ormal distributio, which is the basis for the usual theory of radom measuremet errors, as well for the Maxwell model of molecular kietics. Aother famous applicatio is i the suggestio to diversify ivestmet portfolios, by combiig may idepedet stocks, i relatively small quatities: by Chebyshev s iequality this should reduce their combied variace, aka volatility. While idepedece ca be slightly relaxed (there is a host of results geeralizig the CLT), it bears keepig its importace i mid: if the terms i the sum are sigificatly depedet, the theorem fails, ad igorig this fact ca (ad does) lead to serious misapplicatios of this basic result.