10 Moment generating functions

Transcription

1 10 MOMENT GENERATING FUNCTIONS Momet geeratig fuctios If X is a radom variable, the its momet geeratig fuctio is { φ(t) = φ X (t) = E(e tx x ) = etx P(X = x) i discrete case, etx f X (x)dx i cotiuous case. Example Assume that X is Expoetial(1) radom variable, that is, { e x x > 0, f X (x) = 0 x 0. The, φ(t) = 0 e tx e x dx = 1 1 t, oly whe t < 1. Otherwise the itegral diverges ad the momet geeratig fuctio does ot exist. Have i mid that momet geeratig fuctio is oly meaigful whe the itegral (or the sum) coverges. Here is where the ame comes from. Writig its Taylor expasio i place of e tx ad exchagig the sum ad the itegral (which ca be doe i may cases) E(e tx ) = E[1 + tx t2 X ! t3 X ] = 1 + te(x) t2 E(X 2 ) + 1 3! t3 E(X 3 ) +... The expectatio of the k-th power of X, m k = E(X k ), is called the k-th momet of x. I combiatorial laguage, the, φ(t) is the expoetial geeratig fuctio of the sequece m k. Note also that d dt E(etX ) t=0 = EX, d 2 dt 2E(etX ) t=0 = EX 2, which lets you compute the expectatio ad variace of a radom variable oce you kow its momet geeratig fuctio. Example Compute the momet geeratig fuctio for a Poisso(λ) radom variable.

2 10 MOMENT GENERATING FUNCTIONS 120 By defiitio, φ(t) = =0 e t λ! e λ = e λ (e t λ)! =0 = e λ+λet = e λ(et 1). Example Compute the momet geeratig fuctio for a stadard Normal radom variable. By defiitio, φ X (t) = = 1 2π 1 e 1 2 t2 2π = e 1 2 t2, e tx e x2 /2 dx e 1 2 (x t)2 dx where from the first to the secod lie we have used, i the expoet, tx 1 2 x2 = 1 2 ( 2tx + x2 ) = 1 2 ((x t)2 t 2 ). Lemma If X 1,X 2,...,X are idepedet ad S = X X, the φ S (t) = φ X1 (t)... φ X (t). If X i is idetically distributed as X, the φ S (t) = (φ X (t)). Proof. This follows from multiplicativity of expectatio for idepedet radom variables: E[e ts ] = E[e tx1 e tx2... e tx ] = E[e tx 1 ] E[e tx 2 ]... E[e tx ]. Example Compute the momet geeratig fuctios of a Biomial(, p) radom variable. Here we have S = k=1 I k where I k are idepedet ad I k = I {success o kth trial}, so that φ S (t) = (e t p + 1 p).

3 10 MOMENT GENERATING FUNCTIONS 121 Why are momet geeratig fuctios useful? Oe reaso is the computatio of large deviatios. Let S = X 1 + +X, where X i are idepedet ad idetically distributed as X, with expectatio EX = µ ad momet geeratig fuctio φ. At issue is the probability that S is far away from its expectatio µ, more precisely P(S > a), where a > µ. We ca of course use Chebyshev s iequality to get a boud of order 1. But it turs out that this probability teds to be much smaller. Theorem Large deviatio boud. Assume that φ(t) is fiite for some t > 0. For ay a > µ, where P(S a) exp( I(a)), I(a) = sup{at log φ(t) : t > 0} > 0. Proof. For ay t > 0, usig the Markov s iequality, P(S a) = P(e ts ta 1) E[e ts ta ] = e ta φ(t) = exp ( (at log φ(t))). Note that t > 0 is arbitrary, so we ca optimize over t to get what the theorem claims. We eed to show that ψ(a) > 0 whe a > µ. For this, ote that Φ(t) = at log φ(t) satisfies Φ(0) = 0 ad, assumig that oe ca differetiate uder the itegral sig (which oe ca i this case but provig this requires a bit of abstract aalysis beyod our scope), ad the Φ (t) = a φ (t) φ(t) = a E(XetX ), φ(t) Φ (0) = a µ > 0, so that Φ(t) > 0 for some small eough positive t. Example Roll a fair die times ad let S be the sum of the umbers you roll. Estimate the probability that S exceeds its expectatio by at least, for = 100 ad = We fit this ito the above theorem: observe that µ = 3.5 ad so ES = 3.5, ad that we eed to fid a upper boud o P(S 4.5), i.e., a = 4.5. Moreover φ(t) = i=1 e it = et (e 6t 1) 6(e t 1). ad we eed to compute I(4.5), which by defiitio is the maximum, over t > 0, of the fuctio whose graph is i the figure below. 4.5t log φ(t),

4 10 MOMENT GENERATING FUNCTIONS t It would be ice if we could solve this problem by calculus, but ufortuately we caot (which is very commo i such problems), so we resort to umerical calculatios. The maximum is at t ad as a result I(4.5) is a little larger tha This gives the upper boud P(S 4.5) e 0.178, which is about 0.17 for = 10, for = 100, ad for = The boud for the same probability, obtaied by Chebyshev s iequality, is much much too large for large. Aother reaso why momet geeratig fuctios are useful is that they characterize the distributio, ad covergece of distributios. We will state the followig theorem without proof. Theorem Assume that the momet geeratig fuctios for radom variables X, Y, ad X are fiite for all t. 1. If φ X (t) = φ Y (t) for all t, the P(X x) = P(Y x) for all x. 2. If φ X (t) φ X (t) for all t, ad P(X x) is cotiuous i x, the P(X x) P(X x) for all x. Example Show that the sum of idepedet Poisso radom variables is Poisso. Here is the situatio, the. We have idepedet radom variables X 1,...,X, such that: X 1 is Poisso(λ 1 ), φ X1 (t) = e λ 1(e t 1), X 2 is Poisso(λ 2 ), φ X2 (t) = e λ 2(e t 1),. X is Poisso(λ ), φ X (t) = e λ(et 1).

5 10 MOMENT GENERATING FUNCTIONS 123 The φ X X (t) = e (λ λ )(e t 1) ad so X X is Poisso(λ λ ). Very similarly, oe could also prove that the sum of idepedet Normal radom variables is Normal. We will ow reformulate ad prove the Cetral Limit Theorem i a special case whe momet geeratig fuctio is fiite. This assumptio is ot eeded, ad you should apply it as we did i the previous chapter. Theorem Assume that X is a radom variable with EX = µ ad Var(X) = σ 2, ad assume that φ X (t) is fiite for all t. Let S = X X, where X 1,...,X are i. i. d., ad distrubuted as X. Let T = S µ σ. The, for every x, P(T x) P(Z x), as, where Z is a stadard Normal radom variable. Proof. Let Y = X µ σ Var(Y i ) = 1, ad ad Y i = X i µ σ. The Y i are idepedet, distributed as Y, E(Y i ) = 0, T = Y Y. To fiish the proof, we show that φ T (t) φ Z (t) = exp(t 2 /2) as : φ T (t) = E [ e t T] = E = E = E = = ] [e t Y t Y [e t Y 1 ] [e t Y ] ] E [e t Y ( 1 + t EY + 1 t 2 2 E(Y 2 ) ( t ( 1 + t2 2 e t2 2. ) 1 t 3 t 3 3/2 E(Y 3 ) /2 E(Y 3 ) +... ) )

6 10 MOMENT GENERATING FUNCTIONS 124 Problems 1. The player pulls three cards at radom from a full deck, ad collects as may dollars as the umber of red cards amog the three. Assume 10 people each play this game oce, ad let X be the umber of their combied wiigs. Compute the momet geeratig fuctio of X. 2. Compute the momet geeratig fuctio of a uiform radom variable o [0,1]. 3. This exercise was i fact the origial motivatio for the study of large deviatios, by the Swedish probabilist Harald Cramèr, who was workig as a isurace compay cosultat i 1930 s. Assume that the isurace compay receives a steady stream of paymets, amoutig to (a determiistic umber) λ per day. Also every day, they receive a certai amout i claims; assume this amout is Normal with expectatio µ ad variace σ 2. Assume also day-to-day idepedece of the claims. The regulators require that withi a period of days, the compay must be able to cover its claims by the paymets received i the same period, or else. Itimidated by the fierce regulators, the compay wats to fail to satisfy the regulators with probability less tha some small umber ǫ. The parameters, µ, σ ad ǫ are fixed, but λ is somethig the compay cotrols. Determie λ. 4. Assume that S is Biomial(, p). For every a > p, determie by calculus the large deviatio boud for P(S a). 5. Usig the cetral limit theorem for a sum of Poisso radom variables, compute i lim e i!. i=0 Solutios to problems 1. Compute the momet geeratig fuctio for a sigle game, the raise it to the 10th power: ( (( ) ( )( ) ( )( ) ( ) ) ) φ(t) = ( 52 ) + e t + e 2t + e 3t Aswer: φ(t) = 1 0 etx dx = 1 t (et 1). 3. By the assumptio, a claim Y is Normal N(µ,σ 2 ) ad so X = (Y µ)/σ is stadard ormal. Note that the Y = σx + µ. The combied amout of claims thus is σ(x X ) + µ,

7 10 MOMENT GENERATING FUNCTIONS 125 where X i are i. i. d. stadard Normal, so we eed to boud P(X X λ µ σ ) e I. As log φ(t) = 1 2 t2, we eed to maximize, over t > 0, ad the maximum equals Fially we solve the equatio to get λ µ σ t 1 2 t2, I = 1 ( ) λ µ 2. 2 σ e I = ǫ, 2log ǫ λ = µ + σ. 4. After a computatio, the aswer you should get is I(a) = alog a p + (1 a)log 1 a 1 p. 5. Let S be the sum of i. i. d. Poisso(1) radom variables. Thus S is Poisso(), ad ES =. By the cetral limit theorem P(S ) 1 2, but P(S ) is exactly the expressio i questio. So the aswer is 1 2.