The Central Limit Theorem

Transcription

1 The Cenral Limi Theorem November Convergence in disribuion X n D X is defined o by lim Eh(X n) = Eh(X). n or every bounded coninuous funcion h : R R. However i is no necessary o verify his for each choice of h. so-called convergence deermining family of funcions. We can limi ourselves o a smaller For random variables aking values in he naural numbers {h z (x) = z x ; z < 1} is convergence deermining. In his case we are look a convergence of he probabiliy generaing funcion. For real valued random variables {h (x) = exp x; h < < h} is convergence deermining provided he necessary expeced values exis. Noe ha exp x is no bounded and so we need o make an addiional argumen o include hese funcion. In his case we are looking a convergence of he momen generaing funcion. Example 1. For he binomial disribuion wih parameers n and p he probabiliy generaing funcion is ρ Xn (z) = ((1 p) + pz) n = (1 p(1 z)) n If we ake he success probabiliy p = λ/n o depend on n hen ( ρ Xn (z) = ((1 p) + pz) n = 1 λ ) n n (1 z) exp λ(1 z) = ρ X (z) he probabiliy generaing funcion for a Poisson random variable X wih parameer λ. Thus we have ha he given binomial random variables converges in disribuion o a Poisson random variable. To use his assume ha n is large bu λ = np is moderae he binomial random variable is well approximaed by a Poisson random. In paricular Eh(X n ) Eh(X) for any bounded coninuous h 1 Cenral Limi Theorem If we look a disribuions for he sum T n = X 1 + X X n wha do we see. Le s look firs o he simples case X i Bernoulli random variables. This is looking like he bell curve. To make he comparisons fair le look a sandardized versions of he random variables wih mean µ and variance σ 2 1

2 y y x x Figure 1: a. Successes in 100 Bernoulli rials wih p = and 0.8. b. Successes in Bernoulli rials wih p = 1/2 and n = and 80. Z n = T n nµ σ n and look a he densiy of he sum of sandardized exponenial random variables. Again we see he densiies approaching ha of a bell curve. The classical cenral limi heorem saes ha if {X i ; i 1} and independen and idenically disribued wih common mean µ and common variance σ 2 hen Z n as defined by equaion (??) converges o Z a sandard normal random variable. In erms of he cumulaive disribuion funcion lim P {Z n z} = 1 z e x2 /2 dx = Φ(z) n 2π where Φ is he cumulaive disribuion funcion of he sandard normal. We will prove his in he case ha he X i have a momen generaing funcion M X () for he inerval ( h h) by showing ha lim n M Z n () = exp 2 2 or equivalenly show ha he cumulan generaing funcions K Zn () = log M Zn () saisfy (1) Wrie lim n K Z n () = 2 2 Y i = X i µ σ 2

3 f Z (z) z Figure 2: Densiy of he sandardized version of he sum of n independen exponenial random variables for n = and 32. hen Z n = 1 n For M Y () he momen generaing funcion for he Y i and M Tn () he momen generaing funcion for T n ( n ) n ( ) ( ( )) n M Zn() = E exp(z n ) = E exp Y i = E exp Y i = M Y n n n n Y i. and ( ) ( ) K Zn () = n log M Y = nk Y. n n Recall ha for he cumulan generaing funcion K Y Finally from wo applicaions of L Hôpial s rule lim K Z n () = lim nk Y n n K Y (0) = EY 1 = 0 K Y (0) = Var(Y ) = 1. ( ) K Y (ɛ) K Y = lim n ɛ 0 ɛ 2 = lim (ɛ) 2 K = lim ɛ 0 2ɛ ɛ 0 2 Y (ɛ) Example 2. For Bernoulli rials µ = p and σ 2 = p(1 p). Thus for large enough n Z n = T n np np(1 p). = 2 K Y (0) =

4 has approximaely he disribuion of a sandard normal random variable. For 100 osses of a fair coin and So Z n = T n 50 5 {T n 40} = {Z n 2} P {T n 40} P {Z 2} = Example 3. For an exponenial sample wih mean 1. Then he sandard deviaion is also 1 and for 64 observaions Z n = T n 64 8 {T n 78} = {Z n 1.75} So 2 Slusky s Theorem P {T n 78} P {Z 1.75} = Some useful exensions of he cenral limi heorem are based on Slusky s heorem. Theorem 4. Le X n D X and Y n P a a consan as n. Then 1. Y n X n D ax and 2. X n + Y n D X + a. For example by he law of large numbers he sample variance S 2 n a.s. σ 2 he disribuion variance as n. Thus S n σ a.s. 1. Thus i also converges in probabiliy. So by Slusky s heorem he -saisic a sandard normal as n T n nµ S n n = S n σ T n nµ σ n = S n σ Z n D 1 Z 4

5 3 Dela Mehod For a random sample {X n 1} wih common mean µ and common variance σ 2 we can wrie he cenral limi heorem using he sample mean. n( Xn µ) D σz where Z is a sandard normal. To generalize his assume ha {Y n 1} is a sequence of random variables saisfying n(yn θ) D σz for some value θ Then he dela mehod saes ha if a funcion g has a coninuous derivaive and g (θ) 0 hen n(g(yn ) g(θ)) D σg (θ) Z where Z is also a sandard normal. To prove his expand g is a Taylor s series abou he value θ g(y n ) = g(θ) + g ( θ)(y n θ) or n(g(yn ) g(θ)) = g ( θ) n(y n θ). where θ lies beween Y n and θ. Noe ha since Y n P θ implies θ P θ and g (θ) is coninuous g ( θ) P g (θ). and he heorem follows from applying Slusky s heorem. Example 5. For Bernoulli rials wrie X = ˆp hen n(ˆp p) D p(1 p)z. If we could find g so ha hen g (p) = 1 p(1 p) n(g(ˆp) g(p)) D Z. Such a choice which here is g(p) = 2 arcsin( p) is called a variance sabilizing ransformaion. 5