Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula derived from the Poisson Distribution

Transcription

1 Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebrasa-Lincoln Lincoln, NE Voice: Fax: Topics in Probability Theory and Stochastic Processes Steven R. Dunbar Stirling s Formula derived from the Poisson Distribution Rating Mathematicians Only: prolonged scenes of intense rigor. 1

2 Section Starter Question What is the Poisson distribution? What ind of circumstances does a Poisson distribution describe? Key Concepts 1. Stirling s Formula, also called Stirling s Approximation, is the asymptotic relation n! n n+1/2 e n. 2. The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials. 3. The Poisson distribution with parameter λ is the discrete probability distribution defined on the non-negative integers 0, 1, 2,... with probability mass P [X = ] = p(; λ) = λ! e λ. parameter λ is Vocabulary 1. Stirling s Formula, also called Stirling s Approximation, is the asymptotic relation n! n n+1/2 e n. 2. The Poisson distribution with parameter λ is the discrete probability distribution defined on the non-negative integers 0, 1, 2,... with probability mass P [X = ] = p(; λ) = λ! e λ. 2

3 Mathematical Ideas Stirling s Formula Stirling s Formula, also called Stirling s Approximation, is the asymptotic relation n! n n+1/2 e n. The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials. Another attractive form of Stirling s Formula is n! ( n ) n n. e An improved inequality version of Stirling s Formula is n n+1/2 e n+1/(12n+1) < n! < n n+1/2 e n+1/(12n). See Stirling s Formula in MathWorld.com. Intuitive Probabilistic Proof This intuitive argument is adapted from [6, page ] and also from the short article by Hu, [4]. Let X 1, X 2,... be independent Poisson random variables each having mean 1. Let S n = n j=1 X j and then note that the mean and the variance of S n are equal to n. P [S n = n] = P [n 1 < S n n] = P [ 1/ n < (S n n)/ n 0 ] 0 1/ n 1 1 n 1 e x2 /2 dx 3

4 On the other hand S n is Poisson with mean n, and so P [S n = n] = e n n n n! Equating the two expressions for P [S n = n], and rearranging obtain n! n n+1/2 e n. This proof relies on having a version of the Central Limit Theorem that has not been proved using Stirling s Formula! Such a version of the Central Limit Theorem itself is pretty advanced. It also relies on several other advanced facts, such as the distribution of Poisson random variables, the fact that the sum of independent Poisson random variables is again Poisson and the fact that the variance of the sum of independent random variables is the sum of the variances. On the other hand, it is short and simple! Rigorous Derivation of Stirling s Formula The characteristic function with variable θ R of the Poisson distribution p(; λ) = λ! e λ with parameter λ is ˆp(θ; λ) = p(; λ)e iθ = e λ(eiθ 1). =0 For further properties see Breiman, [1, page 170 ff.], especially Definition 8.26 and Proposition 8.27, or Chung, [2, page 142 ff.], especially item 6 on page 147. The characteristic function is a -periodic function with a series definition which converges uniformly on R, meaning we can integrate the series term-by-term on [ π, π] to obtain p(; λ) = 1 π π ˆp(θ, λ)e iθ dθ (1) which is valid for λ > 0 and = 0, 1, 2, 3,.... This is the Fourier inversion formula for the characteristic function. See also Breiman, [1], Theorem 8.39, page 178; Chung, [2], Theorem 6.2.3; or Feller, [3], page 509. In equation (1) set λ =, and define I by I =! e = 1 π π e (eiθ 1 iθ) dθ = 0, 1, 2, 3,... 4

5 Compare this representation to the Gaussian integral with variance defined as J = 1 = 1 e θ2 /2 dθ. The general plan is to show that I J, so that then we can assert that! e 1 which can be rearranged to Stirling s Formula. The detailed plan is to show that this approximation can be expressed as an asymptotic limit. Brea I into pieces to mae the following definitions and I = 1 θ 1 e (eiθ 1 iθ) dθ + 1 1< θ π e (eiθ 1 iθ) dθ = I (1) + I (2) J = 1 e θ2 /2 dθ + 1 e θ2 /2 dθ = J (1) + J (2) θ 1 1< θ Lemma 1. The complex exponential has the following properties and estimates: 1. For A C 2. For θ R 3. For θ R 4. For A, B C e A = e RA, (2) e iθ 1 iθ θ 2 /2, (3) e iθ 1 iθ + θ 2 /2 θ 3 /3!, (4) e A e B A B e max[ra,rb]. (5) Proof. Left as exercises. 5

6 Use the triangle inequality for integrals and equation (2) to bound I (2) as and J (2) J (2) I (2) 1 1 1< θ π 1< θ π e (cos(1) 1) 1 e (cos(1) 1) e (eiθ 1 iθ) dθ e cos(θ) 1 dθ dθ 1< θ π Use a technique similar to the proof of Marov s inequality to estimate as J (2) = 1 e θ2 /2 dθ 1 θ e θ2 /2 dθ = 1 2 θ >1 θ >1 e /2. Both I (2) and J (2) tend to zero at an exponential rate. Now the effort is to estimate the closeness of I (1) 1 and J (1) I (1) J (1) = For θ 1, use inequality (4) to derive ( ) e (eiθ 1 iθ) e θ2 /2 dθ. 1. Write cos(θ) 1 + θ 2 /2 cos(θ) + i sin(θ) 1 iθ + θ 2 /2 e iθ 1 iθ + θ 2 /2 θ3 6 θ2 6. (6) Therefore, cos(θ) 1 θ 2 /3. Now put these together using inequalities (6) and (4) I (1) J (1) e (eiθ 1 iθ) e θ2 /2 θ 3 dθ 3! e θ2 dθ 1 Change variables with φ = θ to obtain (the derivation is left as an exercise) 1 1 θ 3 3! e θ2 /3 dθ = φ 3 e φ2 /3 dφ = 3 2 e /3 3e /

7 Then putting all these together I J 0. Recalling the definitions of I and J, this is the same as!e 1 0! as. This is equivalent to Stirling s Formula lim e! = 1. An alternate proof using the Lebesgue Dominated Convergence Theorem Start with the definition of I I =! e = 1 π π e (eiθ 1 iθ) dθ. Mae the change of variables y = θ with dy = dθ to obtain I = e = 1 π e (eiy/ 1 iy ) dy.! π Consider the integrand e (eiy/ 1 iy ). The exponent converges pointwise (e iy/ 1 iy/ ) y 2 /2 as by using equation (4), so the integrand converges pointwise to e (eiy/ 1 iy ) e y2 /2. The integrand is bounded pointwise by e (eiy/ 1 iy ) = e (cos(y/ ) 1) using equation (2). Using the half-angle identity, this can be written as e (cos(y/ ) 1) = e 2 sin2 (y/(2 )). Finally, e 2 sin2 (y/(2 )) e 2y2 /π 2 7

8 if and only if 2 sin 2 (y/(2 )) 2y 2 /π 2 or sin 2 (y/(2 )) (y/ ) 2 1 π 2 on the domain of integration [ π, π ]. But the function sin2 (y/(2 )) (y/ has ) 2 a limit of 1/4 as y 0, is symmetric around y = 0, and is decreasing on (0, π ). The minimum value of the function occurs at π and is 1/π 2. Then using the Lebesgue Dominated Convergence Theorem, [2, page 42] or [1, page 33, Theorem 2.44] π π e (eiθ 1 iθ) dθ e y2 /2 dy =. Putting this together! e! as. This is equivalent to Stirling s Formula lim e! = 1. Discussion These proofs establishes the Stirling s Formula asymptotic limit fairly easily, but are not enough to show the inequality n n+1/2 e n+1/(12n+1) < n! < n n+1/2 e n+1/(12n). In order to establish the inequality requires bounds on the rate of approach of I = 1 π e (eiθ 1 iθ) dθ π to J = 1 = 1 e θ2 /2 dθ. Such an estimate requires bounds on the rate of approach of e (eiy/ 1 iy ) e y2 /2 which is possible with careful estimation. 8

9 Sources The heuristic proof using the Central Limit Theorem is adapted from Ross [6, pages ], which in turn is based on Hu [4]. The rigorous proof is adapted from the short article by Pinsy [5]. Problems to Wor for Understanding 1. Show that for A C, e A = e RA 2. Show that for θ R, e iθ 1 iθ θ 2 /2 3. Show that for θ R, e iθ 1 iθ + θ 2 /2 θ 3 /3! 4. Use standard theorems from calculus (either the Fundamental Theorem of Calculus or the Mean Value Theorem) applied to the function f(t) = e ta+(1 t)b to show that for A, B C, e A e B A B e max[ra,rb] 5. Show that 1 φ 3 e φ2 /3 dφ = e /3 3e / Derive inequalities to estimate the size of the difference I J = 1 π π e (eiθ 1 iθ) dθ 1 = 1 e θ2 /2 dθ. Use these inequalities to derive inequalities for! refining Stirling s asymptotic limit formula to an inequality. 9

10 Reading Suggestion: References [1] Leo Breiman. Probability. Addison Wesley, [2] Kai Lai Chung. A Course in Probability Theory. Academic Press, [3] William Feller. A Introduction to Probability Theory and It Applications, Volume II, Second Edition, volume II. John Wiley and Sons, second edition edition, [4] T.-C. Hu. A statistical method of approach to Stirling s formula. American Statistician, 42: , [5] Mar A. Pinsy. Stirling s formula via the Poisson distribution. American Mathematical Monthly, 114(3): , March [6] Sheldon M. Ross. Introduction to Probability Models. Elsevier, 6th edition edition, Outside Readings and Lins: I chec all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I mae every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your ris. I have checed the lins to external sites for usefulness. Lins to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don t guarantee that the lins are active at all times. Use the lins here with the same caution as 10

11 you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer. Information on this website is subject to change without notice. Steve Dunbar s Home Page, to Steve Dunbar, sdunbar1 at unl dot edu Last modified: Processed from L A TEX source on April 21,