Lecture 9 1 Infinitely Divisible Distributions Given a triangular array of independent random variables, X ) 1 i n, we have previously given sufficient conditions for the law of S n := n X to converge to the normal distribution, resp. the Poisson distribution. A natural question is: what other types of distributions can be the limiting distribution of S n as n? To have a non-trivial answer to this question, it is important to assume that no single X is significant in its contribution to S n when n is large. More precisely, we assume that δ > 0, lim 1 i n P X δ) = 0, 1.1) which is known as the uniform infinitesimality condition. In other words, X ) 1 i n converge uniformly to 0 in probability as n. Under the assumption 1.1), we will give a complete characterization of possible limiting distributions for S n as n, which are known as infinitely divisible distributions. Similar to the relation between Poisson distribution and Poisson process, Gaussian distribution and Brownian motion, each infinitely divisible distribution can be realized as the marginal distribution of a Markov process X t ) t 0 with stationary independent increments, known as a Lévy process. Definition 1.1 [Infinitely Divisible Distributions] A probability distribution µ is called infinitely divisible if for every n N, there exist i.i.d. random variables X ) 1 i n with common distribution µ n, such that µ is the distribution of X n,1 + X n,2 + + X n,n. In other words, µ = µ n n, the n-fold convolution of µ n with itself for some distribution µ n. Exercise 1.2 Show that the normal distribution, the exponential distribution, the Poisson distribution, and the compound Poisson distribution are all infinitely divisible. Exercise 1.3 Use characteristic functions to show that if µ is infinitely divisible, and for each n N, µ n n = µ for some distribution µ n, then µ n 0 as n. Exercise 1.3 shows that if µ is infinitely divisible, then X ) 1 i n in Definition 1.1 satisfy 1.1), and hence µ is the limiting distribution of S n for a uniformly infinitesimal triangular array. The non-trivial part is the converse. Theorem 1.4 [Limits for Triangular Arrays] If X ) 1 i n is a triangular array of independent random variables satisfying the uniform infinitesimality condition 1.1), and the distribution of S n := n X converges weakly to a limit µ as n, then µ is infinitely divisible. We will prove Theorem 1.4 by first proving that we can replace X ) 1 i n by shifted compound Poisson distributed random variables Y ) 1 i n without affecting the limiting distribution µ. Since compound distributions are infinitely divisible, we will then show that the set of infinitely divisible distributions is closed under convolution and weak limits, which then 1
implies that µ is also infinitely divisible. This proof will also facilitate a complete characterization of infinitely divisible distributions. Given a triangular array X ) 1 i n, we will replace X by a shifted compound Poisson random variable Y defined as follows. Let a := E[X 1 { X 1}]. We then define N Y = a + j=1 X j), 1.2) j) where N is a Poisson random variable with mean 1, and X ) j N is a collection of i.i.d. random variables equally distributed with X := X a. Note that Y is a compound Poisson random variable shifted by the constant a. Ignoring the shift by a for the moment, the heuristic justification for the compound Poissonization is that, if X is close to 0 in probability, then its characteristic function φ t) is close to 1, and hence φ t) e 1 φt)), which is precisely the characteristic function of the compound Poissonization of X. Theorem 1.5 [Accompanying Laws] Let X ) 1 i n be a triangular array of independent random variables satisfying 1.1), and let Y ) 1 i n be defined as in 1.2). Then the distribution of S n := n X converges weakly to a limit µ as n if and only if the distribution of Ξ n := n Y converges weakly to the same limit µ. Proof. Let X := X a, and let Ỹ := N j=1 be the compound Poissonization of X as in 1.2). Let S n := n X and Ξ n := n Ỹ. Let φ n, ψ n, φ n, and ψ n denote respectively the characteristic functions of S n, Ξ n, S n and Ξ n. The theorem amounts to the statement that φ n and ψ n must converge to the same limit when either limit exists. Since which implies that it suffices to show that S n S n = Ξ n Ξ n = X j) a =: A n, φ n t) ψ n t) = e iant φ n t) ψ n t)) φ n t) ψ n t), lim φ n t) ψ n t) = 0 t R. 1.3) First we note that for any δ > 0, a E[ X 1 { X [δ,1]}]+e[ X 1 ) { X 0,δ)}] P X δ)+δ) δ 1 i n 1 i n 1 i n by the uniform infinitesimality condition 1.1). Therefore lim 1 i n a = 0, 1.4) which implies that X ) 1 i n also satisfies the uniform infinitesimality condition 1.1). Next we explain the advantage of shifting X to X = X a. By 1.4), we can choose n sufficiently large such that 1 i n a 1/4. For such n, we have ã := E[ X 1 { X 1} ] = E[X a )1 { X a 1}] = E[X 1 { X a 1}] a P[ X a 1] = E[X 1 { X a 1}] a + a P[ X a > 1] = E[X 1 { X a 1}] E[X 1 { X 1}] + a P[ X a > 1]. 2
Therefore ã E[ X 1 {X [ 1, 1+a ] [1,1+a ]}] + 1 4 P[ X a > 1] 3P X 3/4) 3P X 1/2). 1.5) We note that such a bound on the truncated mean ã = E[ X 1 { X 1} ] for X could not have been obtained for X. We will use this bound later. Let φ be the characteristic function of X. Then the characteristic function of Ỹ is ψ t) = e 1 φ t)). We then have log φ n t) log ψ n t) = log φ t) log ψ t) ) = log φ t) + 1 φ t) ) log φ t) + 1 φ t) C 1 φ t) 2 Taylor expansion) C max 1 i n 1 φ t) which can be justified provided we show that To establish 1.6), note that 1 φ t) 0, max t) 0, 1 i n 1.6) 1 φ t) <. 1.7) max 1 φ t) max E[ 1 e it X ] 1 i n 1 i n = max E [ 1 e it X [ 1 1 i n { X >δ}] + E 1 e it X ] ) 1 { X δ} max 2P X > δ) + E [ t X ]) 1 1 i n { X δ} Taylor expansion) 2 max 1 i n P X > δ) + tδ tδ uniform infinitesimality). Since δ > 0 can be chosen arbitrarily, this implies 1.6), in fact it implies that for any T > 0, t [ T,T ] max 1 φ t) 0. 1.8) 1 i n On the other hand, φ e it t) 1 = E[ X 1 ) [ E e 1 { X 1}] + it X 1 ) 1 { X >1}] e it E[ X 1 it X ) e 1{ X 1}] + it tã + E[ X 1 ) 1 { X >1}] C 1 t 2 E[ X 2 1 { X 1} ] + 3 t P X 1/2) + 2P X 1) C 1 t 2 E[ X 2 1 { X 1} ] + 3 t + 2)P X 1/2). Therefore to prove 1.7), it suffices to prove that n E[ X 2 1 { X 1} ] <, 1.9) n P[ X 1/2] <. 1.10) 3
To prove 1.9) 1.10), we need to use the assumption that either S n or Ξ n converges weakly to a limit, i.e., either φ n or ψ n converges to a limiting characteristic function. Assume first ψ n s) = e isan ψ n s) = e isan n 1 φ s)) fs) for a limiting characteristic function f. We leave it as an exercise to show that the convergence is in fact uniform on bounded intervals. Exercise 1.6 Let φ n be a sequence of characteristic functions converging pointwise to a characteristic function φ. Use Arzela-Ascoli theorem to show that φ n converges uniformly to φ on any bounded interval. Since f0) = 1 and f is continuous at 0, on some interval [ T, T ] with T > 0, f is uniformly bounded away from 0, and hence the same must be true for ψ n uniformly in large n N, i.e., s [ T,T ] Re1 φ s)) = s [ T,T ] E[1 cos s X )] < C T <. 1.11) Since 1 cos x cx 2 for some c > 0 uniformly in x 1, 1), we can choose s [ T, T ] with 0 < s < 1 such that for all n N sufficiently large, C T E[1 cos s X )1 { X 1} ] cs2 E[ X 1 2 { X 1} ], which implies 1.9). In fact, the same argument shows that for any A > 0, E[ X 1 2 { X A} ] <. 1.12) To prove 1.10), we repeat an argument used in the proof of Lévy s continuity theorem, where we control the tail probability in terms of the characteristic function. More precisely, we integrate the inequality in 1.11) over s [ T, T ] and then divide by 2T to obtain [ E 1 sint X ) ] T X < C T. Since inf x a 1 sin x x ) = c > 0 for any a > 0, for all n N sufficiently large, we have C T [ E 1 sint X ) ] T X [ E 1 sint X ) ) ] T X 1 { X 1/2} c which implies 1.10). In fact, the same argument shows that for any δ > 0, P X 1/2), P X δ) <. 1.13) Lastly we need to verify 1.9) and 1.10) under the assumption that φ n s) = e isan φ n s) = e isan n φ s) fs) 4
for some characteristic function f. By the same reasoning as in 1.11), there exists T > 0 such that lim inf inf φ n n s) 2 = lim inf inf φ s) 2 > 0. s [ T,T ] s [ T,T ] By 1.8), uniformly in n N and s [ T, T ], we can use Taylor expansion to bound n φ n s) 2 = 1 1 φ s) 2 )) e C n 1 φ s) 2 ) for some C > 0, and hence s [ T,T ] 1 φ s) 2 ) < C T <. Note that φ s) 2 is the characteristic function of X X, where X is an independent copy of X. Therefore we are in the same framework as in 1.11), and hence the following analogues of 1.12) and 1.13) hold: A > 0, E[ X X ) 2 1 { X A}] <, 1.14) δ > 0, P X X δ) <. 1.15) Note that by the uniform infinitesimality of X ) 1 i n, X X X. More precisely, by 1.15), which establishes 1.10). P X 1/2) = = P X 1/2) P X 1/4) P X 1/2, X 1/4) P X X 1/4) < To exploit 1.14) to prove 1.9), we note that E [ X X ) 2 1 { X X 2} ] E [ X X ) 2 1 { X 1} 1 { X 1} ] Therefore by 1.5), = 2P X 1) E[ X 2 1 { X 1} ] 2E[ X 1 { X 1} ]2 = 2P X 1) E[ X 2 1 { X 1} ] 2ã2. 2P X 1) E[ X 2 1 { X 1} ] E[ X X ) 2 1 { X X 2} ] + 18P X 1/2) 2. The uniform infinitesimality of X ) 1 i n implies that 2 min 1 i n P X 1) 1 for all n N large enough, therefore E[ X 1 2 { X 1} ] E[ X X ) 2 1 { X X 2}] + 18 P X 1/2) < 5
by 1.14) and 1.10). This finally concludes the proof of Theorem 1.5. Theorem 1.5 shows that every distribution µ which arises as the weak limit of a uniformly infinitesimal triangular array of independent random variables is also the weak limit of an accompanying triangular array of independent compound Poisson random variables shifted by constants. It is easy to see that the distribution of a compound Poisson random variable, shifted by a constant, is infinitely divisible. Therefore Theorem 1.4, i.e., the claim that µ is infinitely divisible, will follow from the next two lemmas. Lemma 1.7 If µ and ν are infinitely divisible distributions, then their convolution µ ν is also infinitely divisible. Lemma 1.8 If µ m), m N, is a sequence of infinitely divisible distributions converging weakly to µ, then µ is also infinitely divisible. Proof of Lemma 1.7. If µ and ν are infinitely divisible, then for each n N, there exist i.i.d. random variables X ) 1 i n with common distribution µ n, and i.i.d. random variables Y ) 1 i n with common distribution ν n, such that n X has distribution µ while n Y has distribution ν. Therefore n X + Y ) has distribution µ ν. Since X + Y ) 1 i n are i.i.d., it follows that µ ν is also infinitely divisible. Proof of Lemma 1.8. Since each µ m) is infinitely divisible, for every n N, there exist i.i.d. random variables X m) ) 1 i n such that S m) := n Xm) has distribution µ m). Let S be a random variable with distribution µ. Then by assumption, S m) S. To show that µ is infinitely divisible, we need to find i.i.d. random variables X ) 1 i n such that S has the same distribution as n X. We claim that it suffices to prove that {X m) n,1 } m N is a tight family of random variables. Indeed, tightness implies that along a subsequence m k, X m k) n,1 converges weakly to a limit X n,1, and hence X m k) ) 1 i n converges weakly to a collection of i.i.d. random variables X ) 1 i n. In particular, S mk) = n Xm k) converges weakly to n X. On the other hand, S mk) converges weakly to S. Therefore S has the same distribution as n X, which proves the infinite divisibility of µ. Recall that the tightness of {X m) n,1 } m N is equivalent to Note that P X m) n,1 A) = 0. 1.16) PX m) n,1 A)n = PX m) A for all 1 i n) PS m) na), and a similar bound holds for PX m) n,1 A). We then have P X m) n,1 A) PSm) na) 1 n + PS m) na) 1 n 2P S m) na) 1 n. Since S m) S, {S m) } m N is a tight family for which the analogue of 1.16) holds. We then have P X m) n,1 A) 2 P S m) na) 1 n which proves the tightness of {X m) n,1 } m N. = 2 P S m) na) ) 1 n = 0, 6