Lecture 10: Characteristic Functions

Transcription

1 Lecture 0: Characteristic Functions. Definition of characteristic functions. Complex random variables.2 Definition and basic properties of characteristic functions.3 Examples.4 Inversion formulas 2. Applications 2. Characteristic function for sum of independent random variables 2.2 Characteristic functions and moments of random variables 2.3 Continuity theorem 2.4 Weak law of large numbers 2.5 Central Limit theorem 2.6 Law of small numbers. Definition of characteristic function. Complex random variables () Complex numbers has a form u = a + ib where a, b are real numbers and i is so-called imaginary unit which has a property i 2 =.

2 (2) The summation of complex numbers u = a+ib and v = c+id is performed according the rule u + v = (a + c) + i(b + d) and the multiplication as uv = (ac bd) + i(ad + bc). (3) For a complex number u = a + ib, the conjugate complex number is defined as ū = a ib and the norm is defined by the formula u 2 = uū = a 2 + b 2 ; uv = u v ; u + v u + v. (4) The basic role in what follows plays the formula e it = cos t + i sin t, t R. Note that e it = cos 2 t + sin 2 t =. A complex random variable is defined as Z = X + iy where X = X(ω) and Y = Y (ω) are usual real-valued random variables defined on some probability space < Ω, F, P >. The distribution of a complex random variable Z is determined by the distribution of the random vector (X, Y ). The expectation of the complex random variable Z = X + iy is defined as the complex number EZ = EX + iey. Two complex random variables Z = X +iy and Z 2 = X 2 +iy 2 are independent if the random vectors (X, Y ) and (X 2, Y 2 ) are independent. (5) The product of two complex random variables Z Z 2 = (X X 2 Y Y 2 )+i(x Y 2 +Y X 2 ) then EZ Z 2 = E(X X 2 Y Y 2 )+ie(x Y 2 + Y X 2 ). If the random variables Z and Z 2 are independent, then 2

3 EZ Z 2 = EZ EZ 2. EZ Z 2 = E(X X 2 Y Y 2 ) + ie(x Y 2 + Y X 2 ) = (EX EX 2 EY EY 2 ) + i(ex EY 2 + EY EX 2 ) = EZ EZ 2. (6) If X M then EX M. Let X = Y +iz, then EX 2 = EY +iez 2 = (EY ) 2 +(EZ) 2 EY 2 + EZ 2 = E X 2 M 2. (7) EX E X..2 Definition and basic properties of characteristic functions Definition 0.. Let X be a random variable with the distribution F X (A) and the distribution function F X (x). The characteristic function of the random variable X (distribution F X (A) and the distribution function F X (x)) is called the complex-valued function φ X (t) defined on a real line as function of t by the formula, φ X (t) = E exp{itx} = eitx F X (dx) = E cos tx +ie sin tx = cos txf X(dx)+i sin txf X(dx). () If X is a discrete random variable with the discrete distribution p X (x n ) = P (X = x n ), n = 0,,... ( n p X (x n ) = ), then φ X (t) = E exp{itx} = n e itx n p X (x n ). 3

4 (2) If X is a continuous random variable with the probability density function f X (x), then φ X (t) = E exp{itx} = eitx f X (x)dx, where the integration the Riemann integration is performed if the density f X (x) is Riemann integrable (otherwise the Lebesgue integration should be used). (3) In the latter case the characteristic function is also known as the Fourier transform for function f X (x). Theorem 0.. The characteristic function φ X (t) of a random variable X possesses the following basic properties: () φ X (t) is a continuous function in t R ; (2) φ X (0) = (3) φ X (t) is positively defined function, i.e., a quadratic form j,l n c j c l φ X (t j t l ) 0 for any complex numbers c,..., c n and real t,..., t n and n. (a) φ X (t+h) φ X (t) Ee i(t+h)x Ee itx = Ee itx (e ihx ) E e itx (e ihx ) = E e ihx. (b) e ihx 2 and also e ihx hx. Choose A > 0 and get the following estimates E e ihx x >A eihx F X (dx) + A A eihx F X (dx) 2 F A X(dx) + hx F X(dx) x >A A 2P ( X A) + hap ( X A) 2P ( X A) + ha. 4

5 (c) Choose an arbitrary ε > 0. Then choose A so large that 2P ( X A) ε 2 and then h so small that ha ε 2. Then the expression on the right hand side in the estimate given in (b) will be less or equal ε. (d) φ X (0) = Ee i0x = E =. (e) To prove (3) we get j,l n = E j n c j c l φ X (t j t l ) = c j e it jx j n j,l n c j e it jx = E c j c l Ee i(t j t l )X j n c j e it jx 2 0. Theorem 0.2 (Bochner)**. A complex-valued function φ(t) defined on a real line is a characteristic function of some random variable X, i.e., can be represented in the form φ(t) = Ee itx = e itx F X (dx), t R if and only if it possesses the properties () - (3) formulated in Theorem. An additional properties of characteristic functions are: (4) φ X (t) for any t R ; (5) φ a+bx (t) = e iat φ X (bt); (6) e itx and, thus, φ X (t) = Ee itx ; (7) φ a+bx (t) = Ee it(a+bx) = e iat Ee itbx = e iat φ X (bt);.3 Examples 5

6 (a) If X a where a = const, then φ X (t) = e iat. () Binomial random variable. X = B(m, p), m =, 2,..., 0 p ; p X (n) = C n mp n ( p) m n, n = 0,,..., m; φ X (t) = (pe it + ( p)) m. (2) Poisson random variable. X = P(λ), λ > 0; p X (n) = λn n! e λ, n = 0,,... ; φ X (t) = e λ(eit ). (3) Normal random variable. X = N(m, σ 2 ), m R, σ 2 > 0; f X (x) = 2π e x2 2, x R ; φ X (t) = e t2 2 Y = m + σx; f Y (x) = 2πσ e (x m)2 2σ 2, x R φ Y (t) = e iat σ2 t 2 2 (4) Gamma random variable. X = Gamma(α, β), α, β > 0; f X (x) = Γ(α) xα e x β, x 0 where Γ(α) = 0 x α e x ; φ X (t) = ( iβt )α. 6

7 .4 Inversion formulas F (A) and G(A) are probability measures on a real line; φ(t) = e itx F (dx) and ψ(t) = e itx G(dx) are their characteristic functions. Lemma 0. (Parseval identity). The following so-called Parseval identity takes place, e ity φ(t)g(dt) = ψ(x y)f (dx). (a) The following equality follows from the definition of a characteristic function, e ity φ(t) = eit(x y) F (dx) (b) Integrating it with respect to the measure G(A) and using Fubinni theorem we get, e ity φ(t)g(dt) = [ eit(x y) F (dx)]g(dt) [ = eit(x y) G(dt)]F (dx) = ψ(x y)f (dx). Theorem 0.3 (inversion). Different characteristic function correspond to different distributions and vise versa. This theorem is a corollary of the inversion formula given below. 7

8 Theorem 0.4. Let F (x) be a distribution function and C F is the set of all point of continuity of this distribution function. Let also φ(t) = e itx F (dx) be the characteristic function that corresponds to this distribution function. The following inversion formula takes place z F (z) = lim σ2 0 2π [ e ity φ(t)e t 2 σ 2 2 dt]dy, z C F. (a) Let G(A) is a normal distribution with parameters m = 0 and σ. Then Parseval identity takes the following form 2 or equivalently e ity φ(t) σ e t2 σ 2 2 dt = 2π 2π e ity φ(t)e t 2 σ 2 2 dt = (x y) 2 e 2σ 2 F (dx), e (y x)2 2σ 2πσ 2 F (dx). (b) Integrating with respect to the Lebesgue measure (Riemann integration can be used!) over the interval (, z] we get z 2π [ e ity φ(t)e t 2 σ 2 z 2 dt]dy = [ = [ z e (y x)2 2σ 2πσ 2 F (dx)]dy e (x y)2 2σ 2πσ 2 dy]f (dx) = P (N(0, σ 2 ) + X z), where N(0, σ 2 ) is a normal random variable with parameters 0 and σ 2 independent of the random variable X. (c) If σ 2 0 then the random variables N(0, σ 2 ) converge in probability to 0 (P ( N(0, σ 2 ) ε) 0 for any ε > 0) and, in 8

9 sequel, the random variables N(0, σ 2 ) + X weakly converge to the random variable X that is P (N(0, σ 2 )+X z) P (X z) as σ 2 0 for any point z that is a point of continuity for the distribution function of the random variable X. Theorem 0.5. Let F (x) be a distribution function and φ(t) = e itx F (dx) be the characteristic function that corresponds to this distribution function. Let φ(t) is absolutely integrable at real line, i.e., φ(t) dt <. Then the distribution function F (x) has a bounded continuous probability density function f(y) and the following inversion formula takes place f(y) = 2π e ity φ(t)dt, y R. (a) The proof can be accomplished by passing σ 2 0 in the Parseval identity 2π e ity φ(t)e t 2 σ 2 2 dt = e (y x)2 2σ 2πσ 2 F (dx). (b) Both expressions on the left and right hand sides of the above identity represent the probability density function f σ (y) for the distribution function F σ (y) of random variable N(0, σ 2 ) + X. (c) F σ ( ) F ( ) as σ 0. (d) From the left hand side expression in the above identity, we see that. f σ (y) f(y) as σ 0. 9

10 (e) Note that f σ (y) and f(y) are bounded continuous functions; (f) It follows from (c), (d) and (e) that for any x < x F σ (x ) F σ (x ) = x x f σ (y)dy x (g) But, if x, x are continuity point of F (x) that (h) Thus, x f(y)dy F σ (x ) F σ (x ) F (x ) F (x ) as σ 0. F (x ) F (x ) = x x f(y)dy, i.e., indeed, f(y) is a probability density of the distribution function F (x). 2. Applications 2. Characteristic function for sum of independent random variables Lemma 0.2. If a random variable X = X + + X n is a sum of independent random variables X,..., X n then φ X (t) = φ X (t) φ Xn (t). φ X (t) = Ee it(x + X n ) = E n k= e itx k = n k= Ee itx k. 0

11 Examples () Let random variables X k = N(m k, σ 2 k), k =,..., n be independent normal random variables. In this case X = X + + X n is also a noramal random variable with parameters m = m + + m n and σ 2 = σ σ 2 n. φ X (t) = n k= e itm k σ 2 k t2 2 = e itm σ2 t 2 (2) Let random variables X k = P(λ k ), k =,..., n be independent Poissonian random variables. In this case X = X + + X n is also a Poissonian random variable with parameters λ = λ + + λ n. φ X (t) = n k= e λ k(e it ) = e λ(eit ). 2.2 Characteristic functions and moments of random variables Theorem 0.6. Let X be a random variable such that E X n = x n F X (dx) < for some n. Then, there exist n-th continuous derivatives for the characteristic function φ X (t) = e itx F X (dx) for all t R given by the formula, 2. φ (k) X (t) = i k xk e itx F X (dx), k =,..., n,

12 and, therefore, the following formula takes place () φ X(t+h) φ X (t) h EX k = i k φ (k) X (0), k =,..., n. = e itx eihx h F X (dx); x that let to perform the limit transition in the (2) eihx h above relation as h 0 using also that eihx h ix as h 0; (3) In general case, the formula for φ (k) X (t) can be proved by induction. Examples () Let X = N(m, σ 2 ) be a normal random variable. Then φ X (t) = e itm σ2 t 2 2 and, therefore, () φ X(t) = (im σ 2 t)e itm σ2 t 2 2 and EX = i φ X(0) = m. (2) φ X(t) = σ 2 e itm σ2 t (im σ 2 t) 2 e itm σ2 t 2 2 and EX 2 = i 2 φ X(0) = σ 2 + m 2. (2) Let random variables X = P(λ) be a Poissonian random variable. Then φ X (t) = e λ(eit ) and, therefore, () φ X(t) = iλe it e λ(eit ) and, thus, EX = i φ X(0) = λ. (2) φ X(t) = i 2 λe it e λ(eit ) +i 2 λ 2 e 2it e λ(eit ) and EX 2 = i 2 φ X(0) = λ + λ Continuity theorem X n, n = 0,,... is a sequence of random variables with distribution functions, respectively, F n (x), n = 0,,...; 2

13 φ n (t) = Ee itx n = e itx F n (dx), n = 0,,... are the corresponding characteristic functions; C F denotes the set of continuity points for a distribution function F (x). Theorem 0.7 (continuity)**. Distribution functions F n F 0 as n if and only if the corresponding characteristic functions φ n (t) φ 0 (t) as n for every t R. (a) Let F n F 0 as n. (b) Then by Helly theorem, for every t R, φ n (t) = eitx F n (dx) (c) Let φ n (t) φ 0 (t) as n, t R. eitx F 0 (dx) = φ 0 (t) as n. (d) As was proved in Lecture 9 using Cantor diagonal method, for any subsequence n k as k, one can select a subsequence n r = n kr as r such that F n r (x) F (x) as n, x C F, where F (x) is proper or improper distribution function (nondecreasing, continuous from the right and such that 0 F ( ) F ( ) ). (e) Let us write down the Parseval identity 2π e ity φ n r (t)e t 2 σ 2 2 dt = e (y x)2 2σ 2πσ 2 F n r (dx). 3

14 (f) By passing to the limit as r in the above identity and takin into account (c), and Lebesque theorem, and the fact that 2πσ e (y x)2 2σ 2 0 as x ±, we get 2π e ity φ 0 (t)e t 2 σ 2 2 dt = e (y x)2 2σ 2πσ 2 F (dx). (g) By multiplying both part in (f) by 2πσ and taking y = 0, we get φ 0(t) σ e t2 σ 2 2 dt = 2π e x 2 2σ 2 F (dx) F (+ ) F ( ). σ 2π e t2 σ 2 (g) By passing to the limit as σ 0 in the above equality and taking into account that (a) σ 2π e t2 σ as σ, for t 0, and (c) φ(t) is continuous in zero and φ(0) =, we get, 2 dt =, (b) = σ lim φ 0(t) σ e t2 σ 2 2 dt F (+ ) F ( ). 2π (h) This, F (+ ) F ( ) =, i.e., the distribution F (x) is a proper distribution. (i) In this case, the weak convergence F n r F as n implies by the first part of the theorem that φ n r (t) φ(t) = eitx F (dx) as n, t R. (j) This relation together with (c) implies that φ(t) = eitx F (dx) = φ 0 (t), t R. 4

15 (k) Therefore due to one-to-one correspondence between characteristic functions and distribution functions F (x) F 0 (x). (l) Therefore, by the subsequence definition of the weak convergence F n F 0 as n. Theorem 0.8* (continuity). Distribution functions F n (x) weakly converge to some distribution function F (x) if and only if the corresponding characteristic functions φ n (t) converge pointwise (for every t R ) to some continuous function φ(t). In this case, φ(t) is the characteristic function for the distribution function F (x). (a) Let φ n (t) φ(t) as n for every t R. Then φ(0) = ; (b) φ(t) is positively defined since functions φ n (t) possess these properties. (c) Since φ(t) is also continuous, the properties (a)-(c) pointed in Theorem 0. hold and, therefore, φ(t) is a characteristic function for some distribution function F (x). Then F n (x) weakly converge to F (x) by Theorem Weak law of large numbers (LLN) < Ω, F, P > is a probability space; X n, n = 0,, 2,... are independent identically distributed (i.i.d.) 5

16 random variables defined on a probability space < Ω, F, P >. Theorem 0.9 (weak LLN). If E X <, EX = a, then random variables X n = X + + X n d a as n. n (a) Let φ(t) = Ee itx, φ n(t) = Ee itx n be the corresponding characteristic functions. Then φ n (t) = (φ( t n ))n. (b) Since E X <, there exists the continuous first derivative φ (t) and, therefore, Taylor expansion for φ(t) in point t = 0 can be written down that is φ(t) = + φ (0)t + o(t) (c) Since φ (0) = iex = ia, the above Taylor expansion can be re-written. in the following form φ(t) = + iat + o(t). (d) Therefore, φ n (t) = (φ( t n ))n = ( + ia t n + o( t n ))n. (e) If u n 0 and v n then lim( + u n ) v n exists if and only if there exists lim u n v n = w and, in this case, lim(+u n ) v n = ew. (f) Using this fact, we get lim n φ n (t) = lim n ( + ia t n + o( t n ))n = e iat, for every t R. (g) Thus by the continuity theorem for characteristic functions d X n a as n. 0.0 Central Limit Theorem (CLT) < Ω, F, P > is a probability space; X n, n = 0,, 2,... are independent identically distributed (i.i.d.) 6

17 random variables defined on a probability space < Ω, F, P >; Z = N(0, ) is standard normal random variable with the distribution function F (x) = nσ x e y2 2 dy, < x <. Theorem 0.0 (CLT). If E X 2 <, EX = a, V arx = σ 2 > 0, then random variables Z n = X + + X n an nσ d Z as n. () Since, the limit distribution F (x) is continuous, the above relation means that P (Z n x) F (x) as n for every < x <. (a) Let introduce random variables Y n = X n a σ, n =, 2,..., then Z n = X + +X n an nσ = Y + +Y n n. (b) Let ψ(t) = Ee ity, ψ n(t) = Ee itz n. Then ψ n(t) = (ψ(t/ n)) n. (c) If E X 2 < then E Y 2 < and, moreover, EY = σ (EX a) = 0, E(Y ) 2 = σ 2 E(X a) 2 =. (d) Since E Y 2 <, there exist the continuous second derivative ψ (t) and, therefore, Taylor expansion for ψ(t) in point t = 0 can be written down that is ψ(t) = + ψ (0)t + 2 ψ (0)t 2 + o(t 2 ). (e) Since ψ (0) = iey = 0 and ψ (0) = i 2 E(Y ) 2 =, the above Taylor expansion can be re-written in the following form ψ(t) = 2 t2 + o(t 2 ). (f) Therefore, ψ n (t) = (ψ( t n )) n = ( t2 2n + o(t2 n ))n. (g) Using the above formula, we get lim n ψ n (t) = lim n ( t 2 2 2n + o(t2 n ))n = e t 2, for every t R. 7

18 (h) Thus by the continuity theorem for characteristic functions d Z n Z as n. 2.6 Law of small numbers Let X n,k, k =, 2,... are i.i.d. Bernoulli random variables defined on a probability space < Ω n, F n, P n > and taking values and 0, respectively, with probabilities p n and p n, where 0 p n. N(λ) = P(λ) is a Poissonian random variable with parameter λ, i.e., P (N(λ) = r) = λr r! e λ, r = 0,,.... Theorem 0. (LSN). If 0 < p n n λ <, then random variables N n = X n, + + X n,n d N(λ) as n. () The random variable N n is the number of successes in n Bernoulli trials. (2) The random variables N n = X n, + + X n,n, n =, 2,... are determined by series of random variables X, X 2,, X 2,2 X 3,, X 3,2, X 3, This representation gave the name to this type of models. as so-called triangle array mode. 8

19 (a) Let φ n (t) = Ee itn n The characteristic function EeitX n, = p n e it + p n and, thus, the characteristic function φ n (t) = Ee itn n = (p ne it + p n ) n. (b) Using the above formula, we get φ n (t) = (p n e it + p n ) n e p n(e it )n e λ(eit ) as n, for every t R. (c) Thus by the continuity theorem for characteristic functions d N n N(λ) as n. LN Problems. What is the characteristic function of random variable X which takes value and with probability Prove that the function cos n t is a characteristic function for every n =, 2, Let X is a uniformly distributed random variable on the interval [a, b]. Please find the characteristic function of the random variable X. 4. Let f(x) is a probability density for a random variable X. Prove that the characteristic function is a real-valued function if f(x) ia a symmetric function, i.e. f(x) = f( x), x Let random variables X and X 2 are independent and have gamma distributions s parameters α, β and α 2, β, respectively. Prove that the random variable Y = X + X 2 also has gamma distribution with parameters α, +α 2, β. 9

20 6. Let X has a geometrical distribution with parameter p. Find EX and V arx using characteristic functions. 7.Let X n are geometrical random variables with discrete distribution p Xn (k) = p n ( p n ) k, k =, 2,..., where 0 < p n 0 d as n. Prove that p n X n X as n, where X = Exp() is an exponential random variable with parameter. 20