Lecture 4: Random Variables

Transcription

1 Lecture 4: Random Variables 1. Definition of Random variables 1.1 Measurable functions and random variables 1.2 Reduction of the measurability condition 1.3 Transformation of random variables 1.4 σ-algebra generated by a random variable 2. Distribution functions of random variables 2.1 Measure generated by a random variable 2.2 Distribution function of a random variable 2.3 Properties of distribution functions 2.4 Random variables with a given distribution function 2.5 Distribution functions of transformed random variables 3. Types of distribution functions 3.1 Discrete distribution functions 3.2 Absolutely continuous distribution functions 3.3 Singular distribution functions 3.4 Decomposition representation for distribution functions 4 Multivariate random variables (random vectors) 4.1 Random variables with values in a measurable space 4.2 Random vectors 4.3 Multivariate distribution functions 1

2 5 Independent random variables 5.1 Independent random variables 5.2 Mutually independent random variables 1. Definition of Random variables 1.1 Measurable functions and random variables < Y, B Y > and < X, B X > are two measurable spaces (space plus σ-algebra of measurable subsets of this space); f = f(y) : Y X is a function acting from Y to X. Definition 4.1, f = f(y) is a measurable function if {y : f(y) A} B Y, A B X. Example (1) Y = {y 1,..., } is a finite or countable set; B Y is the σ-algebra of all subsets of Y. In this case, any function f(x) acting from Y to X is measurable. (2) Y = X = R 1 ; B Y = B X = B 1 is a Borel σ-algebra. In this case, f(x) is called a Borel function. (3) A continuous function f = f(x) : R 1 R 1 is a Borel function. 2

3 < Ω, F, P > is a probability space; X, B X is a measurable space. X = X(ω) : Ω X (a function acting from Ω to X ). Definition 4.2. X = X(ω) is a random variable with values in space X defined on a probability space < Ω, F, P > if it is a measurable function acting from Ω X, i.e., such function that {ω : X(ω) A} F, A B X. < Ω, F, P > is a probability space; X = X(ω) : Ω R 1 (a function acting from Ω to R 1 ); B X = B 1 is a Borel σ-algebra of subsets of R 1. Definition 4.3. X = X(ω) is a (real-valued) random variable defined on a probability space < Ω, F, P > if it is a measurable function acting from Ω R 1, i.e., such function that {ω : X(ω) A} F, A B 1. < Ω, F, P > is a probability space; X = X(ω) : Ω [, + ]; B + 1 is a Borel σ-algebra of subsets of [, + ] (minimal σ- algebra containing all intervals [a, b], a b + ); Definition 4.4. X = X(ω) is a (improper) random variable defined on a probability space < Ω, F, P > if it is a measurable function acting from Ω R 1, i.e., such function that {ω : X(ω) A} F, A B

4 Examples (1) X = {x 1,..., x N }, B X is a σ-algebra of all subsets of X; X is a random variable with a finite set of values; (2) X = R 1, B X = B 1 ; X is a (real-valued) random variable; (3) X = R k, B X = B k ; X is a random vector (random variable with values in R k ; (4) X is a metric space, B X is a Borel σ-algebra of subsets of X; X is a random variable with values in the metric space X. (5) Ω = {ω 1, ω 2...} a discrete sample space; F = F 0 the σ-algebra of all subsets of Ω; Any function X = X(ω) : Ω R 1 is a random variable since, in this case, it is automatically a measurable function. (6) Ω = R 1 ; F = B 1 Borel σ-algebra of subsets of R 1 ; 1.2 Reduction of the measurability condition The following notations are used: X 1 (A) = {X A} = {ω : X(ω) A}. Theorem 4.1. The measurability condition (A) X 1 (A) F, A B 1 hold if and only if (B) A x = {ω : X(ω) x} F, x R 1. 4

5 (A) (B). Indeed, (, x], x R 1 are Borel sets; (B) (A). Indeed, let K be a class of all sets A R 1 such that X 1 (A) F. Then (a) X 1 ((a, b]) = A b \ A a F, a b. Thus, (a, b] K, a < b; (b) A K A K since X 1 (A) = X 1 (A) F; (c) A 1, A 2... K n A n K since X 1 ( n A n ) = n X 1 (A n ) F; (d) Thus K is a σ-algebra which contains all intervals. Therefore, B 1 K. 1.3 Transformation of random variables X 1,..., X k random variables defined on a probability space < Ω, F, P >; f(x 1,..., x k ) : R k R 1 a Borel function, i.e., f 1 (A) = {(x 1,..., x k ) A} B k, A B 1. Theorem 4.2. X = f(x 1,..., X k ) is a random variable. (a) {ω : X 1 (ω) (a 1, b 1 ],..., X k (ω) (a k, b k ]} F, a i b i, i = 1,..., k; (b) Let K be a class of all sets A R k such that {(X 1 (ω),..., X k (ω)) A} F. Then K is a σ-algebra. The 5

6 proof is analogous to those given for Theorem 1. (c) C B 1 {ω : f(x 1 (ω),..., X k (ω)) C} = {ω : (X 1 (ω),..., X k (ω)) f 1 (C)} F. (d) Thus, X = f(x 1,..., X k ) is a random variable. (1) U ± = X 1 ± X 2, V = X 1 X 2, W = X 1 /X 2 (if X 2 0) are random variables; (2) Z + = max(x 1,..., X k ), Z = min(x 1,..., X k ) are random variables. X, X 1, X 2,..., are random variables defined on a probability space < Ω, F, P >; Theorem 4.3. Let X be a random variable defining by one of the relation, X = sup X n n 1 X = inf n 1 X n X = lim n X n = inf X = lim n X n = sup n 1 sup X k n 1 k n inf k n X k X = lim n X n = lim n X n = lim n X n Then X is a random variable (possibly improper). (1) {ω : sup n 1 X n (ω) > x} = n 1 {X n (ω) > x}; (2) {ω : inf n 1 X n (ω) < x} = n 1 {X n (ω) < x}; 6

7 (3) {ω : lim n X n (ω) < x} = l 1 n 1 k n X k (ω) < x 1 l }; (4) {ω : lim n X n (ω) > x} = l 1 n 1 k n X k (ω) > x 1 l }; Let A 1,..., A n F and a 1,..., a n are real numbers. Definition 4.5. X(ω) = n k=1 a k I Ak (ω) is a simple random variable. Theorem 4.4. X = X(ω) is a random variable if and only if X(ω) = lim n X n (ω), ω Ω, where X n, n = 1, 2,... are simple random variables. < Z, B Z >, < Y, B Y > and < X, B X > and are three measurable spaces; f = f(z) : Z Y is a measurable function acting from Z to Y. g = g(y) : Y X is a measurable function acting from Y to X. Theorem 4.5. The superposition h(z) = g(f(x)) of two measurable functions f and g is a measurable function acting from space Z to space X. (1) Let A X. Then h 1 (A) = f 1 (g 1 (A)). (2) Let A B X. Then g 1 (A) B Y ; (3) Then h 1 (A) = f 1 (g 1 (A)) B Z. 1.4 σ-algebra generated by a random variable Theorem 4.6 Let X = X(ω) be a random variable defined 7

8 on a probability space < Ω, F, P >. The class of sets F X =< X 1 (A), A B 1 > is a σ-algebra (generated by the random variable X). (a) C F X C = X 1 (A), where A B 1 C = X 1 (A) = X 1 (A) F X since A B 1 ; (b) C 1, C 2,... F X C n = X 1 (A n ), n = 1, 2,..., where A n B 1, n = 1, 2,... n C n = n X 1 (A n ) = X 1 ( n A n ) F X since n A n B 1 ; (d) Thus F X is a σ-algebra. (1) F X F. Theorem 4.7 Let X = X(ω) be a random variable defined on a probability space < Ω, F, P > and taking values in a space X with σ-algebra of measurable sets B X. The class of sets F X =< X 1 (A), A B X > is a σ-algebra (generated by the random variable X). 2. Distribution functions of random variables 2.1 Measure generated by a random variable X = X(ω) a random variable defined on a probability space < Ω, F, P >. P X (A) = P (ω : X(ω) A) = P (X 1 (A)), A B 1. Theorem 4.8. P X (A) is a probability measure defined on Borel σ-algebra B 1. 8

9 (a) P X (A) 0; (b) A 1, A 2,... B 1, A i A j = X 1 ( n A n ) = n X 1 (A n ) and, therefore, P X ( n A n ) = P (X 1 ( n A n )) = n P (X 1 (A n )) = n P X (A n ); (c) P X (R 1 ) = P (X 1 (R 1 )) = P (Ω) = 1. Definition 4.6. The probability measure P X (A) is called a distribution of the random variable X. X = X(ω) a random variable defined on a probability space < Ω, F, P > and taking values in a space X with σ-algebra of measurable sets B X. P X (A) = P (ω : X(ω) A) = P (X 1 (A)), A B X. Theorem 4.9. P X (A) is a probability measure defined on Borel σ-algebra B X. Definition 4.7. The probability measure P X (A) is called a distribution of the random variable X. 2.2 Distribution function of a random variable X = X(ω) a random variable defined on a probability space < Ω, F, P >; P X (A) = P (X(ω) A) = P (X 1 (A)), A B 1 the distribution of the random variable X. 9

10 Definition 4.8. The function F X (x) = P X ((, x]), x R 1 is called the distribution function of a random variable X. (1) The distribution P X (A) uniquely determines the distribution function F X (x) and, as follows from the continuation theorem, the distribution function of random variable uniquely determines the distribution P X (A). 2.3 Properties of distribution functions A distribution function F X (x) of a random variable X possesses the following properties: (1) F X (x) is non-decreasing function in x R 1 ; (2) F X ( ) = lim x F X (x) = 0, F X ( ) = lim x F X (x)= 1; (3) F X (x) is continuous from the right function, i.e., F X (x) = lim y x,y x F X (y), x R 1. - (a) x x (, x ] (, x ] F X (x ) = P X ((, x ]) F X (x ) = P X ((, x ]); (b) x n z n = max k n x k, F X (x n ) F X (z n ) = P X ((, z n ]) 0 since n (, z n ] = ; (c) x n z n = min k n x k, F X (x n ) F X (z n ) = P X ((, z n ]) 1 since n (, z n ] = R 1 ; (d) x n x, x n x z n = max k n x k, x F X (x n ) F X (z n ) = P X ((, z n ]) F X (x) since n (, z n ] = (, x]; - 10

11 (4) P (X (a, b]) = P X ((a, b]) = F X (b) F X (a), a b; (5) P (X = a) = P X ({a}) = F X (a) F X (a 0), a b; - (e) P X ({a}) = lim n (F X (a) F X (a 1 n )) = F X(a) F X (a 0) since n (a 1 n, a] = {a}. - (6) Any distribution function has not more that n jumps with values 1 n for every n = 1, 2,... and, therefore, the set of all jumps is at most countable. - (f) Let a 1 < < a N be some points of jumps with values 1 n. Then N n=1{x = a n } { < X < }. Thus, N/n Nn=1 P (X = a n ) = P ( N n=1{x = a n } P ( < X < ) = 1 and thus N n Random variables with a given distribution function One can call any function F (x) defined on R 1 a distribution function if it possesses properties (1) (3). According the continuation theorem every distribution function uniquely determines (generates) a probability measure P (A) on B 1 which is connected with this distribution function by the relation P ((a, b]) = F (b) F (a), a < b. 11

12 Theorem For any distribution function F (x) there exists a random variable X that has the distribution function F X (x) F (x). - (a) Choose the probability space < Ω = R 1, F = B 1, P (A) > where P (A) is the probability measure which is generated by the distribution function F (x). (b) Consider the random variable X(ω) = ω, ω R 1. Then P (ω : X(ω) = ω x) = P ((, x]) = F (x), x R 1. - Let F (x) is distribution function. One can define the inverse function F 1 (y) = inf(x : F (x) y), 0 y 1.. Random variable Y has an uniform distribution if it has the following distribution function F Y (x) = 0 if x < 0, x if x [0, 1], 1 if x > 1. Theorem 4.11*. For any distribution function F (x) the random variable X = F 1 (Y ), where Y is a uniformly distributed random variable, has the distribution function F (x). - (a) Consider here only the case where F (x) is a continuous 12

13 strictly monotonic distribution function. In this case F 1 (y) = inf(y : F (x) = y) is also a continuous strictly monotonic function and F 1 (F (x)) = x. (b) F (x) = P (Y F (x)) = P (F 1 (Y ) F 1 (F (x)) = P (X x), x R 1. - Example Let F (x) = 1 e ax, x 0 be an exponential distribution function In this case F 1 (y) = 1 a ln(1 y) and random variable X = 1 a ln(1 Y ) has the exponential distribution function with parameter a. 2.5 Distribution functions of transformed random variables X random variable with a distribution function F X (x) and the corresponding distribution P X (A); f(x) : R 1 R 1 is a Borel function. A f (x) = {y R 1 : f(y) x}, x R 1. Theorem The distribution function of the transformed random variable Y = f(x) is given by the following formula, F Y (x) = P (f(x) x) = P (X A f (x)) = P X (A f (x)), x R 1. Examples (1) Y = ax + b, a > 0; A f (x) = (, x b a ]; 13

14 F Y (x) = F X ( x b a ). (2) Y = e ax, a > 0; A f (x) = if x 0 or (, 1 a ln x] if x > 0; F Y (x) = I(x > 0)F X ( 1 a ln x). (3) Y = X 2 ; A f (x) = if x 0 or [ x, x] if x > 0; F Y (x) = I(x > 0)(F X ( x) F X ( x 0)). 3 Types of distribution functions 3.1 Discrete distribution functions Definition 4.9. A distribution function F (x) is discrete if there exists a finite or countable set of points A = {a 1, a 2,...} such that n(f (a n ) F (a n 0)) = 1. If X is a random variable with the distribution function F (x) then P (X A) = Examples a n A P (X = a n ) = (a) Bernoulli distribution; (b) Discrete uniform distribution; (c) Binomial distribution; (d) Poisson distribution; (e) Geometric distribution; 14 a n A (F (a n ) F (a n 0)).

15 3.2 Absolutely continuous distribution functions. Definition A distribution function F (x) is absolutely continuous if it can be represented in the following form F (x) = x f(y)dy, x R 1, where (a) f(y) is a Borel non-negative function; (b) f(y)dy = 1; (c) Lebesgue integration is used in the formula above (if f(y) is a Riemann integrable function then the Lebesgue integration can be replaced by Riemann integration). Examples (a) Uniform distribution; (b) Exponential distribution; (c) Normal (Gaussian) distribution; (d) Gamma distribution distribution; (e) Guachy distribution; (f) Pareto distribution. 3.3 Singular distribution functions. Definition A distribution function F (x) is singular if it is a continuous function and its set of points of growth S F has Lebesgue measure m(s F ) = 0 (x is point of growth if F (x + ɛ) F (x ɛ) > 0 for any ɛ > 0). Example* 15

16 Define a continuous distribution function F (x) such that F (x) = 0 for x < 0 and F (x) = 1 for x > 1, which set of of points of growth S F is the Cantor set, in the following way: (a) Define function F (x) at the Cantor set in the following way: (1) [0, 1] = [0, 1 3 ] [1 3, 2 3 ] [2 3, 1]: F (x) = 1 2, x [1 3, 2 3 ]; (2) [0, 1 3 ] = [0, 1 9 ] [1 9, 2 9 ] [2 9, 1 3 ]: F (x) = 1 4, x [1 9, 2 9 ]; (3) [ 2 3, 1] = [2 3, 7 9 ] [7 9, 8 9 ] [8 9, 1]: F (x) = 3 4, x [7 9, 8 9 ];... (b) Define a function F (x) as continuous function in points that do not belong to the listed above internal intervals. (c) The sum of length of all internal intervals, where the function F (x) take constant values is equal = 1 3 k=0 ( 2 3 )k = = Decomposition representation for distribution functions Theorem 4.13 (Lebesgue)**. Any distribution function F (x) can be represented in the form F (x) = p 1 F 1 (x)+p 2 F 2 (x)+ p 3 F 3 (x), x R 1 where (a) F 1 (x) is a discrete distribution function, (a) F 2 (x) is an absolutely continuous distribution function, (c) F 3 (x) is singular distribution function, (d) p 1, p 2, p 2 0, p 1 + p 2 + p 3 = 1. 4 Multivariate random variables (random vectors) 16

17 4.1 Random variables with values in a measurable space Let X is an arbitrary space and B(X ) is a σ-algebra of measurable subsets of X. Definition A random variable X = X(ω) defined on a probability space < Ω, F, P > and taking values in the space X (with a σ-algebra of measurable subsets B(X )) is a measurable function acting from Ω X, i.e., such function that {ω : X(ω) A} F for any A B(X ). Examples (1) X = R 1, B(X ) = B 1. In this case, X is a real-valued random variable; (2) X = {0, 1} {0, 1} (the product is taken n times), B(X ) is a σ-algebra of all subsets of X. A random variable X = (X 1,..., X n ) is a Bernoulli vector which components are Bernoulli random variables. (3) X ia a metric space, B(X ) is a Borel σ-algebra of subsets of X (the minimal σ-algebra containing all balls); X is a random variable taking values in the metric space X. 4.2 Random vectors 17

18 X = R k, B = B k and P is a probability measure defined on B k. Definition A multivariate random variable (random vector) is a random variable X = (X 1,..., X n ) defined on a probability space < Ω, F, P > and taking values taking values in the space X = R k (with a σ-algebra of measurable subsets B(X ) = B k ). (1) Every component of a random vector is a real-valued random variable defined on the same probability space. (2) If X k = X k (ω), k = 1,..., n are real-valued random variables defined on some probability space, then X = (X 1,..., X n ) is a random vector defined on the same probability space. 4.3 Multivariate distribution functions Definition. A multivariate distribution function of a random variable (random vector) X = (X 1,..., X n ) is a function F (x 1,..., x n ) defined for x = (x 1,..., x n ) R n by the following relation F X1,...,X n (x 1,..., x n ) = P (X 1 x 1,..., X n x n ). The multivariate distribution function possesses the following properties: (1) lim xk F X1,...,X n (x 1,..., x n ) = 0; (2) F X1,...,X n (x 1,..., x n ) non-decrease in every argument; (3) lim xk,k=1,...,n F X1,...,X n (x 1,..., x n ) = 1; 18

19 (4) the multivariate distribution functions of the random vectors (X 1,..., X n ) and (X 1,..., X k 1, X k+1,..., X n ) are connected by the following relation lim xk F X1,...,X n (x 1,..., x n ) = F X1,...,X k 1,X k+1,...x n (x 1,..., x k 1, x k+1,..., x n ); (5) P (X 1 (a 1, b 1 ],..., X n (a 1, b 1 ]) = F X1,...,X n (b 1,..., b n ) k F X1,...X n (b 1,..., b k 1, a k, b k,..., b n ) + +( 1) n F X1,...,X n (a 1,..., a n ) 0; (6) F X1,...,X n (x 1,..., x n ) is continuous from above functions that is lim yk x k,k=1,...,n F X1,...,X n (y 1,..., y n ) = F X1,...,X n (x 1,..., x n ). Example Let X = (X 1, X 2 ) is a two-dimensional random vector. Then P (X 1 (a 1, b 1 ], X 2 (a 2, b 2 ]) = F X1,X 2 (b 1, b 2 ) F X1,X 2 (b 1, a 2 ) F X1,X 2 (a 1, b 2 ) + F X1,X 2 (a 1, a 2 ). Theorem A multivariate distribution function F X1,...,X n (x 1,..., x n ) of a random vector X = (X 1,..., X n ) uniquely determines a probability measure P X (A) on the Borel σ- algebra B k by its values on the cubes P X ((a 1, b 1 ] (a n, b n ]) = P (X 1 (a 1, b 1 ],..., X n (a 1, b 1 ]). 5 Independent random variables 5.1 Independent random variables Definition Two random variables X and Y with distribution functions, respectively, F X (x) and F Y (y) are independent 19

20 if the two-dimensional distribution of the random vector (X, Y ) satisfies the relation F X,Y (x, y) = F X (x) F Y (y), x, y R 1. (1) If random variables X and Y are independent then P (X A, Y B) = P (X A) P (Y B) for any A, B B Mutually independent random variables Definition Random variables X t, t T with distribution functions F Xt (x) are mutially independent if for any t 1,..., t n, t i t j the multivariate distribution function F Xt1,...,X tn (x 1,..., x n ) of the random vector (X t1,..., X tn ) satisfies the relation F Xt1,...,X tn (x 1,..., x n ) = F Xt1 (x 1 ) F Xtn (x n ). LN Problems 1. Let A is a random event for a probability space < Ω, F, P > and I = I A (ω) is a indicator of event A. Prove that I is a random variable. 2. Let X 1, X 2,... be a sequence of random variables defined on a probability space < Ω, F, P >. Let also Z = max n 1 X n and I is an indicator of event A = {Z < }. Let Y = Z I where the product is counted as 0 if Z =, I = 0. Prove that Y is a random variable. 3. Let F (x) is a distribution function af a random variable X. Prove that P (a X b) = F (b) F (a 0) and 20

21 P (a < X < b) = F (b 0) F (a). 4. Let random variable X has a continuous strictly monotonic distribution function F (x). Prove that the random variable Y = F (X) is uniformly distributed in the interval [0, 1]. 5. Let Ω = {ω 1, ω 2...} be a discrete sample space, F 0 the σ-algebra of all subsets of Ω, and P (A) is a probability measure on F. Let also X is a random variable defined on the discrete probability space < Ω, F 0, P >. Can the random variable X be a continuous or a singular distribution function? 5. Let random variable X has a distribution function F (x). What distribution function have random variables Y = ax 2 + bx + c? 6 Let X and Y are independent random variables uniformly distributed in the interval [0, 1]. What is the two-variate distribution function of the random vector Z = (X, Y )? 7. Let X 1,..., X n be independent random variables with the same distribution functionf (x). What are the distribution functions of random variables Z n + = max(x 1,..., X n ) and Zn = min(x 1,..., X n )? 8. Give the proof of Theorem Give the proof of Theorem Give the proof of Theorem 4.9 for the case of a general 21

22 distribution function (with possible jumps) [see G] 11. Give the proof of related to the example given in Section