Chapter 4. Multivariate Distributions

Transcription

1 1 Chapter 4. Multivariate Distributions Joint p.m.f. (p.d.f.) Independent Random Variables Covariance and Correlation Coefficient Expectation and Covariance Matrix Multivariate (Normal) Distributions Matlab Codes for Multivariate (Normal) Distributions Some Practical Examples The Joint Probability Mass Functions and p.d.f. Let X and Y be two discrete random variables and let R be the corresponding space of X and Y. The joint p.m.f. of X = x and Y = y, denoted by f(x, y) =P (X = x, Y = y), has the following properties: (a) f(x, y) 1for(x, y) R. (b) (x,y) R f(x, y) =1, (c) P (A) = (x,y) A f(x, y), where A R. The marginal p.m.f. of X is defined as f X (x) = y f(x, y),foreach x R x. The marginal p.m.f. of Y is defined as f Y (y) = x f(x, y),foreach y R y. The random variables X and Y are independent iff (if and only if) f(x, y) f X (x)f Y (y) for x R x, y R y. Example 1. f(x, y) = (x + y)/21, x = 1, 2, 3; y = 1, 2, then X and Y are not independent. Example 2. f(x, y) =(xy 2 )/3, x =1, 2, 3; y =1, 2, then X and Y are independent.

2 2 The Joint Probability Density Functions Let X and Y be two continuous random variables and let R be the corresponding space of X and Y. The joint p.d.f. of X = x and Y = y, denoted by f(x, y) =P (X = x, Y = y), has the following properties: (a) f(x, y) for <x,y<. (b) f(x, y)dxdy =1. (c) P (A) = A f(x, y), where A R. The marginal p.d.f. of X is defined as f X (x) = f(x, y)dy, for x R x. The marginal p.d.f. of Y is defined as f Y (y) = f(x, y)dx, for y R y. The random variables X and Y are independent iff (if and only if) f(x, y) f X (x)f Y (y) for x R x, y R y. Example 3. Let X and Y have the joint p.d.f. f(x, y) = 3 2 x2 (1 y ), 1 <x<1. 1 <y<1. Let A = {(x, y) <x<1, <y<x}. Then P (A) = 1 x 3 2 x2 (1 y)dydx = [ ] 1 x 3 2 x2 y y2 dx 2 = [ ] [ x 3 x4 2 dx = 3 x 4 ] 1 x = 9 4 Example 4. Let X and Y have the joint p.d.f. f(x, y) =2, x y 1. Thus R = {(x, y) x y 1}. LetA = {(x, y) x 1 2, y 1 2 }.Then P (A) = P ( X 1 2, Y 1 2 Furthermore, f X (x) = 1 x = 1/2 y 2dxdy = 1 4 2dy =2(1 x), x 1 and f Y (y) = ) ( ) = P X Y, Y 1 2 y 2dx =2y, x 1.

3 3 Independent Random Variables The random variables X and Y are independent iff their joint probability function is the product of their marginal distribution functions, that is, f(x, y) =f X (x)f Y (y), x, y More generally, the random variables X 1,X 2,,X n are mutually independent iff their joint probability function is the product of their marginal probability (density) functions, i.e., f(x 1,x 2,,x n )=f X1 (x 1 )f X2 (x 2 ) f Xn (x n ), x 1,x 2,,x n (1) Let X 1 and X 2 be independent Poisson random variables with respective means λ 1 =2 and λ 2 =3. Then (a) P (X 1 =3,X 2 =5)=P (X 1 =3)P (X 2 =5)= e e ! 5! (b) P (X 1 + X 2 =1)=P (X 1 =1)P (X 2 =)+P (X 1 =)P (X 2 =1)= e e e 2 2 e !! 1! 1! (2) Let X 1 b(3,.8) and X 2 b(5,.7) be independent binomial random variables. Then ( ) 3 (a) P (X 1 =2,X 2 =4)=P (X 1 =2)P (X 2 =4)= (.8) 2 (1.8) 3 2 ( ) 5 (.7) 4 (1.7) 5 4 (b) ( P (X 1 ) + X 2 = 7) = P (X( 1 = ) 2)P (X 2 = 5) + P ((X 1 ) = 3)P (X 2 = 4) = (.8) 2 2 (1.8) 3 2 (.7) 5 5 (1.7) (.8) 3 3 (1.8) 3 3 ( ) 5 (.7) 4 4 (1.7) 5 4 (3) Let X 1 and X 2 be two independent randome variables having the same exponential distribution with p.d.f. f(x) =2e 2x, <x<. Then (a) E[X 1 ]=E[X 2 ]=.5 ande[(x 1.5) 2 ]=E[(X 2.5) 2 ]=.25. ( 1. ) ( 1.2 ) (b) P (.5 <X 1 < 1.,.7 <X 2 < 1.2) = 2e 2x dx 2e 2x dx (c) E[X 1 (X 2.5) 2 ]=E[X 1 ]E[(X 2.5) 2 ]=.5.25 =

4 4 Covariance and Correlation Coefficient For artibrary random variables X and Y, and constants a and b, wehave E[aX + by ]=ae[x]+be[y ] Proof: We ll show for the continuous case, the discrete case can be similarly proved. E[aX + by ] = = = (ax + by)f(x, y)dxdy axf(x, y)dxdy + [ ] ax f(x, y)dy dx + byf(x, y)dxdy [ ] by f(x, y)dx dy Similarly, Furthermore, = a xf X (x)dx + b yf Y (y)dy = ae[x]+be[y ] ( n ) n E a i X i = a i E(X i ) i=1 i=1 E[XY ]= [Example] Letf(x, y) = 1 (x + y), 3 E[XY ]= 1 2 xyf(x, y)dxdy <x<1, <y<2, and f(x, y) =elsewhere. xyf(x, y)dydx = 1 2 xy 1 3 (x + y)dydx = 2 3 Let X and Y be independent random variables, then [ ] [ E(XY )= xyf X (x)f Y (y)dxdy = xf X (x)dx The covariance between r.v. s X and Y is defined as Cov(X, Y )=E[(X μ X )(Y μ Y )] = If X and Y are independent r.v.s, then Cov(X, Y )=. The correlation coefficient is defined by ρ(x, Y )= Cov(X,Y ) σ X σ Y ] yf Y (y)dy = E(X) E(Y ) (x μ X )(y μ Y )f(x, y)dydx = E(XY ) μ X μ Y

5 5 Expectation and Covariance Matrix Let X 1,X 2,...,X n be random variables such that the expectation, variance, and covariance are defined as follows. μ j = E(X j ), σ 2 j = Var(X j)=e[(x j μ j ) 2 ] Cov(X i,x j )=E[(X i μ i )(X j μ j )] = ρ ij σ i σ j Suppose that X =[X 1,X 2,...,X n ] t is a random vector, then the expected mean vector and covariance matrix of X is defined as E(X) = [μ 1,μ 2,...,μ n ] t = μ Cov(X) = E[(X μ)(x μ) t ] = [E((X i μ i )(X j μ j ))] Theorem 1: Let X 1,X 2,...,X n be n independent r.v. s with respective means {μ i } and variances {σ 2 i },theny = n i=1 a i X i has mean μ Y = n i=1 a i μ i and variance σ 2 Y = ni=1 a 2 i σ2 i, respectively. Theorem 2: Let X 1,X 2,...,X n be n independent r.v. s with respective moment-generating functions {M i (t)}, 1 i n, then the moment-generating function of Y = n i=1 a i X i is M Y (t) = n i=1 M i (a i t).

6 6 Multivariate (Normal) Distributions (Gaussian) Normal Distribution: X N(u, σ 2 ) f X (x) =f(x) = 1 2πσ 2 exp (x u)2 /2σ 2 for <x< mean and variance : E(X) =u, V ar(x) =σ 2 (Gaussian) Normal Distribution: X N(u,C) f X (x) =f(x) = 1 (2π) d/2 [det(c)] 1/2 e (x u)t C 1 (x u)/2 for x R d mean vector and covariance matrix : E(X) =u, Cov(X) =C Simulate X N(u, C) (1) C = LL t,wherel is lower-δ. (2) Generate y N(,I). (3) x = u + L y (4) Repeat Steps (2)and(3) M times. % Simulate N([1 3], [4,2; 2,5]) % n=3; X1=random( normal,,1,n,1); X2=random( normal,,1,n,1); Y=[ones(n,1), 3*ones(n,1)]+[X1,X2]*[2 1;, 2]; Yhat=mean(Y) % estimated mean vector Chat=cov(Y) % estimated covariance matrix % Z=[X1, X2];

7 7 Plot a 2D standard Gaussian Distribution x=-3.6:.3:3.6; y=x ; X=ones(length(y),1)*x; Y=y*ones(1,length(x)); Z=exp(-(X.^2+Y.^2)/2+eps)/(2*pi); mesh(z); title( f(x,y)= (1/2\pi)*exp[-(x^2+y^2)/2.] ) f(x,y)= (1/2π)*exp[ (x 2 +y 2 )/2.]

8 8 Some Practical Examples (1) Let X 1, X 2,andX 3 be independent r.v.s from a geometric distribution with p.d.f. ( )( ) 3 1 x 1 f(x) =, x =1, 2, 4 4 Then (a) P (X 1 =1,X 2 =3,X 3 =1) = P (X 1 =1)P (X 2 =3)P (X 3 =1) = f(1)f(3)f(1) (b) = ( ) 3 ( ) = P (X 1 + X 2 + X 3 =5) = 3P (X 1 =3,X 2 =1,X 3 =1)+3P (X 1 =2,X 2 =2,X 3 =1) = (c) Let Y = max{x 1,X 2,X 3 },then P (Y 2) = P (X 1 2)P (X 2 2)P (X 3 2) = ( ) = ( ) (2) Let the random variables X and Y have the joint density function Then f(x, y) = xe xy x, x >, y> f(x, y) = elsewhere (a) f X (x) = xe xy x dx = e x,x>; μ X =1,σX 2 =1. (b) f Y (y) = 1, y > ; μ (1+y) 2 Y = lim y [ln(1 + y) 1] does not exist. (c) X and Y are not independent since f(x, y) f X (x)f Y (y).

9 9 (d) P (X + Y 1) = 1 ( 1 x xe xy x dy ) dx = 1 (e x e 2x+x2 )dx = 1 e x dx e 1 [ 1 e1 2x+x2 dx] = 1 e 1 e 1 ( 1 et2 dt) (3) Let (X, Y ) be uniformly distributed over the unit circle {(x, y) :(x 2 + y 2 ) 1}. Its joint p.d.f is given by f(x, y) = 1 π, x2 + y 2 1 f(x, y) = elsewhere (a) P (X 2 + Y 2 1 )= π π (b) {(x, y) : (x 2 + y 2 ) 1, x>y} is a semicircle, so P (X >Y)= 1. 2 (c) P (X = Y )=. (d) {(x, y) : (x 2 + y 2 ) 1, x<2y} is a semicircle, so P (Y <2X) = 1. 2 (e) Let R = X 2 + Y 2,thenF R (r) =P (R r) =r if r<1, and F R (r) =1ifr 1. (f) Compute f X (x) andf Y (y) andshowthatcov(x, Y ) = but X and Y are not independent.

10 1 Stochastic Process Definition: A Bernoulli trials process is a sequence of independent and identically distributed (iid) Bernoulli r.v. s X 1,X 2,,X n. It is the mathematical model of n repetitions of an experiment under identical conditions, with each experiment producing only two outcomes called success/failure, head/tail, etc. Two examples are described below. (i) Quality control: As items come off a production line, they are inspected for defects. When the ith item inspected is defective, we record X i = 1 and write down X i =otherwise. (ii) Clinical trials: Patients with a disease are given a drug. If the ith patient recovers, we set X i =1andsetX i = otherwise. are mutually independent. A Bernoulli trials process is a sequence of independent and identically distributed (iid) random variables X 1,X 2,,X n,whereeachx i takes on only one of two values, or 1. The number p = P (X i = 1) is called the probability of success, and the number n q =1 p = P (X i = ) is called the probability of failure. The sum T = X i is called the number of successes in n Bernoulli trials, where T b(n, p) hasabinomial distribution. Definition: {X(t), t } is a Poisson process with intensity λ> if (i) For s andt>, the random variable X(s + t) X(s) has the Poisson distribution with parameter λt, i.e., P [X(t + s) X(s) =k] = e λt (λt) k, k =, 1, 2, k! and (ii) For any time points = t <t 1 < <t n, the random variables X(t 1 ) X(t ),X(t 2 ) X(t 1 ),,X(t n ) X(t n 1 ) are mutually independent. The Poisson process is an example of a stochastic process, a collection of random variables indexed by the time parameter t. i=1