Jointly Distributed Random Variables

Transcription

1 Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables Definition Joint cdfs Joint Probability Mass Functions Joint Probability Density Functions Independence and Expectation Independence Expectation Conditional Expectation Covariance and Correlation Examples 6 1 Jointly Distributed Random Variables 1.1 Definition Joint Probability Distribution Suppose we have two random variables X and Y defined on a sample space S with probability measure P(E), E S. Jointly, they form a map (X, Y) : S R 2 where s (X(s), Y(s)). From before, we know to define the marginal probability distributions P X and P Y by, for example, P X (B) = P(X 1 (B)), B R. We now define the joint probability distribution P XY by P XY (B X, B Y ) = P{X 1 (B X ) Y 1 (B Y )}, B X, B Y R. So P XY (B X, B Y ), the probability that X B X and Y B Y, is given by the probability P of the set of all points in the sample space that get mapped both into B X by X and into B Y by Y. More generally, for a single region B XY R 2, find the collection of sample space elements and define S BXY = {s S (X(s), Y(s)) B XY } P XY (B XY ) = P(S BXY ). 1

2 1.2 Joint cdfs Joint Cumulative Distribution Function We thus define the joint cumulative distribution function as F XY (x, y) = P XY ((, x], (, y]), x, y R. It is easy to check that the marginal cdfs for X and Y can be recovered by and that the two definitions will agree. F X (x) = F XY (x, ), x R, F Y (y) = F XY (, y), y R, Properties of a Joint cdf For F XY to be a valid cdf, we need to make sure the following conditions hold F XY (x, y) 1, x, y R; 2. Monotonicity: x 1, x 2, y 1, y 2 R, x 1 < x 2 F XY (x 1, y 1 ) F XY (x 2, y 1 ) and y 1 < y 2 F XY (x 1, y 1 ) F XY (x 1, y 2 ); 3. x, y R, F XY (x, ) = 0, F XY (, y) = 0 and F XY (, ) = 1. Interval Probabilities Suppose we are interested in whether the random variable pair (X, Y) lie in the interval cross product (x 1, x 2 ] (y 1, y 2 ]; that is, if x 1 < X x 2 and y 1 < Y y 2. Y y 2 y 1 x 1 x 2 X First note that P XY ((x 1, x 2 ], (, y]) = F XY (x 2, y) F XY (x 1, y). It is then easy to see that P XY ((x 1, x 2 ], (y 1, y 2 ]) is given by F XY (x 2, y 2 ) F XY (x 1, y 2 ) F XY (x 2, y 1 ) + F XY (x 1, y 1 ). 1.3 Joint Probability Mass Functions If X and Y are both discrete random variables, then we can define the joint probability mass function as p XY (x, y) = P XY ({x}, {y}), x, y R. We can recover the marginal pmfs p X and p Y since, by the law of total probability, x, y R p X (x) = y p XY (x, y), p Y (y) = x p XY (x, y). 2

3 Properties of a Joint pmf For p XY to be a valid pmf, we need to make sure the following conditions hold p XY (x, y) 1, x, y R; 2. p XY (x, y) = 1. y x 1.4 Joint Probability Density Functions On the other hand, if f XY : R R R s.t. P XY (B XY ) = f XY (x, y)dxdy, B XY R R, (x,y) B XY then we say X and Y are jointly continuous and we refer to f XY as the joint probability density function of X and Y. In this case, we have F XY (x, y) = y t= x s= f XY (s, t)dsdt, x, y R, By the Fundamental Theorem of Calculus we can identify the joint pdf as f XY (x, y) = 2 x y F XY(x, y). Furthermore, we can recover the marginal densities f X and f Y : f X (x) = d dx F X(x) = d dx F XY(x, ) = d x f XY (s, y)dsdy. dx s= By the Fundamental Theorem of Calculus, and through a symmetric argument for Y, we thus get f X (x) = f XY (x, y)dy, f Y (y) = x= f XY (x, y)dx. Properties of a Joint pdf For f XY to be a valid pdf, we need to make sure the following conditions hold. 1. f XY (x, y) 0, x, y R; 2. x= f XY (x, y)dxdy = 1. 3

4 2 Independence and Expectation 2.1 Independence Independence of Random Variables Two random variables X and Y are independent iff B X, B Y R., P XY (B X, B Y ) = P X (B X )P Y (B Y ). More specifically, two discrete random variables X and Y are independent iff p XY (x, y) = p X (x)p Y (y), x, y R; and two continuous random variables X and Y are independent iff f XY (x, y) = f X (x) f Y (y), x, y R. Conditional Distributions For two r.v.s X, Y we define the conditional probability distribution P Y X by P Y X (B Y B X ) = P XY(B X, B Y ), B X, B P X (B X ) Y R, P X (B X ) = 0. This is the revised probability of Y falling inside B Y given that we now know X B X. Then we have X and Y are independent P Y X (B Y B X ) = P Y (B Y ), B X, B Y R. For discrete r.v.s X, Y we define the conditional probability mass function p Y X by p Y X (y x) = p XY(x, y), x, y R, p X (x) = 0, p X (x) and for continuous r.v.s X, Y we define the conditional probability density function f Y X by f Y X (y x) = f XY(x, y), x, y R. f X (x) [Justification: P(Y y X [x, x + dx)) = P XY([x, x + dx), (, y]) P X ([x, x + dx)) F(y, x + dx) F(y, x) = F(x + dx) F(x) {F(y, x + dx) F(y, x)}/dx = {F(x + dx) F(x)}/dx d{f(y, x + dx) F(y, x)}/dxdy = f (y X [x, x + dx)) = {F(x + dx) F(x)}/dx f (y, x) = f (y x) = lim f (y X [x, x + dx)) = dx >0 f (x).] R. In either case, X and Y are independent p Y X (y x) = p Y (y) or f Y X (y x) = f Y (y), x, y 4

5 2.2 Expectation E{g(X, Y)} Suppose we have a bivariate function of interest of the random variables X and Y, g : R R R. If X and Y are discrete, we define E{g(X, Y)} by E XY {g(x, Y)} = y g(x, y)p XY (x, y). x If X and Y are jointly continuous, we define E{g(X, Y)} by E XY {g(x, Y)} = x= g(x, y) f XY (x, y)dxdy. Immediately from these definitions we have the following: If g(x, Y) = g 1 (X) + g 2 (Y), E XY {g 1 (X) + g 2 (Y)} = E X {g 1 (X)} + E Y {g 2 (Y)}. If g(x, Y) = g 1 (X)g 2 (Y) and X and Y are independent, E XY {g 1 (X)g 2 (Y)} = E X {g 1 (X)}E Y {g 2 (Y)}. In particular, considering g(x, Y) = XY for independent X, Y we have E XY (XY) = E X (X)E Y (Y). 2.3 Conditional Expectation E Y X (Y X = x) In general E XY (XY) = E X (X)E Y (Y). Suppose X and Y are discrete r.v.s with joint pmf p(x, y). If we are given the value x of the r.v. X, our revised pmf for Y is the conditional pmf p(y x), for y R. The conditional expectation of Y given X = x is therefore Similarly, if X and Y were continuous, E Y X (Y X = x) = y p(y x). y E Y X (Y X = x) = y f (y x)dy. In either case, the conditional expectation is a function of x but not the unknown Y. 5

6 2.4 Covariance and Correlation Covariance For a single variable X we considered the expectation of g(x) = (X µ X )(X µ X ), called the variance and denoted σ 2 X. The bivariate extension of this is the expectation of g(x, Y) = (X µ X )(Y µ Y ). We define the covariance of X and Y by σ XY = Cov(X, Y) = E XY [(X µ X )(Y µ Y )]. Correlation Covariance measures how the random variables move in tandem with one another, and so is closely related to the idea of correlation. We define the correlation of X and Y by ρ XY = Cor(X, Y) = σ XY σ X σ Y. Unlike the covariance, the correlation is invariant to the scale of the r.v.s X and Y. It is easily shown that if X and Y are independent random variables, then σ XY = ρ XY = 0. 3 Examples Example 1 Suppose that the lifetime, X, and brightness, Y of a light bulb are modelled as continuous random variables. Let their joint pdf be given by Are lifetime and brightness independent? The marginal pdf for X is f (x) = f (x, y) = λ 1 λ 2 e λ 1x λ 2 y, x, y > 0. = λ 1 e λ 1x. f (x, y)dy = y=0 λ 1 λ 2 e λ 1x λ 2 y dy Similarly f (y) = λ 2 e λ 2y. Hence f (x, y) = f (x) f (y) and X and Y are independent. Example 2 Suppose continuous r.v.s (X, Y) R 2 have joint pdf { 1 f (x, y) = π, x2 + y 2 1 0, otherwise. Determine the marginal pdfs for X and Y. Well x 2 + y 2 1 y 1 x 2. So 1 x 2 1 f (x) = f (x, y)dy = y= 1 x 2 π dy = 2 1 x π 2. Similarly f (y) = 2 1 y π 2. Hence f (x, y) = f (x) f (y) and X and Y are not independent. 6