Covariance. Lecture 12: Covariance / Correlation & Bivariate Normal. Properties of Covariance. Covariance, cont.

Transcription

1 Covariance Lecture 12: Covariance / Correlation & Bivariate Normal Sta 111 Colin Rundel May 30, 2014 We have previously discussed Covariance in relation to the variance of the sum of two random variables (Review Lecture 8). Var(X + Y ) = Var(X ) + Var(Y ) + 2Cov(X, Y ) Specifically, covariance is defined as Cov(X, Y ) = E[(X E(X ))(Y E(Y ))] = E(XY ) E(X )E(Y ) this is a generalization of variance to two random variables and generally measures the degree to which X and Y tend to be large (or small) at the same time or the degree to which one tends to be large while the other is small (positive or negative linear association). Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Covariance, cont. The magnitude of the covariance is not usually informative since it is affected by the magnitude of both X and X. However, the sign of the covariance tells us something useful about the relationship between X and Y. Consider the following conditions: X > µ X and Y > µ Y then (X µ X )(Y µ Y ) will be positive. X < µ X and Y < µ Y then (X µ X )(Y µ Y ) will be positive. X > µ X and Y < µ Y then (X µ X )(Y µ Y ) will be negative. X < µ X and Y > µ Y then (X µ X )(Y µ Y ) will be negative. Properties of Covariance Cov(X, Y ) = E[(X µ x )(Y µ y )] = E(XY ) µ x µ y Cov(X, Y ) = Cov(Y, X ) Cov(X, Y ) = 0 if X and Y are independent Cov(X, c) = 0 Cov(X, X ) = Var(X ) Cov(aX, by ) = ab Cov(X, Y ) Cov(X + a, Y + b) = Cov(X, Y ) Cov(X, Y + Z) = Cov(X, Y ) + Cov(X, Z) Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21

2 Correlation Correlation, cont. Since Cov(X, Y ) depends on the magnitude of X and Y we would prefer to have a measure of association that is not affected by changes in the scales of the random variables. The most common measure of linear association is correlation which is defined as ρ(x, Y ) = Cov(X, Y ) σ X σ Y 1 < ρ(x, Y ) < 1 Where the magnitude of the correlation measures the strength of the linear association and the sign determines if it is a positive or negative relationship. Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Correlation and Independence Given random variables X and Y X and Y are independent = Cov(X, Y ) = ρ(x, Y ) = 0 Example Let X = { 1, 0, 1} with equal probability and Y = X 2. Clearly X and Y are not independent random variables. Cov(X, Y ) = ρ(x, Y ) = 0 = X and Y are independent Cov(X, Y ) = 0 is necessary but not sufficient for independence! Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21

3 Example - Linear Dependence Let X Unif(0, 1) and Y = ax + b for constants a and b. Find Cov(X, Y ) and ρ(x, Y ) Sums of Normal RVs If we let X N(µ x, σ 2 x) and Y N(µ x, σ 2 y ) what is the distribution of X + Y? Hint the MGF for a Normal RV is exp ( µt σ2 t 2). Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Let Z 1, Z 2 N (0, 1), which we will use to build a general bivariate normal distribution. f (z 1, z 2 ) = 1 [ 2π exp 1 ] 2 (z2 1 + z2 2 ) - Marginals First, lets examine the marginal distributions of X and Y, We want to transform these unit normal distributions to have the follow parameters: µ X, µ Y, σ X, σ Y, ρ X = µ X + σ X Z 1 Y = µ Y + σ Y ( ρ Z ρ 2 Z 2 ) Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21

4 - Cov/Corr Second, we can find Cov(X, Y ) and ρ(x, Y ) - RNG Consequently, if we want to generate a Bivariate Normal random variable with X N (µ X, σx 2 ) and Y N (µ Y, σy 2 ) where Corr(X, Y ) = ρ we can generate two independent unit normals Z 1 and Z 2 and use the transformation: X = µ X + σ X Z 1 Y = µ Y + σ Y ( ρz ρ 2 Z 2 ) Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 - Density - Examples The joint density of X and Y is given by [ ( 1 1 (x µx ) 2 f (x, y) = 2πσ X σ Y (1 ρ 2 exp ) 1/2 2(1 ρ 2 ) σ X 2 + (y µ Y ) 2 σ 2 Y 2ρ (x µ )] X ) (y µ Y ). σ X σ Y Alternatively, we can use matrix notation to get a slightly more compact representation ( ) ( ) ( ) x µx σ 2 x = µ = Σ = X ρσ X σ Y y µ Y ρσ X σ Y σ 2 Y f (x) = 1 [ 2π (det Σ) 1/2 exp 1 ] 2 (x µ)t Σ 1 (x µ) ρ = 0 X N (0, 2), Y N (0, 1) ρ = 0 X N (0, 1), Y N (0, 2) ρ = 0 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21

5 - Examples - Examples ρ = 0.25 ρ = 0.5 ρ = 0.75 ρ = 0.25 ρ = 0.5 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 - Examples Conditional Expectation of the Bivariate Normal Using X = µ X + σ X Z 1 and Y = µ Y + σ Y [ρz 1 + (1 ρ 2 ) 1/2 Z 2 ] where Z 1, Z 2 N (0, 1) we can find E(Y X ). X N (0, 2), Y N (0, 1) X N (0, 1), Y N (0, 2) Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21

6 Conditional Variance of the Bivariate Normal Using X = µ X + σ X Z 1 and Y = µ Y + σ Y [ρz 1 + (1 ρ 2 ) 1/2 Z 2 ] where Z 1, Z 2 N (0, 1) we can find Var(Y X ). Example - Husbands and Wives (Example , degroot) Suppose that the heights of married couples can be explained by a bivariate normal distribution. If the wives have a mean heigh of 66.8 inches and a standard deviation of 2 inches while the heights of the husbands have a mean of 70 inches and a standard deviation of 2 inches. The correlation between the heights is What is the probability that for a randomly selected couple the wife is taller than her husband? Sta 111 (Colin Rundel) Lecture 12 May 30, / 21 Sta 111 (Colin Rundel) Lecture 12 May 30, / 21