General Bivariate Normal Lecture : Bivariate Normal Distribution Statistics 04 Colin Rundel April, 0 Let Z, Z N 0,, which we will use to build a general bivariate normal distribution. f z, z π exp z + z We want to transform these unit normal distributions to have the follow arbitrary parameters: µ, µ Y,, σ Y, ρ Z + µ Y σ Y ρz + ρ Z + µ Y Statistics 04 Colin Rundel Lecture April, 0 / General Bivariate Normal - Marginals General Bivariate Normal - Cov/Corr First, lets examine the marginal distributions of and Y, Second, we can find Cov, Y and ρ, Y Z + µ N 0, + µ N µ, σ Y σ Y ρz + ρ Z + µ Y σ Y ρn 0, + ρ N 0, + µ Y σ Y N 0, ρ + N 0, ρ + µ Y σ Y N 0, + µ Y N µ Y, σy Cov, Y E E Y EY E Z + µ µ σ Y ρz + ρ Z + µ Y µ Y E Z σ Y ρz + ρ Z σ Y E ρz + ρ Z Z σ Y ρez σ Y ρ ρ, Y Cov, Y σ Y ρ Statistics 04 Colin Rundel Lecture April, 0 / Statistics 04 Colin Rundel Lecture April, 0 3 /
General Bivariate Normal - RNG Multivariate Change of Variables Consequently, if we want to generate a Bivariate Normal random variable with N µ, σ and Y N µ Y, σy where the correlation of and Y is ρ we can generate two independent unit normals Z and Z and use the transformation: Z + µ Y σ Y ρz + ρ Z + µ Y We can also use this result to find the joint density of the Bivariate Normal using a d change of variables. Let,..., n have a continuous joint distribution with pdf f defined of S. We can define n new random variables Y,..., Y n as follows: Y r,..., n Y n r n,..., n If we assume that the n functions r,..., r n define a one-to-one differentiable transformation from S to T then let the inverse of this transformation be x s y,..., y n x n s n y,..., y n Then the joint pdf g of Y,..., Y n is { f s,..., s n J for y,..., y n T gy,..., y n 0 otherwise Where s y J det....... s n y s y n s n y n Statistics 04 Colin Rundel Lecture April, 0 4 / Statistics 04 Colin Rundel Lecture April, 0 5 / General Bivariate Normal - Density General Bivariate Normal - Density The first thing we need to find are the inverses of the transformation. If x r z, z and y r z, z we need to find functions h and h such that Z s, Y and Z s, Y. Next we calculate the Jacobian, J det s x s x s y s y det 0 ρ ρ σ Y ρ σ Y ρ Therefore, Y µ Y σ Y Z + µ Z µ Y σ Y ρz + ρ Z + µ Y ρ µ + ρ Z Y µy Z ρ σ Y ρ µ s x, y x µ y µy s x, y ρ σ Y ρ x µ The joint density of and Y is then given by f x, y f z, z J π exp z + z J π σ Y ρ exp π σ Y ρ exp z + z x µ + y µy ρ σ Y x µ π σ Y ρ exp / ρ σ + y µ Y σ Y ρ x µ ρ x µ y µ Y σ Y Statistics 04 Colin Rundel Lecture April, 0 6 / Statistics 04 Colin Rundel Lecture April, 0 7 /
General Bivariate Normal - Density Matrix Notation Obviously, the density for the Bivariate Normal is ugly, and it only gets worse when we consider higher dimensional joint densities of normals. We can write the density in a more compact form using matrix notation, x µ σ x µ Σ ρ σ Y y µ Y ρ σ Y f x π det Σ / exp x µt Σ x µ We can confirm our results by checking the value of det Σ / and x µ T Σ x µ for the bivariate case. det Σ / σ σ Y ρ σ / σ Y σ Y ρ / σ Y General Bivariate Normal - Density Matrix Notation Recall for a matrix, Then, a b A c d A det A d b c a x µ T Σ x µ T x µx σ σ Y σ Y ρ y µ y ρ σ Y σ σ Y ρ ad bc ρ σ Y σ σ Y x µ ρ σ Y y µ Y ρ σ Y x µ + σ y µ Y d b c a x µx y µ y T x µx y µ y σ σ σ Y ρ Y x µ ρ σ Y x µ y µ Y + σ y µ Y x µ ρ σ ρ x µ y µ Y + y µ Y σ Y σy Statistics 04 Colin Rundel Lecture April, 0 8 / Statistics 04 Colin Rundel Lecture April, 0 9 / N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, ρ 0 ρ 0 ρ 0 ρ 0.5 ρ 0.5 ρ 0.75 Statistics 04 Colin Rundel Lecture April, 0 0 / Statistics 04 Colin Rundel Lecture April, 0 /
N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, N 0,, Y N 0, ρ 0.5 ρ 0.5 ρ 0.75 ρ 0.75 ρ 0.75 ρ 0.75 Statistics 04 Colin Rundel Lecture April, 0 / Statistics 04 Colin Rundel Lecture April, 0 3 / Multivariate Normal Distribution Multivariate Normal Distribution - Cholesky Matrix notation allows us to easily express the density of the multivariate normal distribution for an arbitrary number of dimensions. We express the k-dimensional multivariate normal distribution as follows, N k µ, Σ where µ is the k column vector of means and Σ is the k k covariance matrix where {Σ} i,j Cov i, j. The density of the distribution is f x π k/ det Σ / exp x µt Σ x µ In the bivariate case, we had a nice transformation such that we could generate two independent unit normal values and transform them into a sample from an arbitrary bivariate normal distribution. There is a similar method for the multivariate normal distribution that takes advantage of the Cholesky decomposition of the covariance matrix. The Cholesky decomposition is defined for a symmetric, positive definite matrix as L Chol where L is a lower triangular matrix such that LL T. Statistics 04 Colin Rundel Lecture April, 0 4 / Statistics 04 Colin Rundel Lecture April, 0 5 /
Multivariate Normal Distribution - RNG Let Z,..., Z k N 0, and Z Z,..., Z k T then µ + CholΣZ N k µ, Σ this is offered without proof in the general k-dimensional case but we can check that this results in the same transformation we started with in the bivariate case and should justify how we knew to use that particular transformation. Cholesky and the Bivariate Transformation We need to find the Cholesky decomposition of Σ for the general bivariate case where σ Σ ρ σ Y ρ σ Y σy We need to solve the following for a, b, c a 0 b c a b 0 c a ab ab b + c σ ρ σ Y ρ σ Y σy This gives us three unique equations and three unknowns to solve for, a σ ab ρ σ Y b + c σ Y a b ρ σ Y /a ρσ Y c σy b σ Y ρ / Statistics 04 Colin Rundel Lecture April, 0 6 / Statistics 04 Colin Rundel Lecture April, 0 7 / Cholesky and the Bivariate Transformation Conditional Expectation of the Bivariate Normal Let Z, Z N 0, then Y µ + CholΣZ µ µ Y µ µ Y σ 0 + + µ + Z ρσ Y σ Y ρ / Z Z ρσ Y Z + σ Y ρ / Z Y µ Y + σ Y ρz + ρ / Z Z Using µ + Z and Y µ Y + σ Y ρz + ρ / Z where Z, Z N 0, we can find EY. EY x E µ Y + σ Y ρz + ρ / Z x E µ Y + σ Y ρ x µ + ρ / Z x µ Y + σ Y ρ x µ + ρ / EZ x x µ µ Y + σ Y ρ By symmetry, y µy E Y y µ + ρ σ Y Statistics 04 Colin Rundel Lecture April, 0 8 / Statistics 04 Colin Rundel Lecture April, 0 9 /
Conditional Variance of the Bivariate Normal Using µ + Z and Y µ Y + σ Y ρz + ρ / Z where Z, Z N 0, we can find VarY. VarY x Var µ Y + σ Y ρz + ρ / Z x Var µ Y + σ Y ρ x µ + ρ / Z x Varσ Y ρ Z x σ Y ρ Example - Husbands and Wives Example 5.0.6, degroot Suppose that the heights of married couples can be explained by a bivariate normal distribution. If the wives have a mean heigh of 66.8 inches and a standard deviation of inches while the heights of the husbands have a mean of 70 inches and a standard deviation of inches. The correlation between the heights is 0.68. What is the probability that for a randomly selected couple the wife is taller than her husband? By symmetry, Var Y y σ ρ Statistics 04 Colin Rundel Lecture April, 0 0 / Statistics 04 Colin Rundel Lecture April, 0 / Example - Conditionals Example - Conditionals, cont. Suppose that and have a bivariate normal distribution where E 3.7 0.5, E 0.4 0.6, and Var 3.64. Find E, Var, E, Var, and ρ,. Statistics 04 Colin Rundel Lecture April, 0 / Statistics 04 Colin Rundel Lecture April, 0 3 /