To ensure that P{(X, Y ) B} is nonnegative and that it equals one when B is the whole of R 2, we must require

Transcription

1 Page Chapter Joint densities Consider the general problem of describing probabilities involving two random variables, X and Y. If both have discrete distributions, with X taking values x, x 2,... and Y taking values y, y 2,..., then everything about the joint behavior of X and Y can be deduced from the set of probabilities P{X x i, Y y j } for i, 2,... and j, 2,... We have been working for some time with problems involving such pairs of random variables, but we have not needed to formalize the concept of a joint distribution. When both X and Y have continuous distributions, it becomes more important to have a systematic way to describe how one might calculate probabilities of the form P{(X, Y ) B} for various subsets B of the plane. For example, how could one calculate P{X < Y } or P{X 2 + Y 2 9} or P{X + Y 7}? joint density <.> Definition. Say that random variables X and Y have a jointly continuous distribution with joint density function f (, ) if P{(X, Y ) B} {(x,y) B}f(x,y)dx dy for each subset B of R 2. In particular, for a small region around a point (x, y ), at least if f is continuous at (x, y ). P{(x, y) } (area of ) f (x, y ), To ensure that P{(X, Y ) B} is nonnegative and that it equals one when B is the whole of R 2, we must require f and {(x,y) R 2 }f(x,y)dx dy. Apart from the replacement of single integrals by double integrals, and the replacement of intervals of small length by regions of small area, the definition of a joint density is the same as the definition for densities on the real line in Chapter 6. The small region can be chosen in many ways small rectangles, small disks, small blobs, small shapes that don t have any particular name whatever suits the needs of a particular calculation. <.2> Example. When X has density g(x) and Y has density h(y), and X is independent of Y, the joint density is particularly easy to calculate. Let be a small rectangle with one corner at (x, y ) and small sides of length δ x > and δ y > : {(x,y) R 2 :x x x +δ x,y y y +δ y } By independence, P{(X, Y ) } P{x X x +δ x }P{y Y y +δ y }

2 Chapter Joint densities Page 2 Invoke the defining property of the densities g and h to approximate the last product by (g(x )δ x + smaller order terms) ( h(y )δ y + smaller order terms ) δ x δ y g(x )h(y ) marginal densities Thus f (x, y ) g(x )h(y ). That is, the joint density f is the product of the marginal densities g and h. The word marginal is used here to distinguish the joint density for (X, Y ) from the individual densities g and h. When pairs of random variables are not independent it takes more work to find a joint density. The prototypical case, where new random variables are constructed as linear functions of random variables with a known joint density, illustrates a general method for deriving joint densities. <.3> Exercise. Suppose X and Y have a jointly continuous distribution with joint density f (x, y). For constants a, b, c, d with ad bc define U ax +by and V cx + dy Find the joint density function ψ(u,v) for (U, V ). Solution: Think of the pair (U, V ) are defining a new random point in R 2. That is (U, V ) T (X, Y ), where T maps the point (x, y) R 2 to the point (u,v) R 2 with or in matrix notation, u ax + by and v cx + dy, a c (u, v)(x,y)a where A b d Notice that det A ad bc. The assumption that ad bc ensures that the transformation is invertible: (u, v)a (x,y) where A d c ad bc b a That is, du bv ad bc x and cu + av ad bc y Notice that det A /(ad bc) /(det A) It helps to distinguish between the two roles for R 2, referring to the domain of T as the (X, Y )-plane and the range as the (U, V )-plane. The joint density function ψ(u,v) is characterized by the property that P{u U u + δ u,v V v+δ v } ψ(u,v )δ u δ v for each (u,v ) in the (U, V )-plane, and small (δ u,δ v ). To calculate the probability on the left-hand side we need to find the region R in the (X, Y )-plane corresponding to the small rectangle, with corners at (u,v ) and (u + δ u,v +δ v ),inthe(u,v)-plane. The linear transformation A maps parallel straight lines in the (U, V )-plane into parallel straight lines in the (X, Y )-plane. The region R must be a parallelogram, with vertices (x, y + δ y ) (u,v +δ v )A and (x + δ x, y + δ y ) (u + δ u,v +δ v )A More succinctly, (x,y )(u,v )A and (x + δ x, y ) (u + δ u,v )A (δ x,δ y )(δ u,δ v )A δ u α u +δ v α v where A has rows α u and α v.

3 Chapter Joint densities Page 3 (X,Y)-plane (U,V)-plane (x, y )+δ v α v (x, y )+δ u α u +δ v α v (x, y ) R (x, y )+δ u α u (u,v ) (u +δ u,v +δ v ) From the formula in the Appendix, the parallelogram R has area ( det δu α u,δ vα v) δu δ v det ( A ) δ u δ v det A For small δ u > and δ v >, ψ(u,v )δ u δ v P{(U, V ) } P{(X, Y ) R} (area of R) f (x, y ) δ u δ v f (x, y )/ det(a) It follows that (U, V ) have joint density ψ(u,v) f (x, y) det A where (x, y) (u,v)a In effect, we have calculated a Jacobian by first principles. <.4> Example. Suppose X and Y are independent random variables, each distributed N(, ). By Example <.2>, the joint density for (X, Y ) equals f (x, y) ( 2π exp x 2 + y 2 ) 2 By Exercise <.3>, the joint distribution of the random variables U ax +by and V cx + dy has the joint density ( ψ(u,v) 2π(ad bc) exp du bv 2 ) cu + av 2 2 ad bc 2 ad bc ( 2π(ad bc) exp (c2 + d 2 )u 2 2(db + ac)uv + (a 2 + b 2 )v 2 ) 2(ad bc) 2 You ll learn more about joint normal distributions in Chapter 2. The calculations in Exercise <.3> for linear transformations gives a good approximation for more general smooth transformations when applied to small regions. Densities describe the behaviour of distributions in small regions; in small regions smooth transformations are approximately linear; the density formula for linear transformations gives the density formula for smooth transformations in small regions. <.5> Exercise. Suppose X and Y are independent random variables, with X having a gamma(α) distribution and Y having a gamma(β) distribution. Show that X/(X + Y ) has a beta(α, β) distribution, independent of X + Y, which has a gamma(α + β) distribution. Solution: Write U for X/(X +Y ) and V for X +Y. The pair (X, Y ) takes values ranging over the positive quadrant (, ) 2, with joint density function f (x, y) x α e x Ɣ(α) yβ e y Ɣ(β) for x >, y >.

4 Chapter Joint densities Page 4 The pair (U, V ) takes values in the strip (, ) (, ). That is, < U < and < V <. The joint density function, ψ(u,v), for (U, V ) remains to be determined. Consider ψ(, ) near a point (u,v ) in the strip. If U u and V v then X UV u v and Y V UV ( u )v Moreover, (U, V ) lies near (u,v ) when (X, Y ) lies near (x, y ), where Jacobian matrix x u v and y ( u )v Notice how each (u,v ) with < u < and <v < corresponds to a unique (x, y ) with < x < and < y <. For small positive δ u and δ v, determine the region R in the (X, Y ) quadrant corresponding to the small rectangle {(u,v):u u u +δ u,v v v +δ v } in the (U, V ) strip. First locate the points corresponding to the corners of. (u + δ u,v ) (x, y ) + (δ u v, δ u v ) (u,v +δ v ) (x, y ) + (δ v u,δ v ( u )) (u + δ u,v +δ v ) (x, y ) + (δ u v + δ v u, δ u v + δ v ( u )) + (δ u δ v, δ u δ v ) In matrix notation, v v (u,v )+(δ u, ) (x, y ) + (δ u, )J where J u u (u,v )+(,δ v ) (x, y ) + (,δ v )J (u,v )+(δ u,δ v ) (x, y ) + (δ u,δ v )J + smaller order terms You might recognize J as the Jacobian matrix of partial derivatives ( x y ) u u x y v v evaluated at (u,v ). For small perturbations, the transformation from (u,v) to (x, y) is approximately linear. (X,Y)-quadrant (U,V)-strip (x, y ) R v +δ v v u u +δ u The region R is approximately a rectangle, with the edges oblique to the coordinate axes. To a good approximation, the area of R is equal to δ u δ v times the area of the rectangle with corners at (, ) and a (v, v ) and b (u, u ) and a + b From the Appendix, the area of this rectangle equals det(j) v. The rest of the calculation of the joint density ψ(, ) for (U, V ) is easy: δ u δ v ψ(u,v ) P{(U,V) } P{(X,Y) R}

5 Chapter Joint densities Page 5 f (x, y )(area of R) x α e x y β e y δ u δ v v Ɣ(α) Ɣ(β) Substitute x u v and y ( u )v, then rearrange factors, to get the joint density If we write ψ(u,v ) uα v α e u v Ɣ(α) ( u ) β v β e v +u v Ɣ(β) g(u) uα ( u) β the beta(α, β) density B(α, β) h(v) vα+β e v the gamma(α + β) density Ɣ(α + β) then B(α, β)ɣ(α + β) ψ(u,v) g(u)h(v) for < u < and <v< Ɣ(α)Ɣ(β) I have dropped the subscripting zeros because I no longer need to keep your attention fixed on a particular (u,v ) in the (U, V ) strip. The jumble of constants involving beta and gamma functions must reduce to the constant, because P{ < U <, < V < } { < u <, <v< }ψ(u,v)du dv v beta vs. gamma B(α, β)ɣ(α + β) g(u)du h(v) dv Ɣ(α)Ɣ(β) Notice how the double integral has split into a product of two single integrals because the joint density factorized into a product of a function of u and a function of v. Both the single integrals equal because both g and h are density functions. We have earned a bonus, B(α, β) Ɣ(α)Ɣ(β) Ɣ(α + β) for α> and β> which is a useful expression relating beta and gamma functions. The factorization of the joint density implies that the random variables U and V are independent. To see why, consider any pair of subsets A and B of the real line. The defining property of the joint density gives P{U A} P{U A,<V < } {u A, <v< }ψ(u,v)du dv {u A}ψ U (u)du where ψ U (u) ψ(u,v)dv. That is, we get the marginal density ψ U (u) for U by integrating the joint density with respect to V over its whole range. Specifically, ψ U (u) g(u)h(v)dv g(u) That is, U has a beta(α, β) distribution. Similarly, V has a continuous distribution with density ψ V (v) ψ(u,v)du That is, V has a gamma(α + β) distribution. g(u)h(v) du h(v)

6 Chapter Joint densities Page 6 Finally, P{U A, V B} {u A,v B}ψ(u,v)du dv {u A}g(u)du {v A}h(v) dv P{U A}P{V B} The events {U A} and {V B} are independent, for all choices of A and B. That is, the random variables U and V are independent. marginal densities from joint density Remarks: In Chapter 9 we discovered that Ɣ(/2) π. This fact also follow from the equality Ɣ(/2)Ɣ(/2) B(/2, /2) Ɣ() π π/2 t /2 ( t) /2 dt 2 sin(θ) cos(θ) dθ sin(θ) cos(θ) putting t sin2 (θ) Remarks: It is worthwhile to remember the method for deriving marginal densities from a joint density: In general, if X and Y have a jointly continuous distribution with density function f(x,y) then the (marginal) distribution of X is continuous, with (marginal) density f (x, y) dy, and the (marginal) distribution of Y is continuous, with (marginal) density f (x, y) dx, Remember that the word marginal is redundant; it serves merely to stress that a calculation refers only to one of the random variables. <.6> Example. If X, X 2,...,X k are independent random variables, with X i distributed gamma(α i ) for i,...,k, then X + X 2 gamma(α + α 2 ), X + X 2 + X 3 (X + X 2 ) + X 3 gamma(α + α 2 + α 3 ) X + X 2 + X 3 + X 4 (X + X 2 + X 3 ) + X 4 gamma(α + α 2 + α 3 + α 4 )... X + X X k gamma(α + α α k ) A particular case has great significance for Statistics. Suppose Z,...Z k are independent random variables, each distributed N(,). From Chapter 9, the random variables Z 2/2,...,Z2 k /2 are independent gamma(/2) distributed random variables. The sum (Z Z2 k )/2 must have a gamma(k/2) distribution with density t k/2 e t /Ɣ(k/2) for t >. The sum Z Z2 k has density (t/2) k/2 e t/2 2Ɣ(k/2) for t >

7 Chapter Joint densities Page 7 chi-squared This distribution is called the chi-squared on k degrees of freedom, usually denoted by. The letter χ is a lowercase Greek chi. χ 2 k Appendix: area of a parallelogram Let R be a rectangle in the plane R 2 with corners at (, ), a (a, a 2 ), b (b, b 2 ), and a + b. The area of R is equal to the absolute value of the determinant of the matrix a b J (a, b) a 2 b 2 θ b Proof. Let θ denotes the angle between a and b. Remember that a b cos(θ) a b a a+b With the side from to a, which has length a, as the base, the vertical height is b sin θ. The absolute value of the area equals a b sin θ. The square of the area equals a 2 b 2 sin 2 (θ) a 2 b 2 a 2 b 2 cos 2 (θ) (a a)(b b) (a b) 2 a a a b det a b b b det J J (det J) 2 If you are not sure about the properties of determinants used in the last two lines, you should rewrite the area as an explicit function of a, a 2, b 2, b 2 then grind out the algebra. If you know about Jacobians you should recognize what was going on in the proof.