5 Joint Probability Distributions and Random Samples Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage
Two Discrete Random Variables The probability mass function (pmf) of a single discrete rv X specifies how much probability mass is placed on each possible X value. The joint pmf of two discrete rv s X and Y describes how much probability mass is placed on each possible pair of values (x, y): p (x, y) = P(X = x and Y = y) 2
Two Discrete Random Variables Always: p (x, y) 0 p (x, y) = 1 and So what are events now? A can be any set consisting of pairs of (x, y) values (e.g., A = {(x, y): x + y = 5} or {(x, y): max (x, y) 3}). Then the probability P[(X, Y) A] is obtained by summing the joint pmf over pairs in A: P[(X, Y) A] = p (x, y) 3
Example 1 An insurance agency services customers who have both a homeowner s policy and an automobile policy. For each type of policy, a deductible amount must be specified. For an automobile policy, the choices are $100 and $250, whereas for a homeowner s policy, the choices are 0, $100, and $200. Suppose an individual Bob -- is selected at random from the agency s files. Let X = his deductible amount on the auto policy and Y = his deductible amount on the homeowner s policy. 4
Example 1 cont d What are the possible (X, Y) pairs? Suppose the joint pmf is given by the insurance company in the accompanying joint probability table: What is p(100, 100)? What is P(Y 100)? 5
Example 1 What is the probability that X = 100? What is the probability that X = 250? The marginal pmf of X is (focuses only on X, and doesn t care what Y is): What is the marginal distribution of Y? 6
Two Continuous Random Variables 7
Two Continuous Random Variables Let X and Y be continuous rv s. A joint probability density function f (x, y) for these two variables is a function satisfying f (x, y) 0 and Then for any set A 8
Two Continuous Random Variables In particular, if A is the two-dimensional rectangle {(x, y): a x b, c y d}, then We can think of A as specifying a surface with height f(x, y) above each point (x, y). 9
Two Continuous Random Variables Then P[(X, Y) A] is the volume underneath the joint pdf and above the region A, analogous to the area under a curve in the case of a single rv. P[(X, Y ) A] = volume under density surface above A 10
Example 3 A bank operates both a drive-up facility and a walk-up window. Let X = the proportion of time the drive-up facility is in use (at least one customer served or waiting to be served) & Y = the proportion of time the walk-up window in use Say the manager has given us the joint pdf based on his experience: 11
Example 3 cont d As a good probabilist, you first verify that this is a pdf. 1) f (x, y) 0 (check) 2) Does it integrate to 1? What is the probability that neither facility is busy more than one-quarter of the time? 12
Marginal Random Variables The marginal probability density functions of X and Y, denoted by f X (x) and f Y (y), respectively, are given by 13
Independent Random Variables 14
Independent Random Variables In many situations, information about the observed value of one of the two variables X and Y gives information about the value of the other variable. Here, P(X = 250) = 0.5 and P(X = 100) = 0.5 15
Independent Random Variables Two random variables X and Y are said to be independent if for every pair (x,y): p (x, y) = p X (x) p Y (y) when X and Y are discrete Or f (x, y) = f X (x) f Y (y) when X and Y are continuous If this is not satisfied for all (x, y), then X and Y are said to be dependent. 16
Example 6 What about in the insurance example? Independence of X and Y requires that every entry in the joint probability table be the product of the corresponding row and column marginal probabilities. 17
Conditional Distributions 18
Conditional Distributions Suppose X = the number of major defects in a randomly selected new automobile and Y = the number of minor defects in that same auto. If we learn that the selected car has one major defect, what now is the probability that the car will have at least one minor defect? That is, what is P(Y >= 1 X = 1)? 19
Conditional Distributions Let X and Y be two continuous rv s with joint pdf f (x, y) and marginal X pdf f X (x). Then for any X value x for which f X (x) > 0, the conditional probability density function of Y given that X = x is If X and Y are discrete, replacing pdf s by pmf s in this definition gives the conditional probability mass function of Y when X = x. Notice that the definition of f Y X (y x) parallels that of P(B A), the conditional probability that B will occur, given that A has occurred. 20
Example 1, cont Reconsider the insurance example: What is the conditional pmf of Y given that X = 10? What is the expected value of Y given that X=100? This is called the conditional expectation. 21
More Than Two Random Variables If X 1, X 2,..., X n are all discrete random variables, the joint pmf of the variables is the function p(x 1, x 2,..., x n ) = P(X 1 = x 1, X 2 = x 2,..., X n = x n ) If the variables are continuous, the joint pdf of X 1,..., X n is the function f (x 1, x 2,..., x n ) such that for any n intervals [a 1, b 1 ],..., [a n, b n ], we have: 22
How to find an expected value of a function of two variables If X and Y are jointly distributed rv s with pmf p(x, y) or pdf f (x, y), according to whether the variables are discrete or continuous, then Then the expected value of a function h(x, Y), denoted by E[h(X, Y)] is given by if X and Y are discrete if X and Y are continuous 23
Example You have purchased 5 tickets to a concert for you and 4 friends (one is Bob). Tickets are for seats 1 5 in one row. If the tickets are randomly distributed among you, what is the expected number of seats separating you from Bob? 24
Covariance 25
Covariance When two random variables X and Y are not independent, it is frequently of interest to assess how strongly they are related to one another. The covariance between two rv s X and Y is Cov(X, Y) = E[(X µ X )(Y µ Y )] X, Y discrete X, Y continuous 26
Covariance Since X µ X and Y µ Y are the deviations of the two variables from their respective mean values, the covariance is the expected product of deviations. Note that Cov(X, X) = E[(X µ X ) 2 ] = V(X). If both variables tend to deviate in the same direction (both go above their means or below their means at the same time), then the covariance will be positive. If the opposite is true, the covariance will be negative. If X and Y are not strongly related, the covariance will be near 0. 27
Covariance The covariance depends on both the set of possible pairs and the probabilities of those pairs. Below are examples of 3 types of co-varying : (a) positive covariance; (b) negative covariance; (c) covariance near zero Figure 5.4 28
Example Back to the insurance example: are: from which µ X = Σxp X (x) = 175 and µ Y = 125. What is the covariance between X and Y? Why might the covariance not be a useful measure? 29
Covariance shortcut The following shortcut formula for Cov(X, Y) simplifies the computations. Proposition Cov(X, Y) = E(XY) µ X µ Y According to this formula, no intermediate subtractions are necessary; only at the end of the computation is µ X µ Y subtracted from E(XY). This is analogous to the shortcut for the variance computation we saw earlier. 30
Correlation 31
Correlation Definition The correlation coefficient of X and Y, denoted by Corr(X, Y), ρ X,Y, or just ρ, is defined by It represents a scaled covariance correlation ranges between -1 and 1. 32
Example In the insurance example, E(X 2 ) = 36,250, so = 36,250 (175) 2 = 5625, so σ X = 75 E(Y 2 ) = 22,500, so σ Y 2 = 6875, and σ Y = 82.92. This gives 33
Correlation Propositions 1. Cov(aX + b, cy + d) = a c Cov (X, Y) 2. Corr(aX + b, cy + d) = sgn(ac) Corr(X, Y) 3. For any two rv s X and Y, 1 Corr(X, Y) 1 4. ρ = 1 or 1 iff Y = ax + b for some numbers a and b with a 0. 34
Correlation If X and Y are independent, then ρ = 0, but ρ = 0 does not imply independence. The correlation coefficient ρ is a measure of the linear relationship between X and Y, and only when the two variables are perfectly related in a linear manner will ρ be as positive or negative as it can be. A ρ less than 1 in absolute value indicates only that the relationship is not completely linear, but there may still be a very strong nonlinear relation. 35
Correlation Also, ρ = 0 does not imply that X and Y are independent, but only that there is a complete absence of a linear relationship. When ρ = 0, X and Y are said to be uncorrelated. Two variables could be uncorrelated yet highly dependent because there is a strong nonlinear relationship, so be careful not to conclude too much from knowing that ρ = 0. 36
Example Let X and Y be discrete rv s with joint pmf The points that receive positive probability mass are identified on the (x, y) coordinate system What is the correlation of X and Y? Are X and Y independent? 37
Interpreting Correlation A value of ρ near 1 does not necessarily imply that increasing the value of X causes Y to increase. It implies only that large X values are associated with large Y values. For example, in the population of children, vocabulary size and number of cavities are quite positively correlated, but it is certainly not true that cavities cause vocabulary to grow. Association (a high correlation) is not the same as causation. 38
Interpreting Correlation xkcd.com/552/ 39