Notes 11 Autumn 2005

Transcription

1 MAS 08 Probabilit I Notes Autumn 005 Two discrete random variables If X and Y are discrete random variables defined on the same sample space, then events such as X = and Y = are well defined. The joint distribution of X and Y is a list of the probabilities p X,Y (,) = P(X = and Y = ) for all values of and that occur. The list is usuall shown as a two-wa table. This table is called the joint probabilit mass function of X and Y. We abbreviate p X,Y (,) to p(,) if X and Y can be understood from the contet. Eample (Muffins: part ) I have a bag containing 5 chocolate-chip muffins, 3 blueberr muffins and lemon muffins. I choose three muffins from this bag at random. Let X be the number of lemon muffins chosen and Y be the number of blueberr muffins chosen. We calculate the values of the joint probablit mass function. First note that S = 0 C 3 =. Then P(X = 0 and Y = 0) = P(3 chocolate-chip are chosen) = 5 C 3 = 0. P(X = 0 and Y = ) = P( blueberr and chocolate chip are chosen) = 3 C 5 C = 30.

2 The other values are found b similar calculations, giving the following table. Values 0 0 of X 0 5 Values of Y Of course, we check that the probabilities in the table add up to. To obtain the distributions of X and Y individuall, find the row sums and the column sums: these give their probabilit distributions. The marginal probabilit mass function p X of X is the list of probabilities p X () = P(X = ) = p X,Y (,) for all values of which occur. Similarl, the marginal probabilit mass function p Y of Y is given b p Y () = P(Y = ) = p X,Y (,). The distributions of X and Y are said to be marginal to the joint distribution. The are just ordinar distributions. Eample (Muffins: part ) Here the marginal p.m.f. of X is and the marginl p.m.f. of Y is 0 p X () p Y () Therefore E(X) = 3/5 and E(Y ) = 9/0. We can define events in terms of X and Y, such as X < Y or X +Y = 3. To find the probabilit of such an event, find the probabilit of the set of all pairs (,) for which the statement is true. We can also define new random variables as functions of X and Y.

3 Proposition 0 If g is a real function of two variables then E(g(X,Y )) = g(,)p X,Y (,). The proof is just like the proof of Proposition 8. Eample (Muffins: part 3) Put U = X +Y and V = XY. Then E(U) = = 3 = E(X) + E(Y ). However, Theorem 7 E(V ) = = E(X)E(Y ). 5 (a) E(X +Y ) = E(X) + E(Y ) alwas. (b) E(XY ) is not necessaril equal to E(X)E(Y ). Proof (a) E(X +Y ) = = = ( + )p X,Y (,) b Proposition 0 p X,Y (,) + = = E(X) + E(Y ). p X,Y (,) + p X () + p Y () p X,Y (,) p X,Y (,) (b) A single counter-eample is all that we need. In the muffins eample we saw that E(XY ) E(X)E(Y ). Independence Random variables X and Y defined on the same sample space are defined to be independent of each other if the events X = and Y = are independent for all values of and ; that is p X,Y (,) = p X ()p Y () for all and. 3

4 Eample (Muffins: part 4) Here P(X = 0) = 7/5 and P(Y = 0) = 35/ but P(X = 0 and Y = 0) = = P(X = 0)P(Y = 0), so X and Y are not independent of each other. Note that a single pair of values of and where the probabilities do not multipl is enough to show that X and Y are not independent. On the other hand, if I roll a die twice, and X and Y are the numbers that come up on the first and second throws, then X and Y will be independent, even if the die is not fair (so that the outcomes are not all equall likel). If we have more than two random variables (for eample X, Y, Z), we sa that the are mutuall independent if the events that the random variables take specific values (for eample, X = a, Y = b, Z = c) are mutuall independent. Theorem 8 If X and Y are independent of each other then E(XY ) = E(X)E(Y ). Proof E(XY ) = = = [ p X,Y (,) p X ()p Y () = E(X)E(Y ). p X ()] [ p Y () if X and Y are independent Note that the converse is not true, as the following eample shows. Eample Suppose that X and Y have the joint p.m.f. in this table. ] Values of Y 0 Values of X The marginal distributions are 0 p X () 4

5 and 0 p Y () 4 4 Therefore P(X = 0 and Y = ) = 0 but P(X = 0)P(Y = ) = 4 0 so X and Y are not independent of each other. However, E(X) = / and E(Y ) = 0 so E(X)E(Y ) = 0, while so E(XY ) = E(X)E(Y ). Covariance and correlation E(XY ) = = 0, A measure of the joint spread of X and Y is the covariance of X and Y, which is defined b Cov(X,Y ) = E ((X µ X )(Y µ Y )), where µ X = E(X) and µ Y = E(Y ). Theorem 9 (Properties of covariance) (a) Cov(X,X) = Var(X). (b) Cov(X,Y ) = E(XY ) E(X)E(Y ). (c) If a is a constant then Cov(aX,Y ) = Cov(X, ay ) = a Cov(X,Y ). (d) If b is constant then Cov(X,Y + b) = Cov(X + b,y ) = Cov(X,Y ). (e) If X and Y are independent, then Cov(X,Y ) = 0. Proof (a) This follows directl from the definition. (b) Cov(X,Y ) = E ((X µ X )(Y µ Y )) = E (XY µ X Y µ Y X + µ X µ Y ) = E(XY ) + E( µ X Y ) + E( µ Y X) + E(µ X µ Y ) b Theorem 7(a) = E(XY ) µ X E(Y ) µ Y E(X) + µ X µ Y b Theorem 4 = E(XY ) E(X)E(Y ) E(Y )E(X) + E(X)E(Y ) = E(XY ) E(X)E(Y ). 5

6 (c) From part (b), Cov(aX,Y ) = E(aXY ) E(aX)E(Y ). From Theorem 4, E(aXY ) = ae(xy ) and E(aX) = ae(x). Therefore Cov(aX,Y ) = ae(xy ) ae(x)e(y ) = a(e(xy ) E(X)E(Y )) = acov(x,y ). (d) Theorem 4 shows that E(Y + b) = E(Y ) + b, so we have (Y + b) E(Y + b) = Y E(Y ). Therefore Cov(X,Y + b) = E(X E(X))(Y + b E(Y + b)) = E(X E(X))(Y E(Y )) = Cov(X,Y ). (e) From part (b), Cov(X,Y ) = E(XY ) E(X)E(Y ). Theorem 8 shows that E(XY ) E(X)E(Y ) = 0 if X and Y are independent of each other. The covariance of X and Y is the product of the distance of X from its mean and the distance of Y from its mean. Therefore, if X µ X and Y µ Y tend to be either both postive or both negative then Cov(X,Y ) is positive. On the other hand, if X µ X and Y µ Y tend to have opposite signs then Cov(X,Y ) is negative. If Var(X) or Var(Y ) is large then Cov(X,Y ) ma also be large; in fact, multipling X b a constant a multiplies Var(X) b a and Cov(X,Y ) b a. To avoid this dependence on scale, we define the correlation coefficient, corr(x,y ), which is just a normalised version of the covariance. It is defined as follows: corr(x,y ) = Cov(X,Y ) Var(X)Var(Y ). The point of this is the first and last parts of the following theorem. Theorem (Properties of correlation) Let X and Y be random variables. Then (a) corr(x,y ) ; (b) if X and Y are independent, then corr(x,y ) = 0; (c) if Y = mx + c for some constants m 0 and c, then corr(x,y ) = if m > 0, and corr(x,y ) = if m < 0; this is the onl wa in which corr(x,y ) can be equal to ±; (d) corr(x,y ) is independent of the units of measurement in the sense that if X or Y is multilpied b a constant, or has a constant added to it, then corr(x,y ) is unchanged. 6

7 The proof of the first part won t be given here. But note that this is another check on our calculations: if ou calculate a correlation coefficient which is bigger than or smaller than, then ou have made a mistake. Part (b) follows immediatel from part (e) of the preceding theorem. For part (c), suppose that Y = mx + c. Let Var(X) = α. Now we just calculate everthing in sight. Var(Y ) = Var(mX + c) = Var(mX) = m Var(X) = m α Cov(X,Y ) = Cov(X,mX + c) = Cov(X,mX) = mcov(x,x) = mα corr(x,y ) = Cov(X,Y )/ Var(X)Var(Y ) = mα/ m α { + if m > 0, = if m < 0. The proof of the converse will not be given here. Part (d) follows from Theorems 4, 5 and 9. Thus the correlation coefficient is a measure of the etent to which the two variables are linearl related. It is + if Y increases linearl with X; 0 if there is no linear relation between them; and if Y decreases linearl as X increases. More generall, a positive correlation indicates a tendenc for larger X values to be associated with larger Y values; a negative value, for smaller X values to be associated with larger Y values. We call two random variables X and Y uncorrelated if Cov(X,Y ) = 0 (in other words, if corr(x,y ) = 0). The preceding theorem and eample show that we can sa: Independent random variables are uncorrelated, but uncorrelated random variables need not be independent. Eample In some ears there are two tests in the Probabilit class. We can take as the sample space the set of all students who take both tests, and choose a student at random. Put X(s) = the mark of student s on Test and Y (s) = the mark of student s on Test. We would epect a student who scores better than average on Test to do so again on Test, and one who scores worse than average to do so again on Test, so there should be a positive correlation between X and Y. However, we would not epect the Test scores to be perfectl predictable as a linear function of the Test scores. The marks for one particular ear are shown in Figure. The correlation is 0.69 to decimal places. 7

8 Test Test Figure : Marks on two Probabilit tests 8

9 Theorem 0 If X and Y are random variables and a and b are constants then Proof First, Then Var(aX + by ) = a Var(X) + abcov(x,y ) + b Var(Y ). E(aX + by ) = E(aX) + E(bY ) b Theorem 7 = ae(x) + be(y ) b Theorem 4 = aµ X + bµ Y. Var(aX + by ) = E[(aX + by ) E(aX + by )] b definition of variance = E[aX + by aµ X bµ Y ] = E[a(X µ X ) + b(y µ Y )] = E[a (X µ X ) + ab(x µ X )(Y µ Y ) + b (Y µ Y ) ] = E[a (X µ X ) ] + E[ab(X µ X )(Y µ Y )] + E[b (Y µ Y ) ] b Theorem 7 = a E[(X µ X ) ] + abe[(x µ X )(Y µ Y )] + b E[(Y µ Y ) ] b Theorem 4 = a Var(X) + abcov(x,y ) + b Var(Y ). Corollar If X and Y are independent, then (a) Var(X +Y ) = Var(X) + Var(Y ). (b) Var(X Y ) = Var(X) + Var(Y ). 9

10 Mean and variance of binomial Here is the third wa of finding the mean and variance of a binomial distribution, which is more sophisticated than the methods in Notes 8 et easier than both. If X Bin(n, p) then X can be thought of as the the number of successes in n mutuall independent Bernoulli trials, each of which has probabilit p of success. For i =,..., n, let X i be the number of successes at the i-th trial. Then X i Bernoulli(p), so E(X i ) = p and Var(X i ) = pq, where q = p. Moreover, the random variables X,..., X n are mutuall independent. B Theorem 7, B the Corollar to Theorem 0, E(X) = E(X ) + E(X ) + + E(X n ) = p + p + + p } {{ } n times = np. Var(X) = Var(X ) + Var(X ) + + Var(X n ) = pq + pq + + pq } {{ } n times = npq. Note that if also Y Bin(m, p), and X and Y are independent of each other, then Y = X n+ + +X n+m, where X n+,..., X n+m are mutuall independent Bernoulli(p) random variables which are all independent of X,..., X n, so X +Y = X + X n + X n+ + X n+m, which is the sum of n + m mutuall independent Bernoulli(p) random variables, so X +Y Bin(n + m, p). 0