# 4. Joint Distributions of Two Random Variables

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 4. Joint Distributions of Two Random Variables 4.1 Joint Distributions of Two Discrete Random Variables Suppose the discrete random variables X and Y have supports S X and S Y, respectively. The joint distribution of X and Y is given by the set {P(X = x, Y = y)} x SX,y S Y. This joint distribution satisfies P(X = x, Y = y) 0 P(X = x, Y = y)=1 y S Y x S X 1 / 78

2 Joint Distributions of Two Discrete Random Variables When S X and S Y are small sets, the joint distribution of X and Y is usually presented in a table, e.g. Y = 1 Y = 0 Y = 1 1 X = X = / 78

3 4.1.1 Marginal Distributions of Discrete Random Variables Note that by summing along a row (i.e. summing over all the possible values of Y for a particular realisation of X ) we obtain the probability that X takes the appropriate value, i.e. P(X = x) = y S Y P(X = x, Y = y). Similarly, by summing along a column (i.e. summing over all the possible values of X for a particular realisation of Y ) we obtain the probability that Y takes the appropriate value, i.e. P(Y = y) = x S X P(X = x, Y = y). 3 / 78

4 Marginal Distributions of Discrete Random Variables The distributions of X and Y obtained in this way are called the marginal distributions of X and Y, respectively. These marginal distributions satisfy the standard conditions for a discrete distribution, i.e. P(X = x) 0; P(X = x) = 1. x S X 4 / 78

5 4.1.2 Independence of Two Discrete Random Variables Two discrete random variables X and Y are independent if and only if P(X = x, Y = y) = P(X = x)p(y = y), x S X, y S Y. 5 / 78

6 4.1.3 The Expected Value of a Function of Two Discrete Random Variables The expected value of a function, g(x, Y ), of two discrete random variables is defined as E[g(X, Y )] = g(x, y)p(x = x, Y = y). y S Y x S X In particular, the expected value of X is given by E[X ] = xp(x = x, Y = y). y S Y x S X It should be noted that if we have already calculated the marginal distribution of X, then it is simpler to calculate E[X ] using this. 6 / 78

7 4.1.4 Covariance and the Correlation Coefficient The covariance between X and Y, Cov(X, Y ) is given by Cov(X, Y ) = E(XY ) E(X )E(Y ) Note that by definition Cov(X, X ) = E(X 2 ) E(X ) 2 = Var(X ). Also, for any constants a, b Cov(X, ay + b) = acov(x, Y ). The coefficient of correlation between X and Y is given by ρ(x, Y ) = Cov(X, Y ) Var(X )Var(Y ). 7 / 78

8 Properties of the Correlation Coefficient 1. 1 ρ(x, Y ) If ρ(x, Y ) = 1, then there is a deterministic linear relationship between X and Y, Y = ax + b. When ρ(x, Y ) = 1, then a > 0. When ρ(x, Y ) = 1, a < If random variables X and Y are independent, then ρ(x, Y ) = 0. Note that the condition ρ(x, Y ) = 0 is not sufficient for X and Y to be independent (see Example 4.1). 4. If a > 0, ρ(x, ay + b) = ρ(x, Y ), i.e. the correlation coefficient is independent of the units a variable is measured in. If a < 0, ρ(x, ay + b) = ρ(x, Y ). 8 / 78

9 Example 4.1 Discrete random variables X and Y have the following joint distribution: Y = 1 Y = 0 Y = 1 1 X = X = a) Calculate i) P(X > Y ), ii) the marginal distributions of X and Y, iii) the expected values and variances of X and Y, iv) the coefficient of correlation between X and Y. b) Are X and Y independent? 9 / 78

10 Example 4.1 To calculate the probability of such an event, we sum over all the cells which correspond to that event. Hence, P(X > Y )=P(X = 0, Y = 1) + P(X = 1, Y = 1) + +P(X = 1, Y = 0) = / 78

11 Example 4.1 We have Hence, P(X = x) = y S Y P(X = x, Y = y). P(X = 0)= P(X = 1)= 1 y= 1 1 y= 1 P(X = 0, Y = y) = = 1 3 P(X = 1, Y = y) = = 2 3. Note that after calculating P(X = 0), we could calculate P(X = 1) using the fact that P(X = 1) = 1 P(X = 0), since X only takes the values 0 and / 78

12 Example 4.1 Similarly, 1 P(Y = 1)= P(X = x, Y = 1) = = 1 3 P(Y = 0)= x=0 1 P(X = x, Y = 0) = = 1 3 x=0 P(Y = 1)=1 P(Y = 1) P(Y = 0) = / 78

13 Example 4.1 We can calculate the expected values and variances of X and Y from these marginal distributions. E[X ]= E[Y ]= 1 xp(x = x) = = 2 3 x=0 1 y= 1 yp(y = y) = = / 78

14 Example 4.1 To calculate Var(X ) and Var(Y ), we use the formula Var(X ) = E(X 2 ) E(X ) 2. E[X 2 ]= E[Y 2 ]= Thus, 1 x 2 P(X = x) = = 2 3 x=0 1 y= 1 yp(y = y) = ( 1) = 2 3. Var(X )= 2 3 ( 2 3 Var(Y )= = 2 3 ) 2 = / 78

15 Example 4.1 To calculate the coefficient of correlation, we first calculate the covariance between X and Y. We have where E[XY ]= Cov(X, Y ) = E[XY ] E[X ]E[Y ], 1 1 x=0 y= 1 xyp(x = x, Y = y) = ( 1) = / 78

16 Example 4.1 Hence, Cov(X, Y ) = E[XY ] E[X ]E[Y ] = = 0. From the definition of the correlation coefficient Hence, ρ(x, Y ) = 0. Cov(X, Y ) = 0 ρ(x, Y ) = 0. Note that if X and Y are independent, then E[XY ] = E[X ]E[Y ]. However, E[XY ] = E[X ]E[Y ] is not sufficient to ensure that X and Y are independent. 16 / 78

17 Example 4.1 Note that if ρ(x, Y ) 0, then we can immediately state that X and Y are dependent. When ρ(x, Y ) = 0, we must check whether X and Y are independent from the definition. X and Y are independent if and only if P(X = x, Y = y) = P(X = x)p(y = y), x S X, y S Y. To show that X and Y are dependent it is sufficient to show that this equation does not hold for some pair of values x and y. 17 / 78

18 Example 4.1 We have P(X = 0, Y = 1) = 0 P(X = 0)P(X = 1) = It follows that X and Y are dependent. It can be seen that X = Y. ( ) / 78

19 4.1.5 Conditional Distributions and Conditional Expected Values Define the conditional distribution of X given Y = y, where y S Y by {P(X = x Y = y)} x SX. P(X = x Y = y) = P(X = x, Y = y). P(Y = y) Analogously, the conditional distribution of Y given X = x, where x S X is given by {P(Y = y X = x)} y SY. P(Y = y X = x) = P(X = x, Y = y). P(Y = y) Note: This definition is analagous to the definition of conditional probability given in Chapter / 78

20 Conditional Expected Value The expected value of the function g(x ) given Y = y is denoted E[g(X ) Y = y]. We have E[g(X ) Y = y] = x S X g(x)p(x = x Y = y). In particular, E[X Y = y] = x S X xp(x = x Y = y). The expected value of the function g(y ) given X = x is defined analogously. 20 / 78

21 Probabilistic Regression The graph of the probabilistic regression of X on Y is given by a scatter plot of the points {(y, E[X Y = y])} y SY. Note that the variable we are conditioning on appears on the x-axis. Analogously, the graph of the probabilistic regression of Y on X is given by a scatter plot of the points {(x, E[Y X = x])} x SX. These graphs illustrate the nature of the (probabilistic) relation between the variables X and Y. 21 / 78

22 Probabilistic Regression The graph of the probabilistic regression of Y on X, E[Y X = x], may be 1. Monotonically increasing in x. This indicates a positive dependency between X and Y. ρ(x, Y ) will be positive. 2. Monotonically decreasing in x. This indicates a negative dependency between X and Y. ρ(x, Y ) will be negative. 3. Flat. This indicates that ρ(x, Y ) = 0. It does not, however, indicate that X and Y are independent. 4. None of the above (e.g. oscillatory). In this case, X and Y are dependent, but this dependency cannot be described in simple terms such as positive or negative. the correlation coefficient can be 0, positive or negative in such a case (but cannot be -1 or 1). 22 / 78

23 Example 4.2 Discrete random variables X and Y have the following joint distribution: Y = 1 Y = 0 Y = 1 1 X = X = a) Calculate the conditional distributions of X given i) Y = 1, ii) Y = 0, iii) Y = 1. b) Calculate the conditional distributions of Y given i) X = 0, ii) X = 1. c) Draw the graph of the probabilistic regression of i) Y on X, ii) X on Y. 23 / 78

24 Example 4.2 a) i) The distribution of X conditioned on Y = 1 is given by P(X = x Y = 1) = P(X = x, Y = 1), x S X = {0, 1}. P(Y = 1) From Example 4.1, P(Y = 1) = 1/3. Thus P(X = 0, Y = 1) P(X = 0 Y = 1)= 1/3 P(X = 1, Y = 1) P(X = 1 Y = 1)= 1/3 Note that since X only takes the values 0 and 1 = 0 = 1 P(X = 1 Y = 1) = 1 P(X = 0 Y = 1). 24 / 78

25 Example 4.2 ii) Similarly, the distribution of X conditioned on Y = 0 is given by P(X = 0, Y = 0) P(X = 0 Y = 0)= P(Y = 0) P(X = 1 Y = 0)= P(X = 1, Y = 0) P(Y = 0) = 1/3 1/3 = 1 = 0 1/3 = 0 iii) The distribution of X conditioned on Y = 1 is given by P(X = 0, Y = 1) P(X = 0 Y = 1)= P(Y = 1) P(X = 1 Y = 1)= P(X = 1, Y = 1) P(Y = 1) = 0 1/3 = 0 = 1/3 1/3 = 1 25 / 78

26 Example 4.2 b) i) The distribution of Y conditioned on X = 0 is given by P(Y = y X = 0) = P(X = 0, Y = y), y S Y = { 1, 0, 1}. P(X = 0) From Example 4.1, P(X = 0) = 1/3. Thus P(X = 0, Y = 1) P(Y = 1 X = 0)= P(X = 0) P(Y = 0 X = 0)= P(X = 0, Y = 0) P(X = 0) = 0 1/3 = 0 = 1/3 1/3 = 1 Since Y takes the values 1, 0 and 1, it follows that P(Y = 1 X = 0) = 1 P(X = 1 X = 0) P(X = 0 Y = 0) = 0 26 / 78

27 Example 4.2 Similarly, the distribution of Y conditioned on X = 1 is given by P(X = 1, Y = 1) P(Y = 1 X = 1)= P(X = 1) = 1/3 2/3 = 1/2 P(X = 1, Y = 0) P(Y = 0 X = 1)= = 0 P(X = 1) 2/3 = 0 P(Y = 1 X = 1)=1 P(Y = 1 X = 1) P(Y = 0 X = 1) = 1/2 27 / 78

28 Example 4.2 c) i) To plot the probabilistic regression of Y on X, we need first to calculate E[Y X = x], x S X = {0, 1}. We have E[Y X = 0]= y S Y yp(y = y X = 0) = = 0 E[Y X = 1]= y S Y yp(y = y X = 1) = = 0 Since E[Y X = x] is constant, it follows that ρ(x, Y ) = / 78

29 Example 4.2 E[Y X = x] 0 1 x 29 / 78

30 Example 4.2 c) ii) To plot the probabilistic regression of X on Y, we need first to calculate E[X Y = y], y S Y = { 1, 0, 1}. We have E[X Y = 1]= x S X xp(x = x Y = 1) = = 1 E[X Y = 0]= x S X yp(x = x Y = 0) = = 0 E[X Y = 1]= x S X xp(x = x Y = 1) = = 1 Since E[X Y = y] depends on y, it follows that X and Y are dependent. 30 / 78

31 Example 4.2 E[X Y = y] y 31 / 78

32 4.2 Joint Distributions of Two Continuous Random Variables The joint distribution of two continuous random variables can be described by the joint density function f X,Y (x, y). The joint density function f X,Y satisfies the following two conditions f X,Y (x, y) 0 f X,Y (x, y)dxdy=1 In practice, we integrate over the joint support of the random variables X and Y. This is the set of pairs (x, y) for which f X,Y (x, y) > 0 (see Example 4.4). 32 / 78

33 Joint Distributions of Two Continuous Random Variables The definitions used for continuous random variables are analogous to the ones used for discrete random variables. Summations are replaced by integrations. Note that the volume under the density function is equal to 1. The probability that (X, Y ) A is given by P[(X, Y ) A] = f X,Y (x, y)dxdy A 33 / 78

34 4.2.1 Marginal Density Functions for Continuous Random Variables The marginal density function of X can be obtained by integrating the joint density function with respect to Y. f X (x) = f X,Y (x, y)dy. Similarly, the marginal density function of Y can be obtained by integrating the joint density function with respect to X. f Y (y) = f X,Y (x, y)dx. If the joint support of X and Y is not a rectangle, some care should be made regarding the appropriate form of the joint density function according to the variable we are integrating with respect to (see Example 4.4). 34 / 78

35 4.2.2 Independence of Two Continuous Random Variables These marginal density functions satisfy the standard conditions for the density function of a single random variable. The continuous random variables X and Y are independent if and only if f X,Y (x, y) = f X (x)f Y (y), x, y. 35 / 78

36 4.2.3 The Expected Value of the Function of Two Continuous Random Variables The expected value of the function g(x, Y ) is given by E[g(X, Y )] = g(x, y)f X,Y (x, y)dxdy. In particular, the expected value of X is given by E[X ] = xf X,Y (x, y)dxdy. As in the case of discrete variables, the expected values and variances of X and Y can be calculated from the respective marginal distributions. 36 / 78

37 4.2.4 Covariance and Coefficient of Correlation The definitions of the covariance and coefficient of correlation, Cov(X, Y ) and ρ(x, Y ) are identical to the definitions used for discrete random variables. The properties of these measures and the interpretation of the coefficient of correlation are unchanged. 37 / 78

38 Example 4.3 The pair of random variables X and Y has a uniform joint distribution on the triangle A with apexes (0,0), (2,0) and (2,2), i.e. f X,Y (x, y) = c if (x, y) A, otherwise f X,Y (x, y) = 0. The density function is illustrated on the following slide. a) Find the constant c. b) Calculate P(X > 1, Y > 1). c) Find the marginal densities of X and Y. d) Find the expected values and variances of X and Y. e) Find the coefficient of correlation, ρ(x, Y ). f) Are X and Y independent? 38 / 78

39 Example 4.3 y 2 f X,Y (x, y) = 0 f X,Y (x, y) = c 0 2 x 39 / 78

40 Example 4.3 a) Method 1: The clever way Note that if the density function is constant on some set A, then the volume under the density curve (which must be equal to 1) is simply the height of the density curve, c, times the area of the set A. The set A is a triangle with base and height both of length 2. The area of this triangle is thus = 2. It follows that 2c = 1 c = / 78

41 Example 4.3 Method 2 The joint support of X and Y is the triangle A. Hence, f X,Y (x, y)dydx = 1. A We must be careful to get the boundaries of integration correct. Note that in A, x varies from 0 to 2. Fixing x, y can vary from 0 to x. Hence, A 2 f X,Y (x, y)dydx= = x=0 2 x=0 [ x cdy y=0 [cy] x y=0dx = ] dx 2 0 cxdx =c[x 2 /2] 2 0 = 2c c = / 78

42 Example 4.3 It is possible to do the integration in the opposite order. Note that in A, y varies from 0 to 2. Fixing y, x can vary from y to 2. Hence, 2 [ 2 ] f X,Y (x, y)dydx = cdx dy. A y=0 Doing the integration this way obviously leads to the same answer, but the calculation is slightly longer. x=y 42 / 78

43 Example 4.3 Method 1: The Clever Way If (X, Y ) has a uniform distribution over the set A, then P[(X, Y ) B], where B A is given by P[(X, Y ) B] = B A, where A is the area of the set A. 43 / 78

44 Example 4.3 The area of A was calculated to be 2. If both X and Y are greater than 1, then (X, Y ) must belong to the interior of the triangle with apexes (1,1), (2,1) and (2,2). The area of this triangle is 0.5. Hence, P(X > 1, Y > 1) = = / 78

45 Example 4.3 Method 2: By integration When both X and Y are greater than 1, there is only a positive density on the triangle with apexes (1,1), (2,1) and (2,2). In this triangle, 1 x 2. Fixing x, 1 y x. Hence, P(X > 1, Y > 1) = 2 x 1 1 f X,Y (x, y)dydx. 45 / 78

46 Example 4.3 Thus P(X > 1, Y > 1)= = = 2 x dydx [0.5y] x y=1 dx (0.5x 0.5)dx [ x 2 = 4 x ] =(1 1) (1/4 1/2) = 1/4 46 / 78

47 Example 4.3 c) Note that both X and Y take values in [0, 2]. To get the marginal density function at x, f X (x), we integrate the joint density function with respect to y, i.e. we fix x and sum over y. Given x, we should consider for what values of y the joint density is positive. From the graph of the density function, there is a positive density as long as 0 < y < x. We have f X (x) = x y=0 cdy = [cy] x 0 = cx = 0.5x. Note that this holds on the support of X, i.e. 0 x 2. Otherwise f X (x) = / 78

48 Example 4.3 To get the marginal density function at y, f Y (y), we integrate the joint density function with respect to x, i.e. we fix y and sum over x. For a given y, we should consider for what values of x the joint density is positive. From the graph of the density function, there is a positive density as long as y < x < 2. We have f Y (y) = 2 x=y cdx = [cx] 2 y = c(2 y) = 1 0.5y. Note that this holds on the support of Y, i.e. 0 y 2. Otherwise f Y (y) = / 78

49 Example 4.3 d) We can find E(X ) and Var(X ) from the marginal distribution E(X )= E(X 2 )= 2 0 xf X (x)dx = =[x 3 /6] 2 0 = x 2 f X (x)dx = =[x 4 /8] 2 0 = x 2 dx 0.5x 3 dx It follows that Var(X ) = E(X 2 ) E(X ) 2 = 2 ( ) = / 78

50 Example 4.3 Similarly, E(Y )= = E(Y 2 )= = yf Y (y)dy = y(2 y)dy (y 0.5y 2 )dy = [y 2 /2 y 3 /6] 2 0 = 2 3 y 2 f Y (y)dy = y 2 (2 y)dy (y 2 0.5y 3 )dy = [y 3 /3 y 4 /8] 2 0 = 2 3 It follows that Var(Y ) = E(Y 2 ) E(Y ) 2 = 2 3 ( ) = / 78

51 Example 4.3 e) First, we find the covariance, Cov(X, Y ) = E(XY ) E(X )E(Y ). We have E(XY )= A xyf X,Y (x, y)dxdy = 2 = [xy 2 /4] x y=0dx = x=0 =[x 4 /16] 2 0 = x=0 x 3 /4dx It follows that Cov(X, Y ) = 1 4/3 2/3 = 1/9. [ x ] 0.5xydy dx y=0 51 / 78

52 Example 4.3 We have ρ(x, Y )= Cov(X, Y ) Var(X )Var(Y ) 1/9 = = 1/2 2/9 2/9 Since ρ(x, Y ) 0, it follows that X and Y are dependent. 52 / 78

53 Example 4.3 Without calculating the correlation coefficient, we can show that X and Y are dependent from the definition. Intuitively, X and Y are dependent, since the values Y can take depend on the realisation of X. In such cases, it is easiest to show dependence by choosing x and y, such that x S X and y S Y, but (x, y) is not in the joint support, i.e. 0 < x, y < 2, but (x, y) does not lie in the triangle A. (0.5, 1.5) is such a point, thus f X,Y (0.5, 1.5) = 0 f X (0.5)f Y (1.5) > 0. Hence, X and Y are dependent. 53 / 78

54 4.2.5 Conditional Distributions and Conditional Expectations The conditional density function of X given Y = y, where y S Y is given by f X Y =y, where f X Y =y (x) = f X,Y (x, y) f Y (y) The conditional density function of Y given X = x, where x S X is given by f Y X =x, where f Y X =x (y) = f X,Y (x, y) f X (x) 54 / 78

55 Conditional Expectations The conditional expectation of g(x ) given Y = y, where y S Y is given by E[g(X ) Y = y], where In particular, E[g(X ) Y = y] = E[X Y = y] = g(x)f X Y =y (x)dx. xf X Y =y (x)dx. In practice, we integrate over the interval where f X Y =y is positive. The conditional expected value of g(y ) given X = x is defined analogously. 55 / 78

56 Probabilistic Regression for Continuous Random Variables In the case of discrete random variables, the probabilistic regression of Y on X is given by a scatter plot. In the case of continuous random variables, the probabilistic regression of Y on X is given by a curve. The x-coordinate in this plot is given by x, the y-coordinate is given by E[Y X = x]. 56 / 78

57 Probabilistic Regression for Continuous Random Variables The probabilistic regression of X on Y is defined analogously. Since we are conditioning on y, the x-coordinate is given by y. The y-coordinate is given by E(X Y = y). A probabilistic regression curve can be interpreted in an analogous way to the interpretation of probabilistic regression for discrete random variables (see Example 4.4). 57 / 78

58 Example 4.4 For the joint distribution given in Example 4.3 i) Calculate the conditional density function of X given Y = y. Derive E(X Y = y). ii) Calculate the conditional density function of Y given X = x. Derive E(Y X = x). 58 / 78

59 Example 4.4 i) We have f Y (y) = 1 y/2, 0 y 2. Thus S Y = [0, 2). Hence, for y [0, 2) f X Y =y (x) = f X,Y (x, y). f Y (y) Since the joint support of X and Y is not a rectangle, we must be careful with the form of f X,Y (x, y). Here, we are fixing y, there is a positive joint density if y x 2. Hence, for y x 2, Otherwise, f X Y =y (x) = 0. f X Y =y (x) = 1/2 1 y/2 = 1 2 y. 59 / 78

60 Example 4.4 We have f X Y =y (x) = 1/(2 y) for y x 2. Hence, E[X Y = y]= 2 y xf X Y =y (x)dx = [ x 2 = 2(2 y) = ] 2 x=y (2 y)(2 + y) 2(2 y) 2 y = 4 y 2 2(2 y) = 1 + y/2 x 2 y dx Since this regression curve is increasing, it follows that a) X and Y are dependent, b) ρ(x, Y ) > / 78

61 Example 4.4 ii) We have f X (x) = x/2, 0 x 2. Thus S X = (0, 2]. Hence, for x (0, 2] f Y X =x (y) = f X,Y (x, y). f X (x) Again, we must be careful with the form of f X,Y (x, y). Here, we are fixing x, there is a positive joint density if 0 y x. Hence, for 0 y x, f Y X =x (y) = 1/2 x/2 = 1 x. Otherwise, f Y X =x (y) = / 78

62 Example 4.4 We have f Y X =x (y) = 1/x for 0 y x. Hence, x E[Y X = x]= 0 [ y 2 = 2x =x/2 yf Y X =x (y)dy = ] x y=0 = x 2 2x x 0 y x dy Again, since this regression curve is increasing, it follows that a) X and Y are dependent, b) ρ(x, Y ) > / 78

63 4.3 The Multivariate Normal Distribution Let X = (X 1, X 2,..., X k ) be a vector of k random variables. Let E[X i ] = µ i and µ = (µ 1, µ 2,..., µ k ), i.e. µ is the vector of expected values Let Σ be a k k matrix where the entry in row i, column j, Σ i,j is given by Cov(X i, X j ). Σ is called the covariance matrix of the random variables X 1, X 2,..., X k. 63 / 78

64 Properties of the Covariance matrix, Σ 1. Σ is symmetric [since Cov(X i, X j )=Cov(X j, X i )]. 2. Σ i,i =Cov(X i, X i ) =Var(X i ) = σi 2, i.e. the i-th entry on the leading diagonal is the variance of the i-th random variable. 3. Σ is positive definite, i.e. if y is any 1 k vector, yσy T Dividing each element Σ i,j by σ i σ j (where σ i is the standard deviation of the i-th variable), we obtain the correlation matrix R. ρ i,j (the element in row i, column j of this matrix gives the coefficient of correlation between X i and X j. Note ρ i,i = 1 by definition. 64 / 78

65 The Form of the Multivariate Distribution X = (X 1, X 2,..., X k ) has a multivariate normal distribution if and only if the joint density function is given by (in matrix form) f X (x) = (2π) k/2 Σ 1/2 exp[ 1/2(x µ)σ 1 (x µ) T ] where x = (x 1, x 2,..., x n ) and Σ is the determinant of the covariance matrix. If X = (X 1, X 2,..., X k ) has a multivariate normal distribution, then any linear combination of the X i has a (univariate) normal distribution. In particular, each X i has a (univariate) normal distribution. 65 / 78

66 The Bivariate Normal distribution In this case k = 2, X = [X 1, X 2 ], µ = [µ 1, µ 2 ]. The covariance matrix is given by ( σ Σ = 1 2 ) ( ) Cov(X 1, X 2 ) σ 2 Cov(X 1, X 2 ) σ2 2 = 1 ρσ 1 σ 2 ρσ 1 σ 2 σ2 2, where σ i is the standard deviation of X i and ρ is the coefficient of correlation between X 1 and X 2. The correlation matrix is given by ( ) 1 ρ R =. ρ 1 66 / 78

67 The Bivariate Normal distribution In this case, the determinant of the covariance matrix is given by Σ = σ 2 1σ 2 2 ρ 2 σ 2 1σ 2 2 = σ 2 1σ 2 2(1 ρ 2 ). Hence, Σ 1/2 = σ 1 σ 2 1 ρ / 78

68 The Bivariate Normal distribution Also, Σ 1 = 1 σ 2 1 σ2 2 (1 ρ2 ) ( ) σ 2 2 ρσ 1 σ 2 ρσ 1 σ 2 σ 2 1 It follows that ( ) 1/2(x µ)σ 1 (x µ) = 1/2(x 1 µ 1, x 2 µ 2 )Σ 1 x1 µ 1 x 2 µ 2 [ 1 (x1 µ 1 ) 2 = 2(1 ρ 2 ) σ1 2 + (x 2 µ 2 ) 2 σ2 2 2ρ(x ] 1 µ 1 )(x 2 µ 2 ) σ 1 σ 2 68 / 78

69 The Bivariate Normal distribution Hence, the joint density function of X = [X 1, X 2 ] is given by 1 f X (x 1, x 2 )= 2πσ 1 σ exp 2 1 ρ 2 + (x 2 µ 2 ) 2 σ 2 2 { 1 [ (x1 µ 1 ) 2 2(1 ρ 2 ) 2ρ(x 1 µ 1 )(x 2 µ 2 ) σ 1 σ 2 σ 2 1 ]} / 78

70 The Bivariate Normal distribution Note that if ρ = 0, then f X (x 1, x 2 )= 1 { 1 exp 2πσ 1 σ 2 2 =f X1 (x 1 )f X2 (x 2 ). [ (x1 µ 1 ) 2 σ (x 2 µ 2 ) 2 ]} σ2 2 Hence, it follows that if [X 1, X 2 ] has a bivariate normal distribution, then ρ = 0 is a necessary and sufficient condition for X 1 and X 2 to be independent. Note that, In general, in order to prove independence it is not sufficient to show that ρ = 0 (see Example 4.2). 70 / 78

71 Linear Combinations of Random Variables Note that for any variables X 1, X 2,..., X k. k E[ a i X i ]=a 1 E[X 1 ] + a 2 E[X 2 ] a n E[X n ]. Var[ i=1 [ k k ] a i X i ]= ai 2 Var(X i ) + 2 i=1 i=1 i=1 j=i+1 k k a i a j Cov(X i, X j ). In particular, when k = 2, Var[a 1 X 1 + a 2 X 2 ] = a1var(x 2 1 ) + a2var(x 2 2 ) + 2a 1 a 2 Cov(X 1, X 2 ). When the X i are independent, then Var[a 1 X 1 + a 2 X a k X k ] = k ai 2 Var(X i ). i=1 71 / 78

72 Example 4.5 Suppose the height of males has a normal distribution with mean 170cm and standard deviation 15cm and the correlation of the height of a father and son is Derive the joint distribution of the height of two males chosen at random. 2. Calculate the probability that the average height of two such individuals is greater than 179cm. 3. Assuming the joint distribution of the height of father and son is a bivariate normal distribution, derive the joint distribution of the height of a father and son. 4. Calculate the probability that the average height of a father and son is greater than 179cm. 72 / 78

73 Example Since the individuals are picked at random, we may assume that the two heights are independent. The univariate density function of the i-th height (i = 1, 2) is f Xi (x i ) = 1 ( (xi µ i ) 2 ) exp = σ i 2π 2σ 2 i π exp ( (xi 170) The joint density function of the two heights is the product of these density functions, i.e. f X1,X 2 (x 1, x 2 )=f X1 (x 1 )f X2 (x 2 ) ( 1 = π exp (x1 170) 2 (x 2 170) 2 ) ), 73 / 78

74 Example Note that the sum of the two heights also has a normal distribution. Let S = X 1 + X 2. E[S] = E[X 1 ] + E[X 2 ] = 340. Since X 1 and X 2 are independent, Var(S)=Var(X 1 )+Var(X 2 ) = = 450. We must calculate P([X 1 + X 2 ]/2 > 179) = P(X 1 + X 2 > 358) = P(S > 358). 74 / 78

75 Example 4.5 Since S has a normal distribution, by standardising we obtain, ( ) S E(S) P(S > 358)=P > σ S 450 =P(Z > 0.85) = / 78

76 Example The joint distribution of the height of a father and his son is given by { [ 1 1 f X (x 1, x 2 )= 2πσ 1 σ exp (x1 µ 1 ) ρ 2 2(1 ρ 2 ) σ (x 2 µ 2 ) 2 σ2 2 2ρ(x ]} 1 µ 1 )(x 2 µ 2 ) σ 1 σ 2 = 1 { 360π exp 1 [ (x1 170) (x 2 170) 2 1.2(x 1 170)(x 2 170) ]}. 76 / 78

77 Example If the joint distribution of the height of father and son has a bivariate distribution, then the sum of their heights has a normal distribution. In this case, E[S] = E[X 1 ] + E[X 2 ] = 340. Var(S)=Var(X 1 )+Var(X 2 ) + 2Cov(X 1, X 2 ). We have ρ= Cov(X 1, X 2 ) σ 1 σ 2 Cov(X 1, X 2 )=ρσ 1 σ 2 = = / 78

78 Example 4.5 It follows that Var(S)=Var(X 1 ) + Var(X 2 ) + 2Cov(X 1, X 2 ) = = 720. By standardising, we obtain ( S E(S) P(S > 358)=P > σ S =P(Z > 0.67) = ) / 78

### Joint Distributions. Tieming Ji. Fall 2012

Joint Distributions Tieming Ji Fall 2012 1 / 33 X : univariate random variable. (X, Y ): bivariate random variable. In this chapter, we are going to study the distributions of bivariate random variables

### Topic 4: Multivariate random variables. Multiple random variables

Topic 4: Multivariate random variables Joint, marginal, and conditional pmf Joint, marginal, and conditional pdf and cdf Independence Expectation, covariance, correlation Conditional expectation Two jointly

### Covariance and Correlation. Consider the joint probability distribution f XY (x, y).

Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 2: Section 5-2 Covariance and Correlation Consider the joint probability distribution f XY (x, y). Is there a relationship between X and Y? If so, what kind?

### ST 371 (VIII): Theory of Joint Distributions

ST 371 (VIII): Theory of Joint Distributions So far we have focused on probability distributions for single random variables. However, we are often interested in probability statements concerning two or

### STAT 430/510 Probability Lecture 14: Joint Probability Distribution, Continuous Case

STAT 430/510 Probability Lecture 14: Joint Probability Distribution, Continuous Case Pengyuan (Penelope) Wang June 20, 2011 Joint density function of continuous Random Variable When X and Y are two continuous

### Jointly Distributed Random Variables

Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................

### Chapter 4. Multivariate Distributions

1 Chapter 4. Multivariate Distributions Joint p.m.f. (p.d.f.) Independent Random Variables Covariance and Correlation Coefficient Expectation and Covariance Matrix Multivariate (Normal) Distributions Matlab

### Random Variables, Expectation, Distributions

Random Variables, Expectation, Distributions CS 5960/6960: Nonparametric Methods Tom Fletcher January 21, 2009 Review Random Variables Definition A random variable is a function defined on a probability

### ECE302 Spring 2006 HW7 Solutions March 11, 2006 1

ECE32 Spring 26 HW7 Solutions March, 26 Solutions to HW7 Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where

### MULTIVARIATE PROBABILITY DISTRIBUTIONS

MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined

### Section 6.1 Joint Distribution Functions

Section 6.1 Joint Distribution Functions We often care about more than one random variable at a time. DEFINITION: For any two random variables X and Y the joint cumulative probability distribution function

### Chapters 5. Multivariate Probability Distributions

Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

### Joint Exam 1/P Sample Exam 1

Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question

### Topic 8 The Expected Value

Topic 8 The Expected Value Functions of Random Variables 1 / 12 Outline Names for Eg(X ) Variance and Standard Deviation Independence Covariance and Correlation 2 / 12 Names for Eg(X ) If g(x) = x, then

### 3. Continuous Random Variables

3. Continuous Random Variables A continuous random variable is one which can take any value in an interval (or union of intervals) The values that can be taken by such a variable cannot be listed. Such

### Bivariate Distributions

Chapter 4 Bivariate Distributions 4.1 Distributions of Two Random Variables In many practical cases it is desirable to take more than one measurement of a random observation: (brief examples) 1. What is

### SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

### Joint Probability Distributions and Random Samples. Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage

5 Joint Probability Distributions and Random Samples Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Two Discrete Random Variables The probability mass function (pmf) of a single

### The Bivariate Normal Distribution

The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included

### Some probability and statistics

Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,

### SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance

### Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.

Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x

### Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

### Feb 28 Homework Solutions Math 151, Winter 2012. Chapter 6 Problems (pages 287-291)

Feb 8 Homework Solutions Math 5, Winter Chapter 6 Problems (pages 87-9) Problem 6 bin of 5 transistors is known to contain that are defective. The transistors are to be tested, one at a time, until the

### P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

### Random Vectors and the Variance Covariance Matrix

Random Vectors and the Variance Covariance Matrix Definition 1. A random vector X is a vector (X 1, X 2,..., X p ) of jointly distributed random variables. As is customary in linear algebra, we will write

### Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density

HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### The Scalar Algebra of Means, Covariances, and Correlations

3 The Scalar Algebra of Means, Covariances, and Correlations In this chapter, we review the definitions of some key statistical concepts: means, covariances, and correlations. We show how the means, variances,

### MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions...

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo c Copyright 2004-2012 by Jenny A. Baglivo. All Rights Reserved. Contents 3 MT426 Notebook 3 3 3.1 Definitions............................................

### Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23

Probability Theory Uncertainty is key concept in machine learning. Probability provides consistent framework for the quantification and manipulation of uncertainty. Probability of an event is the fraction

### Lecture Notes 1. Brief Review of Basic Probability

Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very

### Covariance and Correlation

Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a data-based relative frequency distribution by measures of location and spread, such

### Lecture 6: Discrete & Continuous Probability and Random Variables

Lecture 6: Discrete & Continuous Probability and Random Variables D. Alex Hughes Math Camp September 17, 2015 D. Alex Hughes (Math Camp) Lecture 6: Discrete & Continuous Probability and Random September

### Math 141. Lecture 7: Variance, Covariance, and Sums. Albyn Jones 1. 1 Library 304. jones/courses/141

Math 141 Lecture 7: Variance, Covariance, and Sums Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Last Time Variance: expected squared deviation from the mean: Standard

### SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation

SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu September 19, 2015 Outline

### Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ).

Correlation Regression Bivariate Normal Suppose that X and Y are r.v. s with joint density f(x y) and suppose that the means of X and Y are respectively µ 1 µ 2 and the variances are 1 2. Definition The

### We have discussed the notion of probabilistic dependence above and indicated that dependence is

1 CHAPTER 7 Online Supplement Covariance and Correlation for Measuring Dependence We have discussed the notion of probabilistic dependence above and indicated that dependence is defined in terms of conditional

### Variances and covariances

Chapter 4 Variances and covariances 4.1 Overview The expected value of a random variable gives a crude measure for the center of location of the distribution of that random variable. For instance, if the

### 3 Multiple Discrete Random Variables

3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f

### Correlation in Random Variables

Correlation in Random Variables Lecture 11 Spring 2002 Correlation in Random Variables Suppose that an experiment produces two random variables, X and Y. What can we say about the relationship between

### Lesson 5 Chapter 4: Jointly Distributed Random Variables

Lesson 5 Chapter 4: Jointly Distributed Random Variables Department of Statistics The Pennsylvania State University 1 Marginal and Conditional Probability Mass Functions The Regression Function Independence

### Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

### Section 5.1 Continuous Random Variables: Introduction

Section 5. Continuous Random Variables: Introduction Not all random variables are discrete. For example:. Waiting times for anything (train, arrival of customer, production of mrna molecule from gene,

### Probability and Statistics

CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

### 5. Continuous Random Variables

5. Continuous Random Variables Continuous random variables can take any value in an interval. They are used to model physical characteristics such as time, length, position, etc. Examples (i) Let X be

### Chap 3 : Two Random Variables

Chap 3 : Two Random Variables Chap 3.1: Distribution Functions of Two RVs In many experiments, the observations are expressible not as a single quantity, but as a family of quantities. For example to record

### Notes on Continuous Random Variables

Notes on Continuous Random Variables Continuous random variables are random quantities that are measured on a continuous scale. They can usually take on any value over some interval, which distinguishes

### 6. Jointly Distributed Random Variables

6. Jointly Distributed Random Variables We are often interested in the relationship between two or more random variables. Example: A randomly chosen person may be a smoker and/or may get cancer. Definition.

### Worked examples Multiple Random Variables

Worked eamples Multiple Random Variables Eample Let X and Y be random variables that take on values from the set,, } (a) Find a joint probability mass assignment for which X and Y are independent, and

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### Chapter 4 - Lecture 1 Probability Density Functions and Cumul. Distribution Functions

Chapter 4 - Lecture 1 Probability Density Functions and Cumulative Distribution Functions October 21st, 2009 Review Probability distribution function Useful results Relationship between the pdf and the

### Variance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers

Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications

### Lecture 9: Introduction to Pattern Analysis

Lecture 9: Introduction to Pattern Analysis g Features, patterns and classifiers g Components of a PR system g An example g Probability definitions g Bayes Theorem g Gaussian densities Features, patterns

### Numerical Summarization of Data OPRE 6301

Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

### Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.

Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()

### Math 431 An Introduction to Probability. Final Exam Solutions

Math 43 An Introduction to Probability Final Eam Solutions. A continuous random variable X has cdf a for 0, F () = for 0 <

### Recitation 4. 24xy for 0 < x < 1, 0 < y < 1, x + y < 1 0 elsewhere

Recitation. Exercise 3.5: If the joint probability density of X and Y is given by xy for < x

### An-Najah National University Faculty of Engineering Industrial Engineering Department. Course : Quantitative Methods (65211)

An-Najah National University Faculty of Engineering Industrial Engineering Department Course : Quantitative Methods (65211) Instructor: Eng. Tamer Haddad 2 nd Semester 2009/2010 Chapter 5 Example: Joint

### Sections 2.11 and 5.8

Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and

### What is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference

0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures

### 4. Continuous Random Variables, the Pareto and Normal Distributions

4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random

### Vector Notation: AB represents the vector from point A to point B on a graph. The vector can be computed by B A.

1 Linear Transformations Prepared by: Robin Michelle King A transformation of an object is a change in position or dimension (or both) of the object. The resulting object after the transformation is called

### M2S1 Lecture Notes. G. A. Young http://www2.imperial.ac.uk/ ayoung

M2S1 Lecture Notes G. A. Young http://www2.imperial.ac.uk/ ayoung September 2011 ii Contents 1 DEFINITIONS, TERMINOLOGY, NOTATION 1 1.1 EVENTS AND THE SAMPLE SPACE......................... 1 1.1.1 OPERATIONS

### Statistics 100A Homework 7 Solutions

Chapter 6 Statistics A Homework 7 Solutions Ryan Rosario. A television store owner figures that 45 percent of the customers entering his store will purchase an ordinary television set, 5 percent will purchase

### Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible:

Cramer s Rule and the Adjugate Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Theorem [Cramer s Rule] If A is an invertible

### Chapter 2, part 2. Petter Mostad

Chapter 2, part 2 Petter Mostad mostad@chalmers.se Parametrical families of probability distributions How can we solve the problem of learning about the population distribution from the sample? Usual procedure:

### ST 371 (IV): Discrete Random Variables

ST 371 (IV): Discrete Random Variables 1 Random Variables A random variable (rv) is a function that is defined on the sample space of the experiment and that assigns a numerical variable to each possible

### Review Course Statistics

Review Course Statistics Probability Theory - Statistical Inference - Matrix Algebra Prof. Dr. Christian Conrad Heidelberg University Winter term 2012/13 Christian Conrad (Heidelberg University) Winter

### MATH 201. Final ANSWERS August 12, 2016

MATH 01 Final ANSWERS August 1, 016 Part A 1. 17 points) A bag contains three different types of dice: four 6-sided dice, five 8-sided dice, and six 0-sided dice. A die is drawn from the bag and then rolled.

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### 7. Tests of association and Linear Regression

7. Tests of association and Linear Regression In this chapter we consider 1. Tests of Association for 2 qualitative variables. 2. Measures of the strength of linear association between 2 quantitative variables.

### MC3: Econometric Theory and Methods

University College London Department of Economics M.Sc. in Economics MC3: Econometric Theory and Methods Course notes: Econometric models, random variables, probability distributions and regression Andrew

### Solutions for the exam for Matematisk statistik och diskret matematik (MVE050/MSG810). Statistik för fysiker (MSG820). December 15, 2012.

Solutions for the exam for Matematisk statistik och diskret matematik (MVE050/MSG810). Statistik för fysiker (MSG8). December 15, 12. 1. (3p) The joint distribution of the discrete random variables X and

### Elliptical copulae. Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke

Elliptical copulae Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke Abstract: In this paper we construct a copula, that is, a distribution with uniform marginals. This copula is continuous and can realize

### Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

### Basic Statistcs Formula Sheet

Basic Statistcs Formula Sheet Steven W. ydick May 5, 0 This document is only intended to review basic concepts/formulas from an introduction to statistics course. Only mean-based procedures are reviewed,

### Multivariate normal distribution and testing for means (see MKB Ch 3)

Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................

### What Does the Correlation Coefficient Really Tell Us About the Individual?

What Does the Correlation Coefficient Really Tell Us About the Individual? R. C. Gardner and R. W. J. Neufeld Department of Psychology University of Western Ontario ABSTRACT The Pearson product moment

### 3.4 Statistical inference for 2 populations based on two samples

3.4 Statistical inference for 2 populations based on two samples Tests for a difference between two population means The first sample will be denoted as X 1, X 2,..., X m. The second sample will be denoted

### Problem Set 4: Covariance functions and Gaussian random fields

Problem Set : Covariance functions and Gaussian random fields GEOS 7: Inverse Problems and Parameter Estimation, Carl Tape Assigned: February 9, 15 Due: February 1, 15 Last compiled: February 5, 15 Overview

### 3. The Multivariate Normal Distribution

3. The Multivariate Normal Distribution 3.1 Introduction A generalization of the familiar bell shaped normal density to several dimensions plays a fundamental role in multivariate analysis While real data

### MTH135/STA104: Probability

MTH135/STA14: Probability Homework # 8 Due: Tuesday, Nov 8, 5 Prof Robert Wolpert 1 Define a function f(x, y) on the plane R by { 1/x < y < x < 1 f(x, y) = other x, y a) Show that f(x, y) is a joint probability

### Expectation Discrete RV - weighted average Continuous RV - use integral to take the weighted average

PHP 2510 Expectation, variance, covariance, correlation Expectation Discrete RV - weighted average Continuous RV - use integral to take the weighted average Variance Variance is the average of (X µ) 2

### Generating Random Numbers Variance Reduction Quasi-Monte Carlo. Simulation Methods. Leonid Kogan. MIT, Sloan. 15.450, Fall 2010

Simulation Methods Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Simulation Methods 15.450, Fall 2010 1 / 35 Outline 1 Generating Random Numbers 2 Variance Reduction 3 Quasi-Monte

### Math 576: Quantitative Risk Management

Math 576: Quantitative Risk Management Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 4 Haijun Li Math 576: Quantitative Risk Management Week 4 1 / 22 Outline 1 Basics

### The Big 50 Revision Guidelines for S1

The Big 50 Revision Guidelines for S1 If you can understand all of these you ll do very well 1. Know what is meant by a statistical model and the Modelling cycle of continuous refinement 2. Understand

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 18. A Brief Introduction to Continuous Probability

CS 7 Discrete Mathematics and Probability Theory Fall 29 Satish Rao, David Tse Note 8 A Brief Introduction to Continuous Probability Up to now we have focused exclusively on discrete probability spaces

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Lecture 8. Generating a non-uniform probability distribution

Discrete outcomes Lecture 8 Generating a non-uniform probability distribution Last week we discussed generating a non-uniform probability distribution for the case of finite discrete outcomes. An algorithm

### Chapter 1 - σ-algebras

Page 1 of 17 PROBABILITY 3 - Lecture Notes Chapter 1 - σ-algebras Let Ω be a set of outcomes. We denote by P(Ω) its power set, i.e. the collection of all the subsets of Ω. If the cardinality Ω of Ω is

### 3.4 The Normal Distribution

3.4 The Normal Distribution All of the probability distributions we have found so far have been for finite random variables. (We could use rectangles in a histogram.) A probability distribution for a continuous

### Probability for Estimation (review)

Probability for Estimation (review) In general, we want to develop an estimator for systems of the form: x = f x, u + η(t); y = h x + ω(t); ggggg y, ffff x We will primarily focus on discrete time linear

### A Tutorial on Probability Theory

Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### Lecture 21. The Multivariate Normal Distribution

Lecture. The Multivariate Normal Distribution. Definitions and Comments The joint moment-generating function of X,...,X n [also called the moment-generating function of the random vector (X,...,X n )]