# Conditional expectation

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 A Conditional expectation A.1 Review of conditional densities, expectations We start with the continuous case. This is sections 6.6 and 6.8 in the book. Let X, Y be continuous random variables. We defined the conditional density of X given Y to be Then f X Y (x y) = f X,Y (x, y) f Y (y) P (a X b Y = y) = b a f X,Y (x y) dx Conditioning on Y = y is conditioning on an event with probability zero. This is not defined, so we make sense of the left side above by a limiting procedure: P (a X b Y = y) = lim P (a X b Y y < ɛ) ɛ 0 + We then define the conditional expectation of X given Y = y to be E[X Y = y] = x f X Y (x y) dx We have the following continuous analog of the partition theorem. E[Y ] = E[Y X = x] f X (x) dx Now we review the discrete case. This was section 2.5 in the book. In some sense it is simpler than the continuous case. Everything comes down to the very first definition involving conditioning. For events A and B P (A B) = P (A B) P (B) assuming that P (B) > 0. If X is a discrete RV, the conditional density of X given the event B is f(x B) = P (X = x B) = and the conditional expectation of X given B is P (X = x, B) P (B) E[X B] = x x f(x B) 1

2 The partition theorem says that if B n is a partition of the sample space then E[X] = n E[X B n ] P (B n ) Now suppose that X and Y are discrete RV s. If y is in the range of Y then Y = y is a event with nonzero probability, so we can use it as the B in the above. So f(x Y = y) is defined. We can change the notation to make it look like the continuous case and write f(x Y = y) as f X Y (x y). Of course it is given by f X Y (x y) = P (X = x, Y = y) P (Y = y) = f X,Y (x, y) f Y (y) This looks identical to the formula in the continuous case, but it is really a different formula. In the above f X,Y and f Y are pmf s; in the continuous case they are pdf s. With this notation we have E[X Y = y] = x x f X Y (x y) and the partition theorem is E[X] = y E[X Y = y] P (Y = y) A.2 Conditional expectation as a Random Variable Conditional expectations such as E[X Y = 2] or E[X Y = 5] are numbers. If we consider E[X Y = y], it is a number that depends on y. So it is a function of y. In this section we will study a new object E[X Y ] that is a random variable. We start with an example. Example: Roll a die until we get a 6. Let Y be the total number of rolls and X the number of 1 s we get. We compute E[X Y = y]. The event Y = y means that there were y 1 rolls that were not a 6 and then the yth roll was a six. So given this event, X has a binomial distribution with n = y 1 trials and probability of success p = 1/5. So E[X Y = y] = np = 1 (y 1) 5 Now consider the following process. We do the experiment and get an outcome ω. (In this example, ω would be a string of 1, 2, 3, 4, 5 s ending with a 6.) Then we compute y = Y (W ). (In this example y would just be the number of rolls. ) Then we compute E[X Y = y]. This process gives a function ω E[X Y = y] 2

3 So this is a random variable. It is usually written as E[X Y ]. In our example ω is mapped to (y 1)/5 where y = Y (ω). So ω is mapped to (Y (ω) 1)/5. So the random variable E[X Y ] is just (Y 1)/5. Note that E[X Y ] is a function of Y. This will be true in general. We try another conditional expectation in the same example: E[X 2 Y ]. Again, given Y = y, X has a binomial distribution with n = y 1 trials and p = 1/5. The variance of such a random variable is np(1 p) = (y 1)4/25. So Using what we found before, E[X 2 Y = y] (E[X Y = y]) 2 = (y 1) 4 25 And so Thus E[X 2 Y = y] ( 1 5 (y 1))2 = (y 1) 4 25 E[X 2 Y = y] = 1 25 (y 1)2 + 4 (y 1) 25 E[X 2 Y ] = 1 25 (Y 1) (Y 1) = 1 25 (Y 2 + 2Y 3) Once again, E[X 2 Y ] is a function of Y. Intuition: E[X Y ] is the function of Y that bests approximates X. This is a vague statement since we have not said what best means. We consider two extreme cases. First suppose that X is itself a function of Y, e.g., Y 2 or e Y. Then the function of Y that best approximates X is X itself. (Whatever best means, you can t do any better than this.) The other extreme case is when X and Y are independent. In this case, knowing Y tells us nothing about X. So we might expect that E[X Y ] will not depend on Y. Indeed, we have A.3 Properties of conditional expectation Before we list all the properties of E[X Y ], we need to consider conditioning on more that one random variable. Let X, Y, Z be discrete random variables. Then E[X Y = y, Z = z] makes sense. We can think of it as a function of the random outcome ω: ω E[X Y = Y (ω), Z = Z(ω)] So it is a random variable. We denote it by E[X Y, Z]. In the continuous case we need to define E[X Y = y, Z = z] by a limiting process. The result is a function of y and z that we can once again interpret as a random variable. 3

4 Theorem 1 Let X, Y, Z be random variables, a, b R, and g : R R. Assuming all the following expectations exist, we have (i) E[a Y ] = a (ii) E[aX + bz Y ] = ae[x Y ] + be[z Y ] (iii) E[X Y ] 0 if X 0. (iv) E[X Y ] = E[X] if X and Y are independent. (v) E[E[X Y ]] = E[X] (vi) E[Xg(Y ) Y ] = g(y )E[X Y ]. In particular, E[g(Y ) Y ] = g(y ). (vii) E[X Y, g(y )] = E[X Y ] (viii) E[E[X Y, Z] Y ] = E[X Y ] Partial proofs: The first three are not hard to prove, and we leave them to the reader. Consider (iv). We prove the continuous case and leave the discrete case to the reader. If X and Y are independent then f X Y (x y) = f X,Y (x, y) f Y (y) = f X(x)f Y (y) f Y (y) = f X (x) So E[X Y = y] = x f X Y (x y) dx = x f X (x) dx = E[X] Consider (v). Suppose that the random variables are discrete. We need to compute the expected value of the random variable E[X Y ]. It is a function of Y and it takes on the value E[X Y = y] when Y = y. So by the law of the unconscious whatever, E[E[X Y ]] = y E[X Y = y] P (Y = y) By the partition theorem this is equal to E[X]. So in the discrete case, (iv) is really the partition theorem in disguise. In the continuous case it is too. Consider (vi). We must compute E[Xg(Y ) Y = y]. Given that Y = y, the possible values of Xg(Y ) are xg(y) where x varies over the range of X. The probability of the value xg(y) given that Y = y is just P (X = x Y = y). So E[Xg(Y ) Y = y] = x xg(y)p (X = x Y = y) = g(y) x x P (X = x Y = y) = g(y)e[x Y = y] This proves (vi). 4

5 Consider (viii). Again, we only consider the discrete case. We need to compute E[E[X Y ; Z] Y = y]. E[X Y ; Z] is a random variable. Given that Y = y, its possible values are E[X Y = y; Z = z] where z varies over the range of Z. Given that Y = y, the probability that E[X Y ; Z] = E[X Y = y; Z = z] is just P (Z = z Y = y). Hence, E[E[X Y ; Z] Y = y] = z = z = z,x = z,x = x = x E[X Y = y, Z = z]p (Z = z Y = y) x P (X = x Y = y, Z = z)p (Z = z Y = y) x x x x P (X = x, Y = y, Z = z) P (Y = y, Z = z) P (X = x, Y = y, Z = z) P (Y = y) P (X = x, Y = y) P (Y = y) x P (X = x Y = y) P (Z = z, Y = y) P (Y = y) = E[X Y = y] (1) Example: Let X and Y be independent; each is uniformly distributed on [0, 1]. Let Z = X + Y. Find E[Z X], E[X Z], E[XZ X], E[XZ Z]. We start with the easy ones. E[Z X] = E[X + Y X] = E[X X] + E[Y X] = X + E[Y ] = X where we have used the independence of X and Y and properties (iv) and the special case of (vi). Using property (vi), E[XZ X] = XE[Z X] = X(X ). Now we do the hard one: E[X Z]. We need the joint pdf of X and Z. So we do a change of variables. Let W = X, Z = X + Y. This is a linear transformation, so the Jacobian will be a constant. We postpone computing it. We need to find the image of the square 0 x, y 1 under this transformation. Look at the boundaries. Since it is a linear transformation, the four edges of the square will be mapped to line segments. To find them we can just compute where the four corners of the square are mapped. (x, y) = (0, 0) (w, z) = (0, 0) (x, y) = (1, 0) (w, z) = (1, 1) (x, y) = (0, 1) (w, z) = (0, 1) (x, y) = (1, 1) (w, z) = (1, 2) 5

6 So the image of the square is the parallelogram with vertices (0, 0), (1, 1), (0, 1) and (1, 2). The joint density of W and Z will be uniform on this region. Let A denote the interior of the parallelogram. Since it has area 1, we conclude f W,Z (w, z) = 1((w, z) A) Note that we avoided computing the Jacobian. Now we can figure out what f X Z (x z) is. We must consider two cases. First suppose 0 z 1. Given Z = z, X is uniformly distributed between 0 and z. So E[X Z = z] = z/2. In the other case 1 z 2. Then X is uniformly distributed between z 1 and 1. So E[X Z = z] = (z 1 + 1)/2 = z/2. So in both cases E[X Z = z] = z/2. Thus E[X Z] = Z/2. Finally, we have E[XZ Z] = ZE[X Z] = Z 2 /2. We get a small check on our answers using property (v). We found E[Z X] = X +1/2. So its mean is E[X] + 1/2 = 1/2 + 1/2 = 1. Property (v) says E[E[Z X]] = E[Z] = E[X] + E[Y ] = 1/2 + 1/2 = 1. We also found E[X Z] = Z/2. Thus the mean of this random variable is E[Z]/2 = 1/2. And property (v) says it should be E[X] = 1/2. Example (random sums): Let X n be an i.i.d. sequence with mean µ and variance σ 2. Let N be a RV that is independent of all the X n and takes on the values 1, 2, 3,. Let N S N = j=1 Note that the number of terms in the sum is random. We will find E[S N N] and E[SN 2 N] and use them to compute the mean and variance of S N. Given that N = n, S N is a sum with a fixed number of terms: n j=1 X j So E[S N N = n] = nµ. Thus E[S N N] = Nµ. Since the X j are independent, their variances add and so So E[S 2 N N = n] (E[S N N = n]) 2 = var[s N N = n] = nσ 2 X j E[S 2 N N] = Nσ2 + (E[S N N]) 2 = Nσ 2 + N 2 µ 2 Now using property (v), we have E[S N ] = E[E[S N N]] = E[Nµ] = µe[n] E[S 2 N ] = E[E[S2 N N]] = E[Nσ2 + N 2 µ 2 ] = σ 2 E[N] + µ 2 E[N 2 ] and so the variance of S N is var(s N ) = E[S 2 N ] (E[S N]) 2 = σ 2 E[N] + µ 2 E[N 2 ] µ 2 E[N] 2 = σ 2 E[N] + µ 2 var(n) 6

7 A.4 Conditional expectation as a best approximation We have said that E[X Y ] is the function of Y that best approximates X. In this section we make this precise. We will assume we have discrete random variables. The main result of this section is true in the continuous case as well. We need to make precise what it means to be a function of Y. In the discrete case we can simply say that X is a function of Y if there is a function g : R R such that X = g(y ). (In the continuous case we need some condition that g be nice.) We start with a characterization of functions of Y. Proposition 1 Let X, Y be discrete RV s. Let y n be the possible value of Y and B n = {Y = y n }. Then there is a function g : R R such that X = g(y ) if and only if X is constant on every event B n. Proof: The direction is easy. For the other direction, suppose X is constant on the B n. Then we can write it as X = n x n 1 Bn where 1 Bn is the random variable that is 1 on B n and 0 on its complement. Define g(y n ) = x n. For y not in the range of Y we define g(y) = 0. (It doesn t matter what we define it to be here.) Then Y = g(x). Theorem 2 For any function h : R R, E[(X E[X Y ]) 2 ] E[(X h(y )) 2 ] and we have equality if and only if h(y ) = E[X Y ]. Proof: Let y n be the possible values of Y and B n = {Y = y n }. Then h(y ) is of the form h(y ) = n x n 1 Bn for some x n. (The number x n is just the value of h(y ) on B n.) So E[(X h(y )) 2 ] = E[(X n x n 1 Bn ) 2 ] We think of this as a function of all the x n and try to find its mininum. It is quadratic in each x n with a positive coef of x 2 n. So the mininum will occur at the critical point given by setting all the partial derivatives with respect to the x n equal to 0. x j E[(X n x n 1 Bn ) 2 ] = 2E[(X n 7 x n 1 Bn )1 Bj ] = x j P (B j ) E[X1 Bj ]

8 It is easy to check that E[X Y = y j ] = E[X1 B j ] P (B j ) which completes the proof. 8

### 3 Multiple Discrete Random Variables

3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f

### Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density

HW MATH 461/561 Lecture Notes 15 1 Definition: Suppose that two random variables, either continuous or discrete, X and Y have joint density and marginal densities f(x, y), (x, y) Λ X,Y f X (x), x Λ X,

### Random Variables, Expectation, Distributions

Random Variables, Expectation, Distributions CS 5960/6960: Nonparametric Methods Tom Fletcher January 21, 2009 Review Random Variables Definition A random variable is a function defined on a probability

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### Change of Continuous Random Variable

Change of Continuous Random Variable All you are responsible for from this lecture is how to implement the Engineer s Way (see page 4) to compute how the probability density function changes when we make

### Properties of Expected values and Variance

Properties of Expected values and Variance Christopher Croke University of Pennsylvania Math 115 UPenn, Fall 2011 Expected value Consider a random variable Y = r(x ) for some function r, e.g. Y = X 2 +

### EE 322: Probabilistic Methods for Electrical Engineers. Zhengdao Wang Department of Electrical and Computer Engineering Iowa State University

EE 322: Probabilistic Methods for Electrical Engineers Zhengdao Wang Department of Electrical and Computer Engineering Iowa State University Discrete Random Variables 1 Introduction to Random Variables

### POL 571: Expectation and Functions of Random Variables

POL 571: Expectation and Functions of Random Variables Kosuke Imai Department of Politics, Princeton University March 10, 2006 1 Expectation and Independence To gain further insights about the behavior

### MATH 201. Final ANSWERS August 12, 2016

MATH 01 Final ANSWERS August 1, 016 Part A 1. 17 points) A bag contains three different types of dice: four 6-sided dice, five 8-sided dice, and six 0-sided dice. A die is drawn from the bag and then rolled.

### Statistiek (WISB361)

Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17

### Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ).

Correlation Regression Bivariate Normal Suppose that X and Y are r.v. s with joint density f(x y) and suppose that the means of X and Y are respectively µ 1 µ 2 and the variances are 1 2. Definition The

### Continuous Random Variables

Continuous Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Continuous Random Variables 2 11 Introduction 2 12 Probability Density Functions 3 13 Transformations 5 2 Mean, Variance and Quantiles

### Random Variables. Consider a probability model (Ω, P ). Discrete Random Variables Chs. 2, 3, 4. Definition. A random variable is a function

Rom Variables Discrete Rom Variables Chs.,, 4 Rom Variables Probability Mass Functions Expectation: The Mean Variance Special Distributions Hypergeometric Binomial Poisson Joint Distributions Independence

### Bivariate Distributions

Chapter 4 Bivariate Distributions 4.1 Distributions of Two Random Variables In many practical cases it is desirable to take more than one measurement of a random observation: (brief examples) 1. What is

### e.g. arrival of a customer to a service station or breakdown of a component in some system.

Poisson process Events occur at random instants of time at an average rate of λ events per second. e.g. arrival of a customer to a service station or breakdown of a component in some system. Let N(t) be

### Topic 2: Scalar random variables. Definition of random variables

Topic 2: Scalar random variables Discrete and continuous random variables Probability distribution and densities (cdf, pmf, pdf) Important random variables Expectation, mean, variance, moments Markov and

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### 1 Definition of Conditional Expectation

1 DEFINITION OF CONDITIONAL EXPECTATION 4 1 Definition of Conditional Expectation 1.1 General definition Recall the definition of conditional probability associated with Bayes Rule P(A B) P(A B) P(B) For

### 5. Conditional Expected Value

Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 5. Conditional Expected Value As usual, our starting point is a random experiment with probability measure P on a sample space Ω. Suppose that X is

### Lecture 6: Discrete & Continuous Probability and Random Variables

Lecture 6: Discrete & Continuous Probability and Random Variables D. Alex Hughes Math Camp September 17, 2015 D. Alex Hughes (Math Camp) Lecture 6: Discrete & Continuous Probability and Random September

### Notes on the second moment method, Erdős multiplication tables

Notes on the second moment method, Erdős multiplication tables January 25, 20 Erdős multiplication table theorem Suppose we form the N N multiplication table, containing all the N 2 products ab, where

### Chapter 4 Expected Values

Chapter 4 Expected Values 4. The Expected Value of a Random Variables Definition. Let X be a random variable having a pdf f(x). Also, suppose the the following conditions are satisfied: x f(x) converges

### 2.8 Expected values and variance

y 1 b a 0 y 0.0 0.2 0.4 0.6 0.8 1.0 1.2 0 a b (a) Probability density function x 0 a b (b) Cumulative distribution function x Figure 2.3: The probability density function and cumulative distribution function

### Expectation Discrete RV - weighted average Continuous RV - use integral to take the weighted average

PHP 2510 Expectation, variance, covariance, correlation Expectation Discrete RV - weighted average Continuous RV - use integral to take the weighted average Variance Variance is the average of (X µ) 2

### What is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference

0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures

### Chapter 3: Discrete Random Variable and Probability Distribution. January 28, 2014

STAT511 Spring 2014 Lecture Notes 1 Chapter 3: Discrete Random Variable and Probability Distribution January 28, 2014 3 Discrete Random Variables Chapter Overview Random Variable (r.v. Definition Discrete

### , where f(x,y) is the joint pdf of X and Y. Therefore

INDEPENDENCE, COVARIANCE AND CORRELATION M384G/374G Independence: For random variables and, the intuitive idea behind " is independent of " is that the distribution of shouldn't depend on what is. This

### Continuous random variables

Continuous random variables So far we have been concentrating on discrete random variables, whose distributions are not continuous. Now we deal with the so-called continuous random variables. A random

### Math 431 An Introduction to Probability. Final Exam Solutions

Math 43 An Introduction to Probability Final Eam Solutions. A continuous random variable X has cdf a for 0, F () = for 0 <

### Statistics 100 Binomial and Normal Random Variables

Statistics 100 Binomial and Normal Random Variables Three different random variables with common characteristics: 1. Flip a fair coin 10 times. Let X = number of heads out of 10 flips. 2. Poll a random

### Lesson 5 Chapter 4: Jointly Distributed Random Variables

Lesson 5 Chapter 4: Jointly Distributed Random Variables Department of Statistics The Pennsylvania State University 1 Marginal and Conditional Probability Mass Functions The Regression Function Independence

### Continuous Distributions

MAT 2379 3X (Summer 2012) Continuous Distributions Up to now we have been working with discrete random variables whose R X is finite or countable. However we will have to allow for variables that can take

### STAT 571 Assignment 1 solutions

STAT 571 Assignment 1 solutions 1. If Ω is a set and C a collection of subsets of Ω, let A be the intersection of all σ-algebras that contain C. rove that A is the σ-algebra generated by C. Solution: Let

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### MULTIVARIATE PROBABILITY DISTRIBUTIONS

MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined

### Good luck! Problem 1. (b) The random variables X 1 and X 2 are independent and N(0, 1)-distributed. Show that the random variables X 1

Avd. Matematisk statistik TETAME I SF2940 SAOLIKHETSTEORI/EXAM I SF2940 PROBABILITY THE- ORY, WEDESDAY OCTOBER 26, 2016, 08.00-13.00. Examinator : Boualem Djehiche, tel. 08-7907875, email: boualem@kth.se

### ST 371 (IV): Discrete Random Variables

ST 371 (IV): Discrete Random Variables 1 Random Variables A random variable (rv) is a function that is defined on the sample space of the experiment and that assigns a numerical variable to each possible

### Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood

Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. b 0, b 1 Q = n (Y i (b 0 + b 1 X i )) 2 i=1 Minimize this by maximizing

### 5. Continuous Random Variables

5. Continuous Random Variables Continuous random variables can take any value in an interval. They are used to model physical characteristics such as time, length, position, etc. Examples (i) Let X be

### Math 461 Fall 2006 Test 2 Solutions

Math 461 Fall 2006 Test 2 Solutions Total points: 100. Do all questions. Explain all answers. No notes, books, or electronic devices. 1. [105+5 points] Assume X Exponential(λ). Justify the following two

### For a partition B 1,..., B n, where B i B j = for i. A = (A B 1 ) (A B 2 ),..., (A B n ) and thus. P (A) = P (A B i ) = P (A B i )P (B i )

Probability Review 15.075 Cynthia Rudin A probability space, defined by Kolmogorov (1903-1987) consists of: A set of outcomes S, e.g., for the roll of a die, S = {1, 2, 3, 4, 5, 6}, 1 1 2 1 6 for the roll

### Finite Markov Chains and Algorithmic Applications. Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet

Finite Markov Chains and Algorithmic Applications Olle Häggström Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE

### Jointly Distributed Random Variables

Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................

### Example. Two fair dice are tossed and the two outcomes recorded. As is familiar, we have

Lectures 9-10 jacques@ucsd.edu 5.1 Random Variables Let (Ω, F, P ) be a probability space. The Borel sets in R are the sets in the smallest σ- field on R that contains all countable unions and complements

### 3.6: General Hypothesis Tests

3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

### Sequences and Convergence in Metric Spaces

Sequences and Convergence in Metric Spaces Definition: A sequence in a set X (a sequence of elements of X) is a function s : N X. We usually denote s(n) by s n, called the n-th term of s, and write {s

### The Expected Value of X

3.3 Expected Values The Expected Value of X Definition Let X be a discrete rv with set of possible values D and pmf p(x). The expected value or mean value of X, denoted by E(X) or μ X or just μ, is 2 Example

### Joint Probability Distributions and Random Samples. Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage

5 Joint Probability Distributions and Random Samples Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Two Discrete Random Variables The probability mass function (pmf) of a single

### 5. Convergence of sequences of random variables

5. Convergence of sequences of random variables Throughout this chapter we assume that {X, X 2,...} is a sequence of r.v. and X is a r.v., and all of them are defined on the same probability space (Ω,

### Discrete and Continuous Random Variables. Summer 2003

Discrete and Continuous Random Variables Summer 003 Random Variables A random variable is a rule that assigns a numerical value to each possible outcome of a probabilistic experiment. We denote a random

### Lecture 13: Martingales

Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of

### Chapters 5. Multivariate Probability Distributions

Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are

### t-statistics for Weighted Means with Application to Risk Factor Models

t-statistics for Weighted Means with Application to Risk Factor Models Lisa R. Goldberg Barra, Inc. 00 Milvia St. Berkeley, CA 94704 Alec N. Kercheval Dept. of Mathematics Florida State University Tallahassee,

### Chapter 4 Lecture Notes

Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,

### This is a good time to refresh your memory on double-integration. We will be using this skill in the upcoming lectures.

Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 1: Sections 5-1.1 to 5-1.4 For both discrete and continuous random variables we will discuss the following... Joint Distributions (for two or more r.v. s)

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES Contents 1. Probabilistic experiments 2. Sample space 3. Discrete probability

### Exercises with solutions (1)

Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### 6. Jointly Distributed Random Variables

6. Jointly Distributed Random Variables We are often interested in the relationship between two or more random variables. Example: A randomly chosen person may be a smoker and/or may get cancer. Definition.

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.

Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()

### Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood

Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization 1. Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 2. Minimize this by

### Lecture 16: Expected value, variance, independence and Chebyshev inequality

Lecture 16: Expected value, variance, independence and Chebyshev inequality Expected value, variance, and Chebyshev inequality. If X is a random variable recall that the expected value of X, E[X] is the

### Stat 110. Unit 3: Random Variables Chapter 3 in the Text

Stat 110 Unit 3: Random Variables Chapter 3 in the Text 1 Unit 3 Outline Random Variables (RVs) and Distributions Discrete RVs and Probability Mass Functions (PMFs) Bernoulli and Binomial Distributions

### Some notes on the Poisson distribution

Some notes on the Poisson distribution Ernie Croot October 2, 2008 1 Introduction The Poisson distribution is one of the most important that we will encounter in this course it is right up there with the

### Lecture 4: Random Variables

Lecture 4: Random Variables 1. Definition of Random variables 1.1 Measurable functions and random variables 1.2 Reduction of the measurability condition 1.3 Transformation of random variables 1.4 σ-algebra

### The Simple Linear Regression Model: Specification and Estimation

Chapter 3 The Simple Linear Regression Model: Specification and Estimation 3.1 An Economic Model Suppose that we are interested in studying the relationship between household income and expenditure on

### Renewal Theory. (iv) For s < t, N(t) N(s) equals the number of events in (s, t].

Renewal Theory Def. A stochastic process {N(t), t 0} is said to be a counting process if N(t) represents the total number of events that have occurred up to time t. X 1, X 2,... times between the events

### Applications of the Central Limit Theorem

Applications of the Central Limit Theorem October 23, 2008 Take home message. I expect you to know all the material in this note. We will get to the Maximum Liklihood Estimate material very soon! 1 Introduction

### Lecture Notes 1. Brief Review of Basic Probability

Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very

### STAT/MTHE 353: Probability II. STAT/MTHE 353: Multiple Random Variables. Review. Administrative details. Instructor: TamasLinder

STAT/MTHE 353: Probability II STAT/MTHE 353: Multiple Random Variables Administrative details Instructor: TamasLinder Email: linder@mast.queensu.ca T. Linder ueen s University Winter 2012 O ce: Je ery

### Joint distributions Math 217 Probability and Statistics Prof. D. Joyce, Fall 2014

Joint distributions Math 17 Probability and Statistics Prof. D. Joyce, Fall 14 Today we ll look at joint random variables and joint distributions in detail. Product distributions. If Ω 1 and Ω are sample

### Review Exam Suppose that number of cars that passes through a certain rural intersection is a Poisson process with an average rate of 3 per day.

Review Exam 2 This is a sample of problems that would be good practice for the exam. This is by no means a guarantee that the problems on the exam will look identical to those on the exam but it should

### 10 BIVARIATE DISTRIBUTIONS

BIVARIATE DISTRIBUTIONS After some discussion of the Normal distribution, consideration is given to handling two continuous random variables. The Normal Distribution The probability density function f(x)

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### Statistics - Written Examination MEC Students - BOVISA

Statistics - Written Examination MEC Students - BOVISA Prof.ssa A. Guglielmi 26.0.2 All rights reserved. Legal action will be taken against infringement. Reproduction is prohibited without prior consent.

### Random Variables of The Discrete Type

Random Variables of The Discrete Type Definition: Given a random experiment with an outcome space S, a function X that assigns to each element s in S one and only one real number x X(s) is called a random

### Some continuous and discrete distributions

Some continuous and discrete distributions Table of contents I. Continuous distributions and transformation rules. A. Standard uniform distribution U[0, 1]. B. Uniform distribution U[a, b]. C. Standard

### CSE 312, 2011 Winter, W.L.Ruzzo. 7. continuous random variables

CSE 312, 2011 Winter, W.L.Ruzzo 7. continuous random variables continuous random variables Discrete random variable: takes values in a finite or countable set, e.g. X {1,2,..., 6} with equal probability

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

### 3. Continuous Random Variables

3. Continuous Random Variables A continuous random variable is one which can take any value in an interval (or union of intervals) The values that can be taken by such a variable cannot be listed. Such

### The Method of Lagrange Multipliers

The Method of Lagrange Multipliers S. Sawyer October 25, 2002 1. Lagrange s Theorem. Suppose that we want to maximize (or imize a function of n variables f(x = f(x 1, x 2,..., x n for x = (x 1, x 2,...,

### Sampling Central Limit Theorem Proportions. Outline. 1 Sampling. 2 Central Limit Theorem. 3 Proportions

Outline 1 Sampling 2 Central Limit Theorem 3 Proportions Outline 1 Sampling 2 Central Limit Theorem 3 Proportions Populations and samples When we use statistics, we are trying to find out information about

### Estimation with Minimum Mean Square Error

C H A P T E R 8 Estimation with Minimum Mean Square Error INTRODUCTION A recurring theme in this text and in much of communication, control and signal processing is that of making systematic estimates,

### Expected Value. Let X be a discrete random variable which takes values in S X = {x 1, x 2,..., x n }

Expected Value Let X be a discrete random variable which takes values in S X = {x 1, x 2,..., x n } Expected Value or Mean of X: E(X) = n x i p(x i ) i=1 Example: Roll one die Let X be outcome of rolling

### Lecture 3: Continuous distributions, expected value & mean, variance, the normal distribution

Lecture 3: Continuous distributions, expected value & mean, variance, the normal distribution 8 October 2007 In this lecture we ll learn the following: 1. how continuous probability distributions differ

### Random Variables and Their Expected Values

Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution

### Statistics 100A Homework 8 Solutions

Part : Chapter 7 Statistics A Homework 8 Solutions Ryan Rosario. A player throws a fair die and simultaneously flips a fair coin. If the coin lands heads, then she wins twice, and if tails, the one-half

### Vectors, Gradient, Divergence and Curl.

Vectors, Gradient, Divergence and Curl. 1 Introduction A vector is determined by its length and direction. They are usually denoted with letters with arrows on the top a or in bold letter a. We will use

### Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS Part 4: Geometric Distribution Negative Binomial Distribution Hypergeometric Distribution Sections 3-7, 3-8 The remaining discrete random

### Worked examples Random Processes

Worked examples Random Processes Example 1 Consider patients coming to a doctor s office at random points in time. Let X n denote the time (in hrs) that the n th patient has to wait before being admitted

### Review. Lecture 3: Probability Distributions. Poisson Distribution. May 8, 2012 GENOME 560, Spring Su In Lee, CSE & GS

Lecture 3: Probability Distributions May 8, 202 GENOME 560, Spring 202 Su In Lee, CSE & GS suinlee@uw.edu Review Random variables Discrete: Probability mass function (pmf) Continuous: Probability density

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### HIGHER ORDER DIFFERENTIAL EQUATIONS

HIGHER ORDER DIFFERENTIAL EQUATIONS 1 Higher Order Equations Consider the differential equation (1) y (n) (x) f(x, y(x), y (x),, y (n 1) (x)) 11 The Existence and Uniqueness Theorem 12 The general solution

### Fisher Information and Cramér-Rao Bound. 1 Fisher Information. Math 541: Statistical Theory II. Instructor: Songfeng Zheng

Math 54: Statistical Theory II Fisher Information Cramér-Rao Bound Instructor: Songfeng Zheng In the parameter estimation problems, we obtain information about the parameter from a sample of data coming

### P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

### TRANSFORMATIONS OF RANDOM VARIABLES

TRANSFORMATIONS OF RANDOM VARIABLES 1. INTRODUCTION 1.1. Definition. We are often interested in the probability distributions or densities of functions of one or more random variables. Suppose we have