STAT511 Spring 2014 Lecture Notes 1 Chapter 3: Discrete Random Variable and Probability Distribution January 28, 2014 3 Discrete Random Variables Chapter Overview Random Variable (r.v. Definition Discrete and continuous r.v. Probability distribution for discrete r.v. Mass function Cumulative distribution function (CDF Some discrete probability distribution Binomial Geometric and Hypergeometric Poisson Uniform Expectation and variance Expectation Variance Poisson Process 3.1 Random Variables Random Variable Definition 1. A random variable (r.v. is a real valued function of the sample space S. It is any rule that associates a number with each outcome in S. i.e., a r.v. is some number associated with a random experiment. Notation: we use upper case letters X, Y, Z,, to denote random variables. Lower case x will be used to denote different values of a random variable X. A random variable is a real valued function, so the expression: X(s = x means that x is the value associated with the outcome s (in a specific sample space S by the r.v. X. Example 2. Toss a coin, the sample space is S = {H, T }. Let X be the r.v. associated with this random experiment, and let X(H = 1, X(T = 0. Or we may simply write X = x, x = 0, 1.
STAT511 Spring 2014 Lecture Notes 2 Random Variable Examples: Bernoulli rv Example 3.1.1 Check if a manufactured computer component is defect. If it is defect X = 1, if not X = 0. Example 3.1.2 Let X = 1 if the life of a light bulb is over 1000 hours, X = 0 if not. Definition 3. A Bernoulli rv has two possible values 0 and 1. A Bernoulli rv is like an indicator variable I: { 1, If event A occurs; I(A = 0, If event A doesn t occur. Example 3.1.3 Toss a coin 3 times. Let I 1 be the Bernoulli variable for the first toss, I 2 be the Bernoulli variable for the second toss, I 3 be the Bernoulli variable for the third toss. I i = 1, if head, I i = 0 if tail; for i = 1, 2, 3. Let X be the totally number of heads tossed, we have: X = I 1 + I 2 + I 3 Types of Random Variables: Discrete & Continuous A discrete rv is an rv whose possible values constitute a finite set or a countably infinite set. A continuous rv is an rv whose possible values consists of an entire interval on the real line. Example 3.1.4 X = number of tosses needed before getting a head. Example 3.1.5 X = number of calls a receptionist gets in an hour. Example 3.1. X = life span of a light bulb. Example 3.1.7 X = weight of a Purdue female student. 3.2 Probability Distributions Probability Distribution for Discrete RVs Distribution of an rv, vaguely speaking, is how an rv distributes its probabilities on real numbers. For a discrete rv, we may list the values and the probability for each value of the rv, this gives the probability distribution of the discrete rv. Such rv is said to be an rv with discrete distribution. Definition 4. The probability distribution or probability mass function (pmf of a discrete rv is defined for every number x by: p(x = P (X = x = P (all s S : X(s = x. Definition 5. Suppose that p(x depends on a quantity that can be assigned any one of a number of possible values, each with different value determining a different probability distribution. Such a quantity is called a parameter of the distribution. PMF Examples Example 3.1.8 Tossing a die, let X = outcome of the die. X = 1, 2,... Find pmf. p(1 = P (X = 1 = P (outcome of the die is 1 = 1 p(2 = P (X = 2 = P (outcome of the die is 2 = 1... =... p( = P (X = = P (outcome of the die is = 1
STAT511 Spring 2014 Lecture Notes 3 Example 3.1.9 Toss a fair coin three times, let X = number of heads, find the pmf. p(0 = P (X = 0 = P (TTT = 1 8 p(1 = P (X = 1 = P (TTH + P (THT + P (HTT = 3 8 p(2 = P (X = 2 = P (HHT + P (THH + P (HTH = 3 8 p(3 = P (X = 3 = P (HHH = 1 8 Example 3.1.10 Let X = 1 if a specific product is defect, X = 0 otherwise. And suppose p(1 = α, then p(0 = 1 α. We write: α, if x=1; p(x, α = 1 α, if x=0; 0, otherwise. Here α is called the parameter of the distribution. Cumulative Distribution Function (CDF for Discrete RVs Definition. The cumulative distribution function (cdf F (x of a discrete rv X with pmf p(x is defined for every number x by F (x = P (X x = p(y. y:y x For any number x, F (x is the probability that the observed value of X will be at most x. Proposition. For any two numbers a and b with a b, P (a X b = F (b F (a. a represents the largest possible X value that is strictly less than a. Notice that F (x is a non-decreasing function. For integers P (a X b = F (b F (a 1. CDF Examples Example 3.1.11 Roll a die. Let X = outcome of the die. X = 1, 2,... Find the cdf. What is the probability that 2 X 4? x 1 2 3 4 5 1 1 1 1 1 1 p(x 1 1 1 2 5 F (x 3 2 3 1 P (2 X 4 = F (4 F (2 = 2 3 1 = 1 2. Example 3.1.12 Toss a fair coin three times, let X = number of heads, find the cdf, what is the probability to get at least 2 heads? x 0 1 2 3 1 3 3 1 p(x 8 8 8 8 1 1 7 F (x 8 2 8 1 P (at least two H = P (X 2 = 1 P (X 1 = 1 F (1 = 1 2.
STAT511 Spring 2014 Lecture Notes 4 CDF Examples Example 7. Here is a probability distribution for a random variable X: 3.3 The Binomial Probability Distribution Binomial Distribution Definition 8 (Binomial experiment. 1. A sequence of n trials. 2. Each trial is a dichotomous trial, i.e., has two results: success (S or fail (F. 3. All trials are independent. 4. Probability of success is constant for all trials, and is denoted by p. Definition 9 (Binomial RV and Distribution. Given a binomial experiment consisting of n trials, the binomial rv X associated with this experiment is defined as: X = the number of S s among n trials The probability dist associated with binomial rv X is the binomial distribution. Binomial Examples Example 3.2.1 Toss a fair coin 4 times, let X = number of heads in n tosses. Find the pmf. ( 4 ( p(0 = P (X = 0 = P (0H = 1 4 0 2 ( 4 ( p(1 = P (X = 1 = P (1H = 1 4 1 2...... ( 4 ( p(4 = P (X = 4 = P (4H = 1 4 4 2 Binomial Examples (continued. Example 3.2.2 Toss a fair coin n times, let X = number of heads in n tosses. Find the pmf. p(0 = P (X = 0 = P (0H = ( 1 0 2 n p(1 = P (X = 1 = P (1H = ( 1 1 2 n
STAT511 Spring 2014 Lecture Notes 5 Binomial Examples (continued........ p(x = P (X = x = P (xh =...... p(n = P (X = n = P (nh = ( 1 x 2 n ( 1 n 2 n Example 3.2.3 Toss an uneven coin, the probability that the coin is head is p. Find the pmf. p(0 = P (X = 0 = P (0H = (1 p n 0 p(1 = P (X = 1 = P (1H = p(1 p (n 1 1...... p(x = P (X = x = P (xh = p x (1 p (n x x...... p(n = P (X = n = P (nh = p n n Binomial PMF Exercise A card is drawn from a standard 52-card deck. If drawing a club is considered a success, find the probability of 1. exactly one success in 4 draws (with replacement. 2. no successes in 5 draws (with replacement. For any binomial experiment with n trials and each trial has a success probability p, the binomial pmf is denoted by b(x; n, p, where n and p are two parameters associated with the binomial dist. We have: Theorem 10. { ( n b(x; n, p = x p x (1 p (n x, x=0,1,2,...,n; 0, otherwise.
STAT511 Spring 2014 Lecture Notes Binomial Example (again Example 3.2.4 (exercise 3.51 20% of all telephones of a certain type are submitted for service while under warranty. Of these 0% can be repaired, whereas the other 40% must be replaced with new units. If a company purchases ten of these telephones, what is the probability that exactly two will end up being replaced under warranty? 3.4 Geometric and Hypergeometric Distributions Geometric and Hypergeometric Distribution Definition 11 (Geometric Distribution. 1. A sequence of trials 2. Each trial is dichotomous, with outcomes S or F and P (S = p. 3. All trials are independent 4. Random variable X = the number of S before F appears is a Geometric rv, or follows a Geometric dist. 5. p is the parameter. Definition 12 (Hyper Geometric Distribution. are S. 2. Draw n out of N elements without replacement. 1. N Dichotomous elements (S and F, M 3. Random variable X = number of S in n draws is a Hypergeometric rv, or follows a Hypergeometric dist. 4. N, M, n are parameters. Geometric PMF Example 3.3.1 Toss an uneven coin, the probability of getting a head is p, so the probability of getting tail is 1 p. Let X = the number of tails before getting a head. X has a geometric dist. p(0 = P (X = 0 = P (H = p p(1 = P (X = 1 = P (T H = (1 pp... p(x = P (X = x = P (x tails and 1H = (1 p x p Proposition. The pmf of a geometric rv X is given by: Where p is the probability of success. p(x = P (X = x = P (xt and 1H = (1 p x p
STAT511 Spring 2014 Lecture Notes 7 Hypergeometric PMF Example 3.3.2 A bag with 10 balls, 3 of them are black, now take out balls from the bag, X = number of black balls follows a hypergeometric dist. With N = 10, M = 3, n =. ( 3 10 3 ( 3 10 3 p(0 = P (X = 0 = 0( ; p(1 = P (X = 1 = 1( 5 ( 10 ( 10 ( 3 10 3 ( 3 10 3 p(2 = P (X = 2 = 2( 4 ; p(3 = P (X = 3 = 3( 3 ( 10 ( 10 If X is the number of S s in a completely random sample of size n drawn from a population consisting of M S s and (NM F s, then the probability distribution of X is called hypergeometric. Proposition. The pmf of a hypergeometric rv X is given by: ( M ( N M x n x p(x = P (X = x = ( N n Where x is an integer satisfying max(0, n N + M x min(n, M, p(x is denoted by h(x; n, M, N in textbook. 3.5 Poisson Distribution Poisson Distribution Definition 13. A random variable X is said to have a Poisson distribution with parameter λ(λ > 0 if the pmf of X is: Where x = 0, 1, 2,... p(x; λ = e λ λ x Proposition. Let λ > 0, lim n b(x; n, p n = p(x; λ, if p n 0 as n and np n λ i.e. Binomial approaches Poisson when n is large ( and p is small ( 0. Thus we can use poisson to approximate binomial when n is large and p is small. Poisson Examples Example 3.3.3 Let X = the number of calls a receptionist receives in an hour, X follows a poisson dist with λ = 5. What is probability that the receptionist receives at least one call in an hour? P (at least one call = 1 P (no calls = 1 p(0 = 1 e λ λ 0 λ = 5, so P (at least one call = 1 e 5 Example 3.3.4 0.2% feral cats are infected with the feline aids (FIV in a region. What is the probability that there are exactly 10 cats infected with FIV among 1000 cats? Let X = the number of cats with FIV among 1000 cats. X Binomial, with n = 1000 and p = 0.002. So, ( 1000 P (10 FIV cats = 0.002 10 (1 0.002 (1000 10 10 Complicated... Use poisson approximation: n = 1000, p = 0.002, λ = np = 1000 0.002 = 2, so, P (10 FIV cats = p(10 = e 2 2 10 10! x! 0!
STAT511 Spring 2014 Lecture Notes 8 3. Uniform Distribution Uniform Distribution Definition 14. If an rv has any of n possible values k 1,..., k n that are equally probable, then X has a discrete uniform distribution with pmf: p(k i = 1, where i = 1, 2,...n. n Example 15. Example 3.3.5 A very simple example is: Roll a fair die, and let X = outcome of the die. pmf: p(1 = p(2 = p(3 = p(4 = p(5 = p( = 1 3.7 Expectation and Variance of a Discrete Distribution Expectation of a Discrete Distribution Definition 1. Let X be a discrete rv and let the set of all possible values of X be D and pmf of X be p(x. The expectation or mean of X, denoted by E(X or µ x is: E(X = µ x = x D x p(x Notice here that the expectation E(X is the population mean µ when a dist is given. Example 17. Let X be outcome of a die, what is the expectation of X? E(X = 1 1 + 2 1 +... + 1 = 1 + 2 +... + Expectation of a Discrete Distribution = 3.5 Exercise Use the data below to find out the expected number of the number of credit cards that a student will possess. x = # of credit cards. x 0 1 2 3 4 5 P (X = x 0.08 0.28 0.38 0.1 0.0 0.03 0.01 Properties of Expectation 1. For any given constant a and b, E(aX + b = ae(x + b E(aX = ae(x E(X + b = E(X + b 2. If the rv X has a set of possible values D and pmf p(x, then the expectation of any function h(x, denoted by E[h(X] = D h(x p(x. 3. For random variables X 1, X 2,..., X n, E(a 1 X 1 + a 2 X 2 +... + a n X n = a 1 E(X 1 + a 2 E(X 2 +... + a n E(X n
STAT511 Spring 2014 Lecture Notes 9 Example 18. Let X be the outcome of a die. What is the expectation of X 2? X 1 2 3 4 5 X 2 1 4 9 1 25 3 1 1 1 1 1 1 p(x E(X 2 = 1 1 + 4 1 +... + 3 1 15.17 Expectations of a List of Discrete Distributions Binomial b(x; n, p: Geometric p(x; p: E(X = np E(X = 1 p p Hyper Geometric h(x; n, M, N: E(X = n M N Poisson p(x; λ: E(X = λ Uniform: E(X = n i=1 k i n Variance of a Discrete Distribution Definition 19. Let X be an rv with pmf p(x and expected value E(X Then the variance of X, denoted by V ar(x or σ 2 X is: V ar(x = D (x E(X 2 p(x = E(X E(X 2 The standard deviation σ X is σ X = V ar(x Notice here that V ar(x is the population variance σ 2 when a dist is given. A shortcut formula: V ar(x = E(X 2 [E(X] 2 Example Still the die example. What is the variance of X = outcome of a fair die?
STAT511 Spring 2014 Lecture Notes 10 Variance of a Discrete Distribution Exercises 1. 5 cards are drawn, with replacement, from a standard 52-card deck. If drawing a club is considered a success, find the mean, variance, and standard deviation of X (where X is the number of successes. 2. If the probability of a student successfully passing this course (C or better is 0.82, find the probability that given 8 students (a all 8 pass. (b none pass. (c at least pass. Properties of Variance 1. Given two numbers a and b, V ar(ax + b = a 2 V ar(x V ar(ax = a 2 V ar(x V ar(x + b = V ar(x 2. V ar[h(x] = D {h(x E[h(X]}2 p(x Example: Bernoulli rv I, p(0 = p, p(1 = 1 p. What is V ar(3i + 5? Example: Still the die example. What is the variance of 2X 2? V ar(2x 2 = 4V ar(x 2 = 4 [E(X 4 (E(X 2 2] Variances of a List of Discrete Distributions Binomial b(x; n, p: V ar(x = n p (1 p Geometric p(x; p: V ar(x = 1 p p 2 Hyper Geometric h(x; n, M, N: ( N n V ar(x = n M ( N 1 N 1 M N Poisson p(x; λ: V ar(x = λ
STAT511 Spring 2014 Lecture Notes 11 Poisson Process Poisson process is a very important application of Poisson distribution. Definition 20 (Poisson Process. per unit time. 1. Given a rate α, i.e., expected number of occurrences 2. Then the X = number of occurrences during a time interval t follows a Poisson distribution with λ = αt. The expected number of occurrences is λ = αt. Exercises 1. Every second there are 2 cosmic rays hit a specific spot on earth. What is the probability that there are more than 20 cosmic rays hitting the spot within 5 seconds? 2. 2-D Poisson process. (problem 3.87 Trees are distributed in a forest according to a 2- dimensional Poisson process with parameter α = the expected number of trees per acre, and α = 80. What is the probability that in a certain quarter acre plot, there will be at most 1 trees?