Random Variables and Their Expected Values

Transcription

1

2 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function

3 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Definition Let S be the sample space of some probabilistic experiment. A function X : S R is called a random variable. Example 1. A unit is selected at random from a population of units. Thus S is the collection of units in the population. Suppose a characteristic (weight, volume, or opinion on a certain matter) is recorded. A numerical description of the outcome is a random variable. 2. S = {s = (x 1..., x n ) : x i R, i}, X (s) = i x i or X (s) = x, or X (s) = max{x 1..., x n }. 3. S = {s : 0 s < } (e.g. we may be recording the life time of an electrical component), X (s) = I (s > 1500), or X (s) = s, or X (s) = log(s).

4 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function A random variable X induces a probability measure on the range of its values, which is denoted by X (S). X (S) can be thought of as the sample space of a compound experiment which consists of the original experiment, and the subsequent transformation of the outcome into a numerical value. Because the value X (s) of the random variable X is determined from the outcome s, we may assign probabilities to the possible values of X. For example, if a die is rolled and we define X (s) = 1 for s = 1, 2, 3, 4, and X (s) = 0 for s = 5, 6, then P(X = 1) = 4/6, P(X = 0) = 2/6. The probability measure P X, induced on X (S) by the random variable X, is called the (probability) distribution of X.

5 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function The distribution of a random variable is considered known if the probabilities P X ((a, b]) = P(a < X b) are known for all a < b. Definition A random variable X is called discrete if X (S) is a finite or a countably infinite set. If X (S) is uncountably infinite, then X is called continuous. For discrete r.v. s X, P X is completely specified by the probabilities P X ({k}) = P(X = k), for each k X (S). (See part 4 of Theorem on p. 15 of 2nd lecture slide.) The function p(x) = P(X = x) is called the probability mass function of X.

6 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Consider a batch of size N = 10 products, 3 of which are defective. Draw 3 at random and without replacement and let the r.v. X denote the number of defective items. Find the pmf of X. Solution: The sample space of X is S X = {0, 1, 2, 3}, and: ( 7 )( 3 3) 1) P(X = 0) = ( 10 3 ), P(X = 1) = ( 7 2 ( 10 3 ), P(X = 2) = ( 7 )( 3 1 ( 2) 10 ), P(X = 3) = 3 ( 3 3) ( 10 3 ) Thus, the pmf of X is x p(x)

7 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Bar Graph Probability Mass Function x values

8 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Let S = {s = (x 1..., x n ) : x i = 0 or 1}, i} (so S is the sample space of n flips of a coin), and let p be the probability of 1 (i.e. probability of heads). Then, the probability measure on S is P({(x 1..., x n )}) = p P i x i (1 p) n P i x i. Let X (s) = i x i, so X (S) = {0, 1,..., n}. Then, the distribution, P X, of X, which is a probability measure on X (S), is called the Binomial distribution. Suppose n = 3 and p = 1/2. Then, P X (0) = P(X = 0) = 1 8, P X (1) = P(X = 1) = 3 8 why?, P X (2) = P(X = 2) = 3 8, P X (3) = P(X = 3) = 1 8.

9 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Three balls are randomly selected at random and without replacement from an urn containing 20 balls numbered 1 through 20. Find the probability that at least one of the balls will have number 17. Solution: Here S = {s = (i 1, i 2, i 3 ) : 1 i 1, i 2, i 3 20}, X (s) = max{i 1, i 2, i 3 }, X (S) = {3, 4,..., 20} and we want to find P(X 17) = P(X = 17) + P(X = 18) + P(X = 19) + P(X = 20). These are found from the formula ) P(X = k) = ( k 1 2 ( 20 3 ) (why?) The end result is P(X 17) =

10 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function The PMF of a Function of X Let X be a discrete random variable with range (i.e. set of possible values) X and distribution P X, and let Y = g(x ) be a function of X with range Y. Then the pmf p Y (y) of Y is given in terms of the pmf p X (x) of X by p Y (y) = x X :g(x)=y p X (x), for all y Y. Example Roll a die and let X denote the outcome. If X = 1 or 2, you win $1; if X = 3 you win $2, and if X 4 you win $4. Let Y denote your prize. Find the pmf of Y. Solution: The pmf of Y is: y p Y (y)

11 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Definition The function F X : R [0, 1] (or simply F if no confusion is possible) defined by F X (x) = P(X x) = P X ((, x]) is called the (cumulative) distribution function of the rv X. Proposition F X determines the probability distribution, P X, of X. Proof: We have that P X is determined by its value P X ((a, b]) for any interval (a, b]. However, P X ((a, b]) is determined from F X by P X ((a, b]) = F X (b) F X (a).

12 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Consider a batch of size N = 10 products, 3 of which are defective. Draw 3 at random and without replacement and let the r.v. X denote the number of defectives. Find the cdf of X. Solution: x p(x) F (x) Moreover, F ( 1) = 0. F (1.5) = Also, p(1) = F (1) F (0)

13 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Consider a random variable X with cumulative distribution function given by F (x) = 0, for all x that are less than 1, F (x) = 0.4, for all x such that 1 x < 2, F (x) = 0.7, for all x such that 2 x < 3, F (x) = 0.9, for all x such that 3 x < 4, F (x) = 1, for all x that are greater than or equal to 4.

14 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Figure: The CDF of a Discrete Distribution is a Step or Jump Function

15 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Example Let X have cdf as shown above. Use the form of the cdf to deduce the distribution of X. Solution. Since its cdf is a jump function, we conclude that X is discrete with sample space the jump points of its cdf, i.e. 1,2,3, and 4. Finally, the probability with which X takes each value equals the size of the jump at that value (for example, P(X = 1) = 0.4). These deductions are justified as follows: a) P(X < 1) = 0 means that X cannot a value less than one. b) F (1) = 0.4, implies that P(X = 1) = 0.4. c) The second of the equations defining F also implies that P(1 < X < 2) = 0, and so on.

16 Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Proposition (Properties of F X ) The distribution function F X, of a rv X satisfies: 1. lim x F X (x) = 0, and lim x F X (x) = F X (x) is an increasing (i.e. non-decreasing) function. 3. F X (x) is right continuous. Proof: See Section 4.10.

17 A unit is selected at random from a population of N units, and let the random variable X be the (numerical) value of a characteristic of interest. Let v 1, v 2,..., v N be the values of the characteristic of interest of each of the N units. Then, the expected value of X, denoted by µ X or E(X ) is defined by N E(X ) = 1 N i=1 v i Example 1. Let X denote the outcome of a roll of a die. Find E(X ). 2. Let X denote the outcome of a roll of a die that has the six on four sides and the number 8 on the other two sides. Find E(X ).

18 The expected value, E(X ) or µ X, of a discrete r.v. X having a possibly infinite sample space S X and pmf p(x) = P(X = x), for x S X, is defined as µ X = x in S X xp(x). Example Roll a die and let X denote the outcome. If X = 1 or 2, you win $1; if X = 3 you win $2, and if X 4 you win $4. Let Y denote your prize. Find E(Y ). Solution: The pmf of Y is: y p Y (y) Thus, E(Y ) = = 2.667

19 Example You are given a choice between accepting $ = or roll a die and win X 2. What will you choose and why? Solution: If the game will be played several times your decision will be based on the value of E(X 2 ). (Why?) To find this use = 91. Proposition Let X be a discrete r.v. taking values x i, i 1, having pmf p X. Then, E[g(X )] = i g(x i )p X (x i ).

20 Example A product that is stocked monthly yields a net profit of b dollars for each unit sold and a net loss of l dollars for each unit left unsold at the end of the month. The number monthly demand (i.e. # of units ordered) for this product at a particular store location during a given month is a rv having pmf p(k), k 0. If the store stocks s units, find the expected profit. Solution: Let X be the monthly demand. The random variable of interest here is the profit Y = g(x ) = bx (s X )l, if X s = bs, if X > s We want the expected profit, i.e. E(Y ).

21 Corollary For constants a, b we have Definition E(aX + b) = ae(x ) + b. The variance, σx 2 or Var(X ), and standard deviation, σ X or SD(X ), of a rv X are σx 2 = Var(X ) = E(X µ X ) 2, σ X = σx 2. Proposition Two common properties of the variance are 1. Var(X ) = E(X 2 ) µ 2 X 2. Var(aX + b) = a 2 Var(X )

22 The Bernoulli Random Variable A r.v. X is called Bernoulli if it takes only two values. The two values are referred to as success (S) and failure (F), or are re-coded as 1 and 0. Thus, always, S X = {0, 1}. Experiments resulting in a Bernoulli r.v. are called Bernoulli. Example 1. A product is inspected. Set X = 1 if defective, X = 0 if non-defective. 2. A product is put to life test. Set X = 1 if it lasts more than 1000 hours, X = 0 otherwise. If P(X = 1) = p, we write X Bernoulli(p) to indicate that X is Bernoulli with probability of success p.

23 The Bernoulli Random Variable If X Bernoulli(p), then Its pmf is: x 0 1 p(x) 1 p p Its expected value is, E(X ) = p Its variance is, σ 2 X = p(1 p). (Why?)

24 The Binomial Random Variable A experiment consisting of n independent replications of a Bernoulli experiment is called a Binomial experiment. If X 1, X 2,..., X n are the Bernoulli r.v. for the n Bernoulli experiments, Y = n X i = the total number of 1s, i=1 is the Binomial r.v. Clearly S Y = {0, 1,..., n}. We write Y Bin(n, p) to indicate that Y is binomial with probability of success equal to p for each Bernoulli trial.

25 The Binomial Random Variable If Y Bin(n, p), then Its pmf is: ( n P(Y = k) = k ( ) ( ) ( ) n n 1 n Use x = n and x 2 = nx x x 1 x 1. Its expected value is E(Y ) = np 2. Its variance is σy 2 = np(1 p) ) p k (1 p) n k, k = 0, 1,..., n ( ) n 1 to get. x 1

26 Example A company sells screws in packages of 10 and offers a money-back guarantee if two or more of the screws are defective. If a screws is defective with probability 0.01, independently of other screws, what proportion of the packages sold will the company replace? Solution: 1 P(X = 0) P(X = 1) = 0.004

27 Example Physical traits, such as eye color, are determined from a pair of genes, each of which can be either dominant (d) or recessive (r). One inherited from the mother and one from the father. Persons with genes (dd) (dr) and (rd) are alike in that physical trait. Assume that a child is equally likely to inherit either of the two genes from each parent. If both parents are hybrid with respect to a particular trait (i.e. both have pairs of genes (dr) or (rd)), find the probability that three of their four children will be hybrid with respect to this trait. Solution: Probability that an offspring of two hybrid parents is also hybrid is Thus, the desired probability is ( ) =

28 Example In order for the defendant to be convicted in a jury trial, at least eight of the twelve jurors must enter a guilty vote. Assume each juror makes the correct decision with probability 0.7 independently of other jurors. If 40% of defendants in such jury trials are innocent, what is the probability that the jury renders the correct verdict to a randomly selected defendant? Solution: Let B = {jury renders the correct verdict}, and A = {defendant is innocent}. Then, according to the Law of Total Probability, P(B) = P(B A)P(A) + P(B A c )P(A c ) = P(B A)0.4 + P(B A c )0.6.

29 Solution Continued: Next, let X denote the number of jurors who reach the correct verdict in a particular trial. Thus, X Bin(12, 0.7), and P(B A) = P(X 5) = 1 12 P(B A c ) = P(X 8) = k=8 4 k=0 ( 12 k ( ) k k = , k ) 0.7 k k = Thus, P(B) = P(B A)0.4 + P(B A c )0.6 =

30 Example A communications system consisting of n components works if at least half of its components work. Suppose it is possible to add components to the system, and that currently the system has n = 2k 1 components. 1. Show that by adding one component the system becomes more reliable for all integers k Show that this is not necessarily the case if we add two components to the system. Solution: 1. Let A n = {the system works when it has n components}. Then A 2k 1 = {k or more of the 2k 1 work} A 2k = A 2k 1 {k 1 of the original 2k 1 work, and the 2kth works}

31 Solution Continued: It follows that A 2k 1 A 2k. Thus, P(A 2k 1 ) P(A 2k ). 2. Using the same notation, A 2k+1 = {k + 1 or more of the original 2k 1 work} {k of the original 2k 1 work, and at least one of the 2kth and (2k + 1)th work} {k 1 of the original 2k 1 work, and both the 2kth and (2k + 1)th work}. It is seen that A 2k 1 is not a subset of A 2k+1, since, for example, A 2k 1 includes the outcome {k of the original 2k 1 work} but A 2k+1 does not. It is also clear that A 2k+1 is not a subset of A 2k 1. Thus, more information is needed to compare the reliability of the two systems.

32 Example (Example Continued) Suppose each component functions with probability p independently of the others. For what value of p is a (2k + 1)-component system more reliable than a (2k 1)-component system? Solution: Let X denote the number of the first 2k 1 that function. Then, P(A 2k 1 ) = P(X k) = P(X = k) + P(X k + 1) P(A 2k+1 ) = P(X k + 1) + P(X = k)(1 (1 p) 2 ) + P(X = k 1)p 2 and P(A 2k+1 ) P(A 2k 1 ) > 0 iff p > 0.5.

33 The Hypergeometric Random Variable The hypergeometric distribution arises when a simple random sample of size n is taken from a finite population of N units of which M are labeled 1 and the rest are labeled 0. The number X of units labeled 1 in the sample is a hypergeometric random variable with parameters n, M and N. This is denoted by X Hypergeo(n, M, N) If X Hypergeo(n, M, N), its pmf is ( M )( N M ) x n x P(X = x) = ( N n) Note that P(X = x) = 0 if x > M, or if n x > N M.

34 Applications of the Hypergeometric Distribution Read Example 8i, p.161. (LTP with hypergeometric)) Quality control is the primary use of the hypergeometric distribution. The following is an example of a different use. Example (The Capture/Recapture Method) This method is used to estimate the size N of a wildlife population. Suppose that 10 animals are captured, tagged and released. On a later occasion, 20 animals are captured. Let X be the number of recaptured animals. If all ( N 20) possible groups are equally likely, X is more likely to take small values if N is large. The precise form of the hypergeometric pmf can be used to estimate N from the value that X takes.

35 If X Hypergeo(n, M, N) then, Its expected value is: µ X = n M N Its variance is: σx 2 = n M ( 1 M ) N n N N N 1 N n N 1 is called finite population correction factor Binomial Approximation to Hypergeometric Probabilities If n 0.05 N, then P(X = x) P(Y = x), where Y Bin(n, p = M/N).

36 Example n = 10 electronic toys are selected at random from a batch delivery. We want the probability that 2 of the 10 will be defective. 1. If the batch has N = 20 toys with 5 of them defective, )( 15 ) 8 P(X = 2) = ( 5 2 ( 20 ) = Application of the binomial(n = 10, p = 0.25) approximation gives P(Y = 2) = If N = 100 and M = 25 (so p remains 0.25), we have )( 75 ) 8 P(X = 2) = ( 25 2 ( ) = 0.292, It is seen that the binomial probability of provides a better approximation when N is large.

37 Binomial Approximation to Hypergeometric Probabilities As N, M, with M/N p, and n remains fixed, ( M )( N M ) x n x ( N n) ( ) n p x (1 p) n x, x = 0, 1,..., n. x One way to show this is via Stirling s formula for approximating factorials: n! 2πn( n e )n, or more precisely n! = ( n ) n 2πn e λ n where e 1 12n + 1 < λ n < 1 12n Use this on the left hand side and note that the terms resulting from 2πn tend to 1, and powers of e cancel. Thus,

38 ( M )( N M ) x n x ( N n) ( ) n x M M (N M) N M (N n) N n N N (M x) M x (N M n + x) N M n+x M M (M x) M x = (1 + x M x )M x M x (N n) N n N N = (1 n N )N (N n) n (N M N M n x (N M n + x) N M n+x = (1 + N M n + x )N M n+x (N M) n

39 The Negative Binomial Random Variable In the negative binomial experiment, a Bernoulli experiment is repeated independently until the r th 1 is observed. For example, products are inspected, as they come off the assembly line, until the r th defective is found. The number, Y, of Bernoulli experiments until the rth 1 is observed is the negative binomial r.v. If p is the probability of 1 in a Bernoulli trial, we write Y NBin(r, p) If r = 1, Y is called the geometric r.v.

40 The Negative Binomial Random Variable If Y NBin(r, p), then Its pmf is: P(Y = y) = Its expected value is: ( ) y 1 p r (1 p) y r, y = r, r + 1,... r 1 E(Y ) = r p Its variance is: σ 2 Y = r(1 p) p 2

41 If r = 1 the Negative Binomial is called Geometric: P(X = x p) = p(1 p) x 1, x 1. The memoryless property: For integers s > t, Example P(X > s X > t) = P(X > s t) Independent Bernoulli trials are performed with probability of success p. Find the probability that r successes will occur before m failures. Solution: r successes will occur before m failures iff the rth success occurs no later than the (r + m 1)th trial. Hence the desired probability is found from r+m 1 k=r ( k 1 r 1 ) p r (1 p) k r

42 Example A candle is lit every evening at dinner time with a match taken from one of two match boxes. Assume each box is equally likely to be chosen and that initially both contained N matches. What is the probability that there are exactly k matches left, k = 0, 1,..., N, when one of the match boxes is first discovered empty? Solution: Let E be the event that box #1 is discovered empty and there are k matches in box #2. E will occur iff the (N + 1)th choice of box #1 is made at the (N N k)th trial. Thus, ( ) 2K k P(E) = 0.5 2N k+1, N and the desired probability is 2P(E).

43 Example Three electrical engineers toss coins to see who pays for coffee. If all three match, they toss another round. Otherwise the odd person pays for coffee. 1. Find the probability of a round of tossing resulting in a match. Answer: = Let Y be the number of times they toss coins until the odd person is determined. What is the distribution of Y? Answer: Geometric with p = Find P(Y 3). Answer: P(Y 3) = 1 P(Y = 1) P(Y = 2) = =

44 Poisson X Poisson(λ) if P(X = x λ) = e Use e t = i=0 t i i! EX = λ, σ 2 X = λ. to get. λ λx I (x {0, 1,...}). x! A recursion relation: P(X = x) = λ P(X = x 1), x 1. x This recursion relation, and that for the Binomial, can also be used to establish that if Y Bin(n, p) then as np λ P(Y = k) P(X = k). Poisson approximation to hypergeometric probabilities.

45 Proposition If Y Bin(n, p), with n 100, p 0.01, and np 20, then P(Y k) P(X k), k = 0, 1, 2,..., n, where X Poisson(λ = np). Example Due to a serious defect, n = 10, 000 cars are recalled. The probability that a car is defective is p = If Y is the number of defective cars, find: (a) P(Y 10), and (b) P(Y = 0). Solution. Let X Poisson(λ = np = 5). Then, (a) P(Y 10) P(X 10) = 1 P(X 9) = (b) P(Y = 0) P(X = 0) = e 5 =

46 The Poisson process When we record the number of occurrences as they accumulate with time, we denote X (t) = number of occurrences in the time interval [0, t]. Definition The number of occurrences as a function of time, X (t), t 0, is called a Poisson process, if the following assumptions are satisfied. 1. The probability of exactly one occurrence in a short time period of length t is approximately α t. 2. The probability of more than one occurrence in a short time period of length t is approximately The number of occurrences in t is independent of the number prior to this time.

47 The parameter α in the first assumption specifies the rate of the occurrences, i.e. the average number of occurrences per time unit. Because in an interval of length t 0 time units we would expect, on average, λ = α t 0 occurrences, we have the following Proposition Let X (t), t 0 be a Poisson process. 1. For each fixed t 0, X (t 0 ) Poisson(λ = α t 0 ). Thus, P(X (t 0 ) = k) = e αt (αt 0 0) k, k = 0, 1, 2, 2. If t 1 < t 2 are two positive numbers, k! then X (t 2 ) X (t 1 ) Poisson(α (t 2 t 1 ))

48 Example Continuous electrolytic inspection of a tin plate yields on average 0.2 imperfections per minute. Find: 1. The probability of one imperfection in three minutes 2. The probability of at most one imperfection in 0.25 hours. Solution. 1) Here α = 0.2, t = 3, λ = αt = 0.6. Thus, P(X (3) = 1) = F (1; λ = 0.6) F (0; λ = 0.6) = = ) Here α = 0.2, t = 15, λ = αt = 3.0. Thus, P(X (15) 1) = F ( 1; λ = 3.0 ) =.199.