Chapter 4: Probability Distributions

Transcription

1 Chapter 4: Probability Distributions 4.1 Random Variables A random variable is a function X that assigns a numerical value x to each possible outcome in the sample space An event can be associated with a single value of the random variable, or it can be associated with a range of values of the random variable. The probability of an event can then be described as: P A = P(X = x i ) or P A = P(x l X x u ) There could also be other topology for the random variable to describe the event. If x i, i = 1,2,, N are all the possible values of random variable associated with the sample space, then N i=1 P(X = x i ) = 1

2 e.g. Each (composite) outcome consists of 3 ratings (M,P,C). Let M 1, P 1 and C 1 be preferred ratings. Let X be the function that assigns to each outcome the number of preferred ratings each outcome possesses. Since each outcome has a probability, we can compute the probability of getting each value x = 0,1,2,3 of the function X x P(X = x) M 1 M 2 P 1 P 2 P 1 probabilities 0.03 C 1 C 2 C 3 C 1 C 2 C 3 C 1 C 2 C x

3 Random variables X can be classified by the number of values x they can assume. The two common types are discrete random variables with a finite or countably infinite number of values continuous random variables having a continuum of values for x 1. A value of a random variable may correspond to several random events. 2. An event may correspond to a range of values (or ranges of values) of a random variable. 3. But a given value (in its legal range) of a random variable corresponds to a random event. 4. Different random values of the random variable correspond to mutually exclusive random events. 5. Each value of a random variable has a corresponding probability. 6. All possible values of a random variable correspond to the entire sample space. 7. The summation of probabilities corresponding to all values of a random variable must equal to unity.

4 A fundamental problem is to find the probability of occurrence for each possible value x of the random variable X. P X = x = P(A) all outcomes A assigned value x This is the problem of identifying the probability distribution for a random variable. The probability distribution of a discrete random variable X can be listed as a table of the possible values x together with the probability P(X = x) for each e.g. x 1 P(X = x 1 ) x 2 P(X = x 2 ) x 3 P(X = x 3 ) It is standard notation to refer to the values P(X = x) of the probability distribution by f(x) f(x) P(X = x) The probability distribution always satisfies the conditions f x 0 and f x = 1 all x

5 e.g. f x = x 2 2 for x = 1,2,3,4 e.g. f x = x2 25 for x = 0,1,2,3,4 Since the probability distribution for a discrete random variable is a tabular list, it can also be represented as a histogram, the probability histogram. For a discrete random variable, the height for the bin value x is f(x), the width of the bin is meaningless. For a discrete random variable, the probability histogram is commonly drawn either with touching bins (left) or in Pareto style (right - also referred to as a bar chart). f(x) for number preferred ratings

6 Of course one can also compute the cumulative distribution function (or cumulative probability function) F x = P X x for all x and plot it in the ways learned in chapter 2 (with consideration that the x-axis is not continuous but discrete). F(x) for number preferred ratings We now start to discuss the probability distributions for many discrete random variables that occur in nature

7 4.2 Binomial Distribution Bernoulli distribution: In probability theory and statistics, the Bernoulli distribution, named after Swiss scientist Jacob Bernoulli, is a discrete probability distribution, which takes value 1 with success probability p and value 0 with failure probability q = 1 p. So if X is a random variable with this distribution, we have: P X = 1 = p; p X = 0 = q = 1 p. Mean and variance of a random variable X: (1) Mean (mathematical expectation, expectation, average, etc): (2) Variance: μ = x = E X = xp(x = x) Var X = E x x 2 = σ 2 = x μ 2 P(X = x) i σ is called the standard deviation. For random variable with Bernoulli distribution, we have μ = E X = p Var X = σ 2 = 1 p 2 p + p 2 q = q 2 p + p 2 q = pq p + q = pq i

8 Binomial Distribution: We can refer to the ordered sequence of length n as a series of n repeated trials, where each trial produces a result that is either success or failure. We are interested in the random variable that reports the number x successes in n trials. Each trial is a Bernoulli trial which satisfies a) there are only two outcomes for each trial b) the probability of success is the same for each trial c) the outcomes for different trials are independent We are talking about the events A i in the sample space S where A 1 = s. _; A 2 = _ s _. _; A 3 = s. _; ; A n = _. s; where by b) P(A 1 ) = P(A 2 ) = = P(A n ) and by c) P(A i A j ) = P(A i ) P(A j ) for all distinct pairs i, j

9 e.g. police roadblock checking for drivers who are wearing seatbelts condition a): two outcomes: y or n conditions b) &c): if the events A 1 to A n contain all cars stopped, then b) and c) will be satisfied If however, event A 1 is broken into two (mutually exclusive sub-events), A 1< which is all events s _ _ and driver 1 is less than 21 and A 1> which is all events s _ _ and driver 1 is 21 or older it is entirely likely that P(A 1< ) P(A 1> ), and we would not be dealing with Bernoulli trials. If the someone caught not wearing a seatbelt began to warn oncoming cars approaching the roadblock, then P(A i A j ) P(A i ) P(A j ) for all i, j pairs and we would also not be dealing with Bernoulli trials. Note that in our definition of Bernoulli trials the number of trials n is fixed in advance

10 All Bernoulli trials of length n have the same probability distribution!!!! (a consequence of the assumptions behind the definition of Bernoulli trials) This probability distribution is called the Binomial probability distribution for n. (it is called this because each trial has a binomial outcome s or f and the sequences generated (the composite outcomes) are binomial sequences.)

11 e.g. Binomial probability distribution for n = 3. Sample space has 2 3 = 8 outcomes sss ssf sff fff sfs fsf fss ffs RV values P(sss) = 1/8 = ½ ½ ½; P(ssf) = 1/8 = ½ ½ (1 ½); P(fsf) = 1/8 = (1 ½) ½ (1 ½); etc. Probability Distribution x f(x) 1/8 3/8 3/8 1/8 3 0 ½ 0 1 ½ ½ 1 1 ½ ½ 2 1 ½ ½ 3 1 ½ 0

12 From this example, we see that the binomial probability distribution, which governs Bernoulli trials of length n is: f(x) b x; n, p = n x px 1 p n x (BPD) where p is the (common) probability of success in any trial, and x = 0, 1, 2,., n Note: 1. The term on the RHS of (BPD) is the x th term of the binomial expansion of i.e. p + (1 p) n = which also proves that n x=0 n x p + (1 p) n n x=0 n x px (1 p) n x px (1 p) n x = 1 n = 1 2. (BPD) is a 2-parameter family of distribution functions characterized by choice of n and p.

13 e.g. In 60% of all solar-heat installations, the utility bill is reduced by at least 1/3. What is the probability that the utility bill will be reduced by at least 1/3 in a) 4 of 5 installations? b) at least 4 of 5 installation? a) s = at least 1/3 (i.e. 1/3 or greater) f = less than 1/3 P(A i ) = p = 0.6 Assume c) of Bernoulli trial assumptions holds. 5 Then f(4) = b(4; 5, 0.6) = b) We want f(4) + f(5) = b(4; 5, 0.6) + b(5; 5, 0.6) =

14 Examples of binomial distribution

15 Cumulative binomial probability distribution B x; n, p b k; n, p x k=0 (CBPD) is the probability of x or fewer successes in n Bernoulli trials, were p is the probability of success on each trial. From (CBPD) we see b x; n, p = B x; n, p B(x 1; n, p) Values of B x; n, p are tabulated for various n and p values in Table 1 of Appendix B

16 Cumulative binomial distribution cumulative probability

17 e.g. probability is 0.05 for flange failure under a given load L. What is the probability that, among 16 columns, a) at most 2 will fail b) at least 4 will fail a) B 2; 16, 0.05 = b 0; 16, b 1; 16, b(2; 16, 0.05) b) 1.0 B 3; 16, 0.05

18 e.g. Claim: probability of repair for a hard drive within 12 months is 0.10 Preliminary data show 5 of 20 hard drives required repair in first 12 months of manufacture Does initial production run support the claim? s = repair within 12 months. p = Assume Bernoulli trials. 1.0 B(4; 20, 0.10) = is the probability of seeing 5 or more hard drives requiring repair in 12 months. This says that in only 4% of all year-long periods (i.e. in roughly 1 year out of 25) should one see 5 or more hard drives needing repair. The fact that we saw this happen in the very first year makes us suspicious of the manufacturers claim (but does NOT prove that manufacturers claim is wrong!!!!!!!)

19 Shape of binomial probability histograms e.g. b(x; 5, p) positively skewed symmetric negatively skewed b(x; n, 0.5) will always be symmetric: b(x; n, 0.5) = b(n x; n, 0.5) b(x; n, p) will always be positively skewed for p < 0.5 (Tail on positive side) will always be negatively skewed for p > 0.5 (Tail on negative side)

20 4.3 Hypergeometric probability distribution In Bernoulli trials, one can get s with probability p and f with probability 1 p in every trial (i.e. Bernoulli trials can be thought of as sample with replacement ) Consider a variation of the problem, in which there are total of only a outcomes available that are successes (have RV values = s ) and N a outcomes that are failures. (e.g. there are N radios, a of them are defective and N a of them work.) We want to run n trials, (e.g. in each trial we pick a radio), but outcomes are sampled without replacement (that is, once a radio is picked, it is no longer available to be picked again). As we run each trial, we assume that whatever outcomes are left, whether having RV value s or f, have the same chance of being selected in the next trial (i.e. we are assuming classical probability where the chance of being picking a particular value of a RV is in proportion to the number of outcomes that have that RV value). Thus, for x a, the probability of getting x successes in n trials if there will be a successes in N trials is the number of n-arrangements (permutations) having x successes and n x failures the number n arrangements (permutations) of N things

21 That is... _ trial n pick x of the trials: pick x of the a outcomes and arrange them in all possible ways in those x trials: pick n x of the N a outcomes and arrange them in all possible ways in the remaining n x trials: total possible n outcomes n C x ways a P x ways N a P n x ways P n N Therefore f(x) = n C x ap x P n x N a N P n

22 i.e. f x = n! n x! x! a! a x! N! N n! N a! N a n x! = a! a x! x! = N a! N a n x! n x! N! N n! n! a x N a n x N n, This defines the hypergeometric probability distribution h x; n, a, N = a x N a n x N n, x = 0, 1,2,, a; n N

23 e.g. PC has 20 identical car chargers, 5 are defective. PC will randomly ship 10. What is the probability that 2 of those shipped will be defective? h 2; 10,5,20 = = 5! 3! 2! 15! 7! 8! 20! 10! 10! = 5! 15! 10! 10! 3! 2! 7! 8! 20! = 5! 15! 10! 10! 3! 2! 20! 7! 8! = = = = e.g. redo using 100 car chargers and 25 defective h 2; 10,25,100 = e.g. approximate this using the binomial distribution = = b 2; 10, p 25/100 = = 0.282

24 The hypergeometric distribution h x; n, a, N approaches the binomial distribution b(x; n, p = a ) in the limit N N i.e. the binomial distribution can be used to approximate the hypergeometric distribution when n N 10

25 4.4 Mean and Variance of a Probability Distribution Consider the values x 1, x 2,, x n As discussed in Chapter 2, the sample mean is x = n i=1 n x i n = x i 1 n i=1 We can view each term in the RHS as x i f(x i ) where f x i = 1 is the probability n associated with each value (each value appears once in the list, and each is equally likely) Let X be a discrete random variable having values x 1, x 2,, x n, with probabilities f(x i ). The mean value of the RV, aka. the mean value of the probability distribution, is μ = all x x f(x)

26 e.g. Mean value for the probability distribution of the number of heads obtained in 3 flips of a coin. There are 2 3 = 8 outcomes. The RV number of heads in 3 flips has 4 possible values, 0 1, 2, and 3 heads having probabilities f(0) = 1/8; f(1) = 3/8; f(2) = 3/8; f(3) = 1/8. Therefore the mean value is μ = = 3 2 The mean value for the Binomial distribution n = x x=1 n μ = x b x; n, p = x x=0 n! n x! x! px (1 p) n x = n p Let y = x 1 and m = n 1 μ = n p m y=0 n x=1 n 1! n x! = n x=0 n x=1 m! m y! y! py (1 p) m y n x n n 1! n x! px (1 p) n x 1 (x 1)! px 1 (1 p) n x The mean value for the binomial distribution b x; n, p is x x! p px 1 (1 p) n x = n p [p 1 p ] m = n p 1 m μ = n p

27 e.g. Since the RV number of heads in three tosses is a Bernoulli trial RV with p = 0.5, its mean value must be n p = 3 ½ = 3/2 as shown on the previous slide. The mean value of the hypergeometric distribution h(x; n, a, N) is given by μ = n a N (This is easy to remember. The formula is similar to the binomial distribution if one recognizes p = a N as the hypergeometric probability in the limit of large N.) e.g. PC has 20 identical car charges, 5 are defective. PC will randomly ship 10. On average (over many trials of shipping 10), how many defective car chargers will be included in the order. We want the mean of h(x; 10,5,20). The mean value is μ = 10 5/20 = 2.5

28 Recall from chapter 2, that the sum of the sample deviations n i=1 x i x = 0 If μ is the mean of the probability distribution f(x), then note that x μ f(x) = x f x μ f x = μ μ = 0 all x all x all x Therefore, in analogy to the sample variance defined in Chapter 2, we define the variance of the probability distribution f(x) as σ 2 = x μ 2 f(x) all x Similarly we define the standard deviation of the probability distribution f(x) as σ = σ 2 = x μ 2 f(x) all x

29 The variance for the binomial distribution b(x; n, p) σ 2 = n p 1 p = μ (1 p) e.g. The standard deviation for throwing heads in 3 flips of a coin is σ = (1 1 2 ) = 3 4 = 3 2 = The variance for the hypergeometric distribution is σ 2 = n a N 1 a N n N N 1 1 as N e.g. The standard deviation for the number of defective car chargers in shipments of 10 is σ = = = 0.99

30 The moments of a probability distribution The k th moment about the origin (usually just called the k th moment) of a probability distribution is defined as μ k = x k f(x) Note: the mean of a probability distribution is the 1 st moment (about the origin) all x The k th moment about the mean of a probability distribution is defined as μ k = (x μ) k f(x) all x Notes: the 1 st moment about the mean, μ 1 = 0 the 2 nd moment about the mean μ 2 is the variance the 3 rd moment about the mean μ 3 /σ 3 is the skewness (describes the symmetry) the 4 th moment about the mean μ 3 /σ 4 is the kurtosis (describes the peakedness )

31 Note: σ 2 = all x x μ 2 f(x) = x 2 f x 2μ x f x all x Therefore we have the result all x = (x 2 2xμ + μ 2 ) f(x) all x + μ 2 f x σ 2 = μ 2 μ 2 all x = μ 2 2μ 2 + μ 2 Since computation of μ 2 and μ 2 does not involve squaring differences within the sum, they can be more straightforward to compute. e.g. Consider the R.V. which is the number of points obtained on a single roll of a die. The R.V. has values 1,2,3,4,5,6. What is the variance of the probability distribution behind this RV? The probability distribution is f(x) = 1/6 for each x. Therefore the mean is μ = = = 7 2 The second moment about the origin is μ 2 = = 91 6 Therefore σ 2 = =

32 4.5 Chebyshev s Theorem Theorem 4.1 If a probability distribution has mean μ and standard deviation σ, then the probability of getting a value that deviates from μ by at least k σ is a most 1 k 2 i.e. the probability P(x) for getting a result x such that x μ k σ satisfies P x 1 k 2 Chebyshev s theorem quantifies the statement that the probability of getting a result x decreases as x moves further away from μ Theorem 4.1 can be stated as P( x μ kσ) 1 k 2 Note: k can be any positive number (it does not have to be an integer). Corollary 4.1 If a probability distribution has mean μ and standard deviation σ, then the probability of getting a value that deviates from μ by at most k σ is at least 1 1 k 2 P x μ kσ 1 1 k 2

33 e.g. The number of customers who visit a car dealer s showroom on a Saturday morning is an RV with mean 18 and standard deviation 2.5. With what probability can we assert there will be more than 8 but fewer than 28 customers. This problem sets k σ = 10, making k = 4. Thus P x = Chebyshev s theorem holds for all probability distributions, but it works better for some than for others (gives a sharper estimate).

34 4.6 Poisson distribution Consider the binomial distribution b x; n, p = n x px (1 p) n x Write p as p = λ/n where λ is a constant. In the limit n, then p 0 and the binomial distribution becomes the Poisson probability distribution f x; λ = λx e λ x! for x = 0, 1, 2, 3, As derived, the Poisson distribution describes the probability distribution for an infinite (in practice very large) number of Bernoulli trials when the probability of success in each trial is vanishingly small (in practice very small).

35 As the Poisson distribution describes probabilities for a sample space in which each outcome is countably infinite in length, we have to technically modify the third Axiom (property) that probabilities must obey to include such sample spaces. The third axiom stated that the probability function is an additive set function. The appropriate modification is Axiom 3 If A 1, A 2, A 3, is a countably infinite sequence of mutually exclusive events in S, then P A 1 UA 2 A 3 = P A 1 + P A 2 + P A 3 + Note that the Poisson distribution satisfies all x f(x; λ) = 1 Proof: x=0 λ x e λ x! = e λ x=0 λ x x! = e λ e λ = 1 Taylors series expansion of e λ The cumulative Poisson distribution F x; λ = f(k; λ) x k=0 is tabluated for select values of x and λ in Appendix B (Table 2)

36 e.g. 5% of bound books have defective bindings. What is the probability that 2 out of 100 books will have defective bindings using (a) the binomial distribution, (b) the Poisson distribution as an approximation (a) b(2;100,0.05) = = (b) λ = = 5. f 2; 5 = 52 e 5 2! = e.g. There are 3,840 generators. The probability is 1/1,200 that any one will fail in a year. What is the probability of finding 0, 1, 2, 3, 4, failures in any given year λ = 3840 /1200 = 3.2. We want the probabilities f(0; 3.2), f(1; 3.2), f(2; 3.2) etc. Using the property f x; λ = F x; λ F x 1; λ we can compute these probabilities from Table 2 Appendix B x f x;

37 The mean value for the Poisson probability distribution is μ = λ The variance for the Poisson probability distribution is σ 2 = λ i.e. the standard deviation for the Poisson distribution is σ = λ Proof for mean: μ = x λx e λ Let y = x 1 x=0 μ = λ e λ x! y=0 = λ y y! = λe λ x=1 λ x 1 (x 1)! λ e λ e λ = λ The average λ is usually approximated by running many long (but finite) trials. e.g. An average of 1.3 gamma rays per millisec is recorded coming from a radioactive substance. Assuming the RV number of gamma rays per millisec has a probability distribution that is Poisson (aka, is a Poisson process), what is the probability of seeing 1 or more gamma rays in the next millisec λ = 1.3. Want P X 1 = 1.0 P X = 0 = e 1.3 = 1.0 e 1.3 = !

38 4.7 Poisson Processes Consider a random process (a physical process controlled, wholly or in part, by a chance mechanism) in time. To find the probability of the process generating x success over a time interval T, divide T into n equal interval t = T/n. (n is large, t is small) Assume the following hold: 1. The probability of success during t is α t 2. The probability of more than one success during t is negligible 3. The probability of success during each time interval t does not depend on what happened in a prior interval. These assumptions describe Bernoulli trials, with n = T/ t and p = α t and the probability of x successes in n intervals is b(x; T, α t). t As n, p 0 (as t 0) and the probability of x successes is governed by the Poisson probability distribution with λ = np = αt Since λ is the mean (average) number of successes over time T, we see that α is the mean number of successes per unit time.

39 e.g. A bank receives, on average, 6 bad checks per day. What are the probabilities it will receive (a) 4 bad checks on a given day (b) 10 bad checks over a 2 day period (a) α = 6. λ = 6 1 Therefore f(4; 6) = 64 e 6 4! (b) α = 6. λ = 6 2 = 12 Therefore f 10; 12 = 1210 e 12 10! = = F 10; 12 F 9; 12 = e.g. a process generates 0.2 imperfections per minute. Find probabilities of (a) 1 imperfection in 3 minutes (b) at least 2 imperfections in 5 minutes (c) at most 1 imperfection in 15 minutes (a) λ = = 0.6. Want f 1; 0.6 = F 1; 0.6 F(0; 0.6) (b) λ = = 1.0. Want 1.0 F 1; 1.0 (c) λ = = 3.0. Want F 1; 3.0

40 4.8 Geometric and Negative Binomial Distributions Consider the sample space of outcomes for countably infinite Bernoulli trials (i.e. the three Bernoulli assumptions hold) In particular s occurs with probability p and f with probability 1-p We want to know the probability that the first success occurs on the x th trial. Divide the sample space into the following events A 1 s A1 f A 1 A1 = S A 2 f s _ A 2 f f _ A 2 A 2 = A1 A 3 f f s A 3 f f f A 3 A 3 = A 2 A 4 f f f s _ A 4 f f f f _ A 4 A 4 = A 3 etc A 2 A 1 P(A 1 ) = p A 3 P(A 3 ) = p 1 p 2 P(A 2 ) = p 1 p A 5 P(A 5 ) = p 1 p 4 A 4 P(A 4 ) = p 1 p 3 A 6 P(A 6 ) = p 1 p 5 A 7

41 Since the sum of the probabilities of all outcomes must =1, from the diagram we see that P A 1 + P A 2 + P A 3 + P A 4 + = p + p 1 p + p 1 p 2 + p 1 p 3 + = p(1 p) x 1 x=1 Let the sample space consist of outcomes each of which consists of infinitely countable Bernoulli trials. Let p be the probability of success in each Bernoulli trial. Then the geometric probability distribution = 1 g x; p = p(1 p) x 1, x = 1, 2, 3, 4, describes the probability that the first success occurs on the x th trial. e.g. A measuring device has a 5% probability of showing excessive drift during a measurement. What is the probability that the first time the device exhibits successive drift occurs on the sixth measurement? p = We want g 6; 0.05 = 0.05(0.95) 5 = 0.039

42 Assume you are dealing with Bernoulli trials governed by probability p and you would like to know how many trials x you need to make in order to observe r successes. (Clearly r x) To have exactly r successes in x trials, the r th success has to occur on trial x, and the previous r 1 successes have to occur in the previous x 1 trials. Therefore the probability that the r th success occurs on the x th trial must be f(x) = (probability of r 1 successes in x 1 trials) x (probability of s on trial x) = b r 1; x 1, p p f(x) = x 1 r 1 pr 1 (1 p) x r p = x 1 r 1 pr (1 p) x r This is the negative binomial probability distribution f x = x 1 r 1 pr 1 p x r for x = r, r + 1, r + 2, As n k = n n k, the negative binomial probability distribution can also be written f x = x 1 x r pr 1 p x r It can be shown that distribution x 1 x r = x 1x r x r explaining the name negative binomial

43 Recap: Sample space: outcomes are Bernoulli trials of fixed length n. Probability of s is p. Probability of getting x outcomes in the n trials is given by the binomial distribution b x; n, p, x = 0,1, 2, 3,, n If n is large and p is small, b x; n, p f x; λ where λ = np and f x; λ is the Poisson distribution Sample space: outcomes are Bernoulli trials of countably infinite length. Probability of s is p. Probability of getting the first success on the x th trial is given by the geometric distribution g x; p, x = 1, 2, 3, 4,. Probability of getting exactly r successes in x trials is given by f x = b r 1; x 1, p p, x = r, r + 1, r + 2,

44 Recap: Sample space: Time recordings of a random process occurring over a continuous time interval T. The random process produces only s or f. Let α denote the average number of s produced per unit time. Further assume 1. probability of s during small time interval t is α t 2. probability of more than one s in t is negligible 3. probability of s in a later t is independent of what occurs earlier Then: Probability of x successes during time interval T is given by the Poisson distribution f x; λ where λ = αt

45 4.9 The Multinomial Distribution Sample space: sequences of trials of length n We assume: 1) Each trial has k possible distinct outcomes, type 1, type 2, type 3,., type k k 2) Outcome type i occurs with probability p i for each trail, where i=1 p i = 1 3) The outcomes for different trials are independent. (i.e. we assume multinomial Bernoulli trials. In the n trials, we want to know the probability f(x 1, x 2, x 3,, x k ) that there are where x 1 outcomes of type 1 x 2 outcomes of type 2 x k outcomes of type k k i=1 x i = n

46 For fixed values of x 1, x 2, x 3,, x k, there are n n x 1 n x 1 x 2 x 1 x 2 x n x 1 x 2 x k 1 3 x k n! = x 1! x 2! x 3! x k! outcomes that have these k values. (AMS 301 students will recognize this as P(n; x 1, x 2, x 3,, x k ), the number of ways to arrange n objects, when there are x 1 of type 1, x 2 of type 2,, and x k of type k ) x Each outcome has probability p 1 x 1 p 2 x 2 p 3 x 3 p k k. Summing the probabilities for theses outcomes we have n! f x 1, x 2, x 3,, x k = x 1! x 2! x 3! x k! p 1 x 1p x 2 2 p x 3 3 p x k k This is the multinomial probability distribution with the conditions that each x i 0 and that x i = n k i=1

47 e.g % of light bulbs will survive less that 40 hours of continuous use 2. 50% will survive from 40 to 80 hours of continuous use 3. 20% will survive longer than 80 hours of continuous use What is the probability that, among 8 light bulbs, 2 will be of type 1, 5 of type 2 and 1 of type 3? We want f 2,5,1 = 8! 2! 5! 1! (0.3)2 (0.5) 5 (0.2) 1 =

48 4.10 Generating discrete random variables that obey different probability distributions Observation: It is relatively simple to generate the random values 0, 1, 2,, 9 with equallikelihood (i.e. each with probability 1/10) draw the numbers (with replacement) from a hat flip a balanced, 10-sided dice It is also relatively straightforward to write a computer program that generates the integers 0, 1, 2,, 9 with equal-likelihood. Consequently, it is possible to generate all 2-digit numbers (outcomes) 00 to 99 with equal-likelihood (1/100) all 3-digit numbers (outcomes) 000 to 999 with equal-likelihood (1/1000) etc. outcomes

49 Consider the RV number of heads in 3 tosses of the dice The probability distribution for this RV is x f(x) 1/8= /8= /8= /8=0.125 F(x) F(2) F(1) F(0) x 1 = 0 x 2 = 1 x 3 = 2 x 4 = 3

50 i.e. all the outcomes are assigned the RV 0 all the outcomes are assigned the RV 1 all the outcomes are assigned the RV 2 all the outcomes are assigned the RV 3 Thus RV 0 occurs with probability 1/8 RV 1 occurs with probability 3/8 RV 2 occurs with probability 3/8 RV 3 occurs with probability 1/8 Thus the sequence of outcomes generated randomly (with equal-likelihood) 197, 365, 157, 520, 946, 951, 948, 568, 586, 089 are interpreted as the random values (number of heads) 1, 1, 1, 2, 3, 3, 3, 2, 2, 0 Table 7 in Appendix B presents a long list of the integers 0,, 9 generated with equallikelihood. One can use the table to randomly generate lists of 1-digit, 2-digit, 3-digit, etc. outcomes (by taking non-overlapping combinations and starting in different places)

51 e.g. RV = number cars arriving at a toll booth per minute x f(x) F(x) F(4) F(3) F(2) F(1) F(0)

52 Classical probability versus frequentist probability Recall: classical probability counts outcomes and assumes all outcomes occur with equal likelihood. Frequentist probability measures the frequency of occurrence of outcomes from past experiments. So what do two dice really do when thrown at the same time? Classic probability: distinct (i.e. different colored) dice: There are 36 distinct outcomes, each appears with equal likelihood, therefore the (unordered) outcome 1,2 has probability 2/36 identical dice: There are 21 distinct outcomes, each appears with equal likelihood, therefore the (unordered) outcome 1,2 has probability 1/21 Frequentist probability: distinct dice: The (unordered) outcome 1,2 has measured probability 2/36 in agreement with classic probability identical dice: The (unordered) outcome 1,2 has measured probability 2/36 (!!) in disagreement with classic probability For identical dice, the classic view of probability for throwing two identical dice assumes all 21 outcomes occur with equal probability. This is not what occurs in practice. in practice, each of the (unordered) outcomes i, j where i j occurs more frequently than the outcomes i, i.

53 Why is the frequentist approach correct. Clearly the frequency of getting unordered outcomes cannot depend on the color of dice being thrown (i.e. the color of the dice cannot affect frequency of occurrence). Thus two identical dice must generate outcomes with the same frequency as two differently-colored dice. Note: That is not to say that the classic probability view is completely wrong. The classic view correctly counts the number of different outcomes in each case ( identical and different dice). However it computes probability incorrectly for the identical case. The frequentist view concentrates on assigning probabilities to each outcome. In the frequentist view, the number of outcomes for two identical dice is still 21, but the probabilities assigned to i,i and i,j outcomes are different.