Math/Stat 360-1: Probability and Statistics, Washington State University Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 3 Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 1 / 31
Outline 1 Section 2.4: Conditional Probability 2 Section 2.5: Independence 3 Section 3.1: Random Variables 4 Section 3.2: Probability Distributions for Discrete Random Variables Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 2 / 31
Probabilistic Modeling Three Basic Ingredients: 1 Sample space Ω 2 Events E 3 Probability measure P(E) Motivation for Conditional Probability Measures It should be easier to estimate probabilities if more relevant information is given. The probability P(E) can be calculated by analyzing what could possibly happen under various possible scenarios. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 3 / 31
Cosindering three possible scenarios... Figure: If B 1 occurs, then A occurs. If B 3 occurs, then A will not occur. If B 2 occurs, likelihood of A depends on P(A B 2 ). Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 4 / 31
Definition Let A and B be two events with P(B) > 0. The conditional probability of A given that B occurs is defined as P(A B) := P(A B), A, B Ω. P(B) Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 5 / 31
Example In a city, 60% of all households get Internet service from the local cable company, 80% get television service from that company, and 50% get both services from that company. Let A = {getting Internet service}, B = {getting TV service}. 1 What is the probability that a randomly selected household gets Internet service given that it gets TV service from that company? P(A B) = P(A B) P(B) = 0.5 0.8 = 0.625. 2 What is the probability that a randomly selected household gets TV service given that it gets Internet service from that company? P(B A) = P(A B) P(A) = 0.5 0.6 = 0.833. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 6 / 31
The Multiplication Rule Theorem 1 For any two events A 1, A 2 Ω, P(A 1 A 2 ) = P(A 1 A 2 )P(A 2 ) = P(A 2 A 1 )P(A 1 ). 2 For any three events A 1, A 2, A 3 Ω, P(A 1 A }{{} 2 A 3 ) = P(A 3 A 1 A 2 )P(A }{{} 1 A 2 ) = }{{} B B B P(A 3 A 1 A 2 )P(A 2 A 1 )P(A 1 ). 3 This can be extended to multiple events. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 7 / 31
Example Four individuals have responded to a request by a blood bank for blood donations, and their blood types are unknown. Suppose only type O+ is desired and only one of the four actually has this type. If the potential donors are selected in random order for typing, what is the probability that at least three individuals must be typed to obtain the desired type? Let A = {first type is not O+}, B = {second type is not O+} P(at least three individuals are typed) = P(A B) = P(B A)P(A) = 2 3 3 4 = 0.5. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 8 / 31
Example (cont d) Four individuals have responded to a request by a blood bank for blood donations, and their blood types are unknown. Suppose only type O+ is desired and only one of the four actually has this type. What is the probability that type O+ is typed on the third donor? Let C = {third type is O+}. P(O+ is typed on the third donor) = P(C A B)P(B A)P(A) = 1 2 2 3 3 4 = 0.25. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 9 / 31
The Total Probability Law Theorem For any events A, B Ω, P(B) = P(A B) + P(A B) = P(B A)P(A) + P(B A )P(A ). Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 10 / 31
Remark Events {A 1, A 2,..., A k } constitute a partition of sample space Ω if they are mutually exclusive and k i=1 A i = Ω. For any event B, k k P(B) = P(A i B) = P(B A i )P(A i ). i=1 where P(B A i ), i = 1,..., k, are usually easier to calculate. i=1 Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 11 / 31
Example An individual has 3 different email accounts. 70% of her messages come into account #1, whereas 20% come into account #2 and the remaining 10% into account #3. Of the messages into account #1, only 1% are spam, whereas the corresponding percentages for accounts #2 and #3 are 2% and 5%, respectively. What is the probability that a randomly selected message is spam? Let A i = {message is from account # i}, i = 1, 2, 3, B = {message is spam}. It follows from the total probability law that P(B) = P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ) + P(B A 3 )P(A 3 ) = 0.01 0.7 + 0.02 0.2 + 0.05 0.1 = 0.016. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 12 / 31
Bayes Rule Theorem Let {A 1, A 2,..., A k } be a partition of sample space Ω. For any events B Ω, P(A j B) = P(A j B) P(B) = P(B A j )P(A j ) k i=1 P(B A, j = 1,..., k. i)p(a i ) Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 13 / 31
Example An individual has 3 different email accounts. 70% of her messages come into account #1, whereas 20% come into account #2 and the remaining 10% into account #3. Of the messages into account #1, only 1% are spam, whereas the corresponding percentages for accounts #2 and #3 are 2% and 5%, respectively. What is the probability that a randomly selected message is from account #1 given that it is spam? Let A i = {message is from account # i}, i = 1, 2, 3, B = {message is spam}. It follows from Bayes rule that = P(A 1 B) = P(B A 1 )P(A 1 ) P(B A 1 )P(A 1 ) + P(B A 2 )P(A 2 ) + P(B A 3 )P(A 3 ) 0.01 0.7 0.01 0.7 + 0.02 0.2 + 0.05 0.1 = 0.007 0.016 = 0.4375. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 14 / 31
Example (Incidence of a rare disease) Only 1 in 1000 adults is afflicted with a rare disease for which a diagnostic test has been developed. The test is such that when an individual actually has the disease, a positive result will occur 99% of the time, whereas an individual without the disease will show a positive test result only 2% of the time. If a randomly selected individual is tested and the result is positive, what is the probability that the individual has the disease? Let A 1 = individual has the disease, A 2 = individual does not have the disease, and B = positive test result. P(A 1 ) = 0.001, P(A 2 ) = 0.999, P(B A 1 ) = 0.99, P(B A 2 ) = 0.02. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 15 / 31
Example (cont d) P(A 1 B) = P(A 1 B) P(B) = 0.00099 0.00099 + 0.01998 = 0.047. Figure: Path probabilities Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 16 / 31
Learning via Bayes Rule Let H be an event of interest, and E be an event representing the evidence. Figure: P(H) is updated to P(H E). Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 17 / 31
Independence Two events A and B are independent if P(A B) = P(A). The Product Form: Two events A and B are independent is equivalent to that P(A B) = P(A)P(B). Independence and mutually exclusive are different. Two mutually exclusive events are in fact highly dependent. If A and B are independent, then P(A B) = P(A)+P(B) P(A B) = P(A)+P(B) P(A)P(B). Definition Events A 1, A 2,..., A n are mutually independent if for any subset {i 1,..., i k } {1,..., n}, P(A i1 A ik ) = P(A i1 ) P(A ik ). Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 18 / 31
Example Consider a system consisting of two components #1 and #2. Assume that components work independently of one another and P(component works) = 0.9. Components #1 and #2 are connected in series, so that system works iff both #1 and #2 work. Calculate P(system works). P(system works) = P(#1)P(#2) = 0.9 2 = 0.81. Components #1 and #2 are connected in parallel, so that system works iff either #1 or #2 works. Calculate P(system works). P(system works) = P(#1) + P(#2) P(#1)P(#2) = 0.9 + 0.9 0.9 2 = 0.99. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 19 / 31
Example (cont d) Consider a system consisting of two components #1 and #2. Assume that components work independently of one another and P(component works) = 0.9. Components #1 and #2 are connected in parallel. Given that the system fails, what is the probability that component #1 fails? P(component #1 fails system works) = = 0.9 0.99 = 0.909. P(#2 works) P(system works) Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 20 / 31
Example Consider the system of 4 components. Components 1 and 2 are connected in parallel; 3 and 4 are connected in series. If components work independently of one another and P(component works) = 0.9, calculate P(system works). P(1 or 2) = 0.99, P(3 and 4) = 0.81. P(system works) = 0.99 + 0.81 (0.99)(0.81) = 0.9981. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 21 / 31
Random Variables Random variable (RV): A function defined on the sample space. Example: Toss a coin three times. Let N = number of heads in three tosses. N(TTH) = 1, N(HHH) = 3, N(HTH) = 2, N(TTT ) = 0. Example: Sample a product from an assembly line. Let T = lifelength of the item. Discrete random variable: Its values are limited to discrete points (i.e., finite or countably infinite) on the real line. Continuous random variable: It takes on continuous measurements. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 22 / 31
Example Toss a fair coin three times. The sample space = {HHH, HHT, HTH, THH, TTH, THT, HTT, TTT }. Let N denote the number of heads in three tosses. P(N = 0) = 1/8, P(N = 1) = 3/8, P(N = 2) = 3/8, P(N = 3) = 1/8. Table: Probability Masses N = x 0 1 2 3 P(N = x) 1/8 3/8 3/8 1/8 Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 23 / 31
Discrete Random Variables The distribution of a discrete random variable X is described by the probability mass function (PMF) p(x i ) = P(X = x i ), for all the possible values x i of X. Distribution of RV X: Likelihoods or relative frequencies of various values of X. Properties of PMF 1 0 p(x) 1. 2 all x s p(x) = 1. Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 24 / 31
Example Consider a group of five potential blood donors, a, b, c, d, and e, of whom only a and b have type O+ blood. Five blood samples, one from each individual, will be typed in random order until an O+ individual is identified. Let RV Y = the number of typings necessary to identify an O+ individual. Then the PMF of Y is Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 25 / 31
Example (cont d) Figure: The line graph for the PMF Figure: The histogram for the PMF Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 26 / 31
Cumulative Distribution Function = Cumulative Frequency Cumulative Distribution Function (CDF) of X F(x) = P(X x) = y:y x 1 F(x) is step-wise, non-decreasing. 2 0 F(x) 1. 3 F(x) 1 as x +. p(y). Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 27 / 31
PMF vs CDF P(a X b) = y:a y b P X(y) = F(b) F(a ). Figure: P X (x) = PMF, F X (x) = CDF Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 28 / 31
Example (Five Blood Samples) Let RV Y = the number of typings necessary to identify an O+ individual. The PMF of Y is given by The CDF of Y is given by F(x) = 0 if x < 1 0.4 if 1 x < 2 0.7 if 2 x < 3 0.9 if 3 x < 4 1 if 4 x Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 29 / 31
Example (Geometric Distribution) Consider testing items coming off an assembly line one by one until a defective item (labeled F ) is found. Let X be the number of testing items necessary to find the first defective item. If P(F) = p, find the PMF and CDF of X. Let S denote a non-defective item, and so P(S) = 1 p. The PMF of X: p(k) = P(X = k) = (1 p) k 1 p, k 1. For the CDF, for any positive integer x 1, F(x) = P(X x) = k x p(k) = x (1 p) k 1 p k=1 = 1 (1 p)x p = 1 (1 p) x. p Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 30 / 31
Example (cont d): Geometric CDFs Haijun Li Math/Stat 360-1: Probability and Statistics, Washington State University Week 3 31 / 31