Summary of some Rules of Probability with Examples

Transcription

1 Summary of some Rules of Probability with Examples CEE 201L. Uncertainty, Design, and Optimization Department of Civil and Environmental Engineering Duke University Henri P. Gavin Spring, 2016 Introduction Engineering analysis involves operations on input data (e.g., elastic modulus, wind speed, static loads, temperature, mean rainfall rate) to compute output data (e.g., truss bar forces, evaporation rates, stream flows, reservoir levels) in order to assess the safety, serviceability, efficiency, and suitability of the system being analyzed. There can be considerable uncertainty regarding the input data, but this uncertainty often can be characterized by a range of values (minimum and maximum expected values), an average expected value with some level of variability, or a distribution of values. For example, Figures 1 shows probability distributions of daily precipitation and daily max and min temperatures for Durham NC (27705) from June 1990 to June Clearly, any calculated prediction of the level of Lake Michie involves temperature and rainfall rate. Long-term predictions of lake levels should be reported in terms of expected values and uncertainty or variability, reflecting uncertainties in the input data. These calculations involve the propagation of uncertainty. In this course you will propagate uncertainty to determine the probability of failure of the systems you design. This kind of failure analysis requires: 1. the identification of all possible modes of failure; 2. the evaluation of failure probabilities for each failure mode; and 3. combining these failure probabilities to determine an overall failure probability This, in turn, requires methods based on the theory of sets (e.g., the union and intersection of sets and their complements) and the theory of probability (e.g., the probability that an event belongs to a particular set among all possible sets). In the following sections probability concepts are illustrated using Venn diagrams. In these diagrams the black rectangle represents the space of all possible events with an associated probability of 1. The smaller rectangles represent subsets and their intersections. It is helpful to associate the areas of these rectangles with the probability of occurence of the associated events. So think of the black rectangle as having an area of 1 and the smaller rectangles having areas less than 1.

2 2 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin P.D.F C.D.F max and min temp, deg C daily precipitation, mm Figure 1. Probability Density Functions and Cumulative Distribution Functions of precipitation, minimum temperature, and maximum temperature in Durham NC from June 1990 to June 2013.

3 Rules of Probability 3 Complementary Events A A If the probability of event A occurring is P [A] then the probability of event A not occurring, P [A ], is given by P [A ] = 1 P [A]. (1) Example: This and following examples pertain to traffic and accidents on a certain stretch of highway from 8am to 9am on work-days. If a year has 251 work-days and 226 work-days with no accident (on the stretch of highway between 8am and 9am) the probability of a work-day with no accident is 226/251=0.90. The probability of a work-day with an accident (one or more) is =0.10. Mutually Exclusive (Non-Intersecting) Events (ME) A B Mutually exclusive events can not occur together; one precludes the other. If A and B are mutually exclusive events, then P [A and B] = 0. (2) All complementary events are mutually exclusive, but not vice versa. Example: If X represents the random flow of traffic in cars/minute, the following sets of events are mutually exclusive: 0 < X 5, 5 < X 10, and 20 < X 50. The pair of events X = 0 and 20 < X 50 are mutually exclusive but not complementary. Collectively Exhaustive Events (CE) A B If A and B are collectively exhaustive events then the set [A and B] represents all possible events, and P [A or B] = 1. (3) A pair of complementary events are ME and CE. The sum of the probabilities of a set of ME and CE events must be Example: Let Y represent the random number of accidents from 8am to 9am on a work-day. If P [Y = 0] = 0.90 (no accident) and P [Y = 1] = 0.04 then P [Y > 1] = Example: The following sets are mutually exclusive and collectively exhaustive: X = 0, 0 < X 5, 5 < X 10, 10 < X 20, 20 < X 50, and 50 < X. In the above if the < were replaced by the sets would be CE but not ME.

4 4 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin Conditionally Dependent Events A B A B If the occurrence (or non-occurrence) of an event B affects the probability of event A, then A and B are conditionally dependent; P [A B = P [A and B] / P [B] (4) In this equation the set B can be thought of as a more restrictive set of all possibilities. The set B is therefore the new set of all possibilities and the probability of A given B is represented as the area associated with the intersection of A and B (the green rectangle) divided by the area associated with the set B (the red rectangle). Example: The term P [Y > 1 10 < X 20] denotes the probability of more than one accident from 8am to 9am given the traffic flow is between 11 and 20 cars per minute (inclusive). If A and B are mutually exclusive, the occurrence of one implies the non-occurrence of the other, and therefore a dependence between A and B. So P [A B] = P [B A] = 0. If A and B are mutually exclusive, A is a subset of B, and P [A B ] = 1. So all mutually exclusive events are conditionally dependent, but not vice versa. Example: The events [Y > 1] and [10 < X 20] are dependent but not mutually exclusive. Statistically Independent Events A B If the occurrence (or non-occurrence) of event B has no bearing on the probability of A occurring, then A and B are statistically independent; P [A B] = P [A] and P [B A] = P [B]. (5) Example: If Z represents the random number of migrating geese above above the certain stretch of highway from 9am-10am on a work-day, P [Y > 1 Z > 20] = P [Y > 1]. Intersecting sets of events can be dependent or independent. Example: The events [Y > 1] and [Z > 20] can occur on the same day (are not mutually exclusive; are intersecting) but are independent. A pair of events cannot be both mutually exclusive and independent. Example: The events [Z 10] and [Z > 20] are mutually exclusive. But the occurrence of one implies the non-occurrence of the other, and therefore a dependence.

5 Rules of Probability 5 Intersection of Events A B If A and B are conditionally dependent events, then (from equation (4)) the intersection: P [A and B] = P [A B] = P [AB] = P [A B] P [B] (6) Example: If P [Y > 1 10 < X 20] = 0.05 and P [10 < X 20] = 0.25 then P [Y > 1 10 < X 20] = (0.05)(0.25) = If A and B are independent events: P [A B] = P [A]; and P [A and B] = P [A B] = P [AB] = P [A] P [B] (7) Example: P [Y > 1 Z 20] = P [Y > 1]. If P [Z > 20] = 0.04 then P [Y > 1 Z 20] = (0.06)(0.96) = If A, B, and C, are dependent events: P [A B C] = P [ABC] = P [A BC] P [BC] = P [A BC] P [B C] P [C]. (8) If A, B, and C are independent events, P [A BC] = P [A], P [B C] = P [B], and P [A B C] = P [ABC] = P [A] P [B] P [C] (9) Union of Events A B A B For any two events A and B, P [A or B] = P [A B] = P [A] + P [B] P [A and B]. (10) Example: P [Y > 1 10 < X 20] = = If A and B are mutually exclusive, P [A and B] = 0 and P [A or B] = P [A B] = P [A] + P [B]. (11) Example: P [Y = 0 Y > 1] = = the same as (1 P [Y = 1]). For any n events E 1, E 2,, E n, P [E 1 E 2 E n ] = 1 P [E 1 E 2 E n] (12) If E 1, E 2,, E n are n mutually exclusive events, P [E 1 E 2 E n ] = P [E 1 ] + P [E 2 ] + + P [E n ] (13)

6 6 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin E 1 E 2 E 3 E 4 Theorem of Total Probability If E 1, E 2,, E n are n mutually exclusive (ME) and collectively exhaustive (CE) events, and if A is an event that shares the same space as the events E i, (P [A E i ] > 0 for at least some events E i ) then via the intersection of dependent events and the union of mutually exclusive events: P [A] = P [A E 1 ] + P [A E 2 ] + + P [A E n ] P [A] = P [A E 1 ] P [E 1 ] + P [A E 2 ] P [E 2 ] + + P [A E n ] P [E n ] (14) Example: The table below shows the probabilities of a number of events. P [X = 0] P [0 < X 10] P [10 < X 20] P [20 < X 50] P [50 < X] P [Y > 1 X = 0] P [Y > 1 0 < X 10] P [Y > 1 10 < X 20] P [Y > 1 20 < X 50] P [Y > 1 50 < X] So... P [Y > 1] = (0)(0)+(0.0)(0.20)+(0.05)(0.25)+(0.07)(0.35)+(0.115)(0.20) = 0.06 A Bayes Theorem A B For two dependent events A and B, P [A B] is the fraction of the area of B that is in A and P [A B]P [B] is the area of B in A. Likewise, P [B A] is the fraction of the area of A that is in B and P [B A]P [A] is the area of A in B. Clearly, The area of A in B equals the area of B in A. P [A and B] = P [A B] P [B] = P [B A] P [A]. So, P [A B] = P [B A] P [A]. (15) P [B] If event A depends on n ME and CE events, E 1,..., E n P [E i A] = P [A E i ] P [E i ] P [A E 1 ] P [E 1 ] + P [A E 2 ] P [E 2 ] + + P [A E n ] P [E n ]. (16) Example: Using the probabilities from the table above, given an observation of more than one accident, find the probability that the traffic was very heavy (50 < X). P [50 < X Y > 1] = P [Y > 1 50 < X]P [50 < X]/P [Y > 1] = (0.115)(0.20)/(0.06) = Note that if A is a subset of B, then P [B A] = 1 and P [A B] = P [A]/P [B]. Note also that P [A B] P [B A] if and only if P [A] P [B].

7 Rules of Probability 7 Bernoulli sequence of independent events with probability p A Bernoulli sequence is a sequence of trials for which an event may or may not occur. In a Bernoulli sequence the occurrence of an event in one trial is independent of an occurrence in any other trial. If we have a sequence of trials, we may wish to know the probability of multiple occurrences in a given number of trials. This will depend upon the probability of the event in any single trial and the number of ways multiple events can occur in the given number of trials. Example: Given that the probability of an accident on any given work-day is 0.10, and assuming the probability of an accident today in no way depends on the occurrence of an accident on any other day, what is the probability of two days with an accident (and three days with zero accidents) in one work-week? Given: P [one or more accident on any work-day] = p = 0.10 From complementary events: P [no accident on any work-day] = 1 p = = 0.90 From the intersection of independent events: P [an accident on any two work-days] = (p)(p) = (0.10) 2 = From the intersection of independent events: P [no accident on any three work-days] =.... (1 p)(1 p)(1 p) = (1 0.10) 3 = From the intersection of independent events: P [an accident on any two work-days no accident on any three work-days]. = (p) 2 (1 p) 5 2 = (0.010)(0.729) From the union of independent events: number of ways to pick two days out of five = = 5!/(2! 3!) = 120/((2)(6)) = 10 P [two days out of five with an accident] = (10)(.010)(.729) = This is called the binomial distribution. P [n events out of m attempts] = As a special case, P [0 events out of m attempts] = (1 p) m The expected number of events out of m attempts is pm. m! n!(m n)! pn (1 p) m n (17) The more data we have in determining the probability of an event, the more accurate our calculations will be. If n events are observed in m attempts, p n/m, and p becomes more precise as m (and n) get bigger.

8 8 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin Poisson process with mean occurrence rate ν (mean return period T = 1/ν). For a Bernoulli sequence with a small event probability p and a large number of trials, the binomial distribution approaches the Poisson distribution, in which the event probability p is replaced by a mean occurrence rate, ν, or a return period, T = 1/ν. (t/t )n P [n events during time t] = exp( t/t ) (18) n! Special cases: P [time between two events > t] = P [0 events during time t] = e t/t P [time between two events t] = 1 P [0 events during time t] = 1 e t/t Example: Given that the probability of an accident on any work-day is 0.10, the return period for work-days with accidents is 1/0.10 work-days or 10 work-days. On average, drivers would expect a day with one or more accidents every other work-week. Assuming such accidents are a Poisson process, P [two work-days with accidents in five days] =.... (5/10) 2 /2! exp( 5/10) = only slightly more than what we got assuming a Bernoulli sequence. P [time between work-days with accidents > T work-days] =.... P [0 accidents in T work days] = e T/T = e 1 = P [one or more accidents in T work-days] = 1 e T/T = 1 e 1 = The Normal distribution The figures below plot P [n days with accidents in m workdays] vs. n, the number days with accidents, for m = 5, 20, and 50 work-days. As m and t become large the Binomial distribution and the Poisson distribution approach the Normal distribution with a mean value of pm or t/t and a standard deviation of pm or t/t Binomial Poisson Normal Binomial Poisson Normal 0.2 Binomial Poisson Normal Probability Probability Probability number of days with multi-car accidents in one work-week number of days with multi-car accidents in four work-weeks number of days with multi-car accidents in ten work-weeks

9 Rules of Probability 9 The Exponential distribution Consider the random time T 1 between two random events in a Poisson process. The complementary event is P [T 1 > t] = P [no events in time t] = exp( t/t ) P [T 1 t] = 1 P [no events in time t] = 1 exp( t/t ) This is the distribution function for an exponential distribution, and describes the inter-event time, or the time between successive events. Examples 1. What is the probability of flipping a head in three coin tosses? (a) P [ a H on any toss ] = P [ a H in 3 tosses ] = = 1.5 > 1??? Clearly wrong. Probabilities must be between 0 and 1! A H on the first toss is not mutually exclusive of a H on any other toss. (b) Instead, using the union of multiple mutually exclusive events... There are seven mutually exclusive ways to get a H in three tosses: (H 1 T 2 T 3 ); (T 1 H 2 T 3 ); (T 1 T 2 H 3 ); (H 1 H 2 T 3 ); (H 1 T 2 H 3 ); (H 1 T 2 H 3 ); (H 1 H 2 H 3 ); The probability of each is the same (0.5) 3, so P [ a H in 3 tosses ] = 7 (0.5) 3 = (c) In fact, there is only one way not to get a head in three tosses: (T 1 T 2 T 3 ). Making use of complementary probabilities and the intersection of independent events (tossing three T ): P [ a H in 3 tosses ] = 1 P [ 0 H in 3 tosses ] = 1 (0.5) 3 = (d) Invoking the Bernouli distribution, we can say there are three mutually exclusive groups of making a H in three tosses: P [ a H in 3 tosses ] = P [ 1 H in 3 tosses ]+P [ 2 H in 3 tosses ]+P [ 3 H in 3 tosses ] = = Make a Venn diagram of three intersecting circles to help illustrate these four solutions. 2. Consider a typical year having 36 days with rain. The probability of rain on any given day is The return period for days with rain is 10 days. Use the Binomial and Poisson distributions to find: Bernoulli Poisson P [one rainy day in 31 days] (31!)/(1!30!) (1 0.1) 30 = ((31/10) 1 /1!) exp( 31/10) = P [one rainy day in 60 days] (60!)/(1!59!) (1 0.1) 59 = ((60/10) 1 /1!) exp( 60/10) = What is the probability of exactly one event in one day for a Poisson process with return period of T days? P [one occurrence in one day] = ((1/T ) 1 /1!) exp[ 1/T ] = (1/T ) exp( 1/T ) For T 1 day, P [one occurrence in one day] (1/T ). 4. What is the probability of exactly one event in T days for a Poisson process with return period of T days? P [one occurrence in T days] = ((T/T ) 1 /1!) exp[ T/T ] = exp( 1) = 0.368

10 10 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin 5. What is the probability of one or more events in T days for a Poisson process with return period of T days? P [one or more occurrences in T days] = 1 P [no occurrences in T days] = 1 ((T/T ) 0 /0!) exp[ T/T ] = 1 exp( 1) = What is the return period T of a Poisson process with a 2% probability of exceedance in fifty years? 0.02 = P [n > 0] = 1 P [n = 0] ln(1 0.02) = 50/T = 1 ((50/T ) 0 /0!) exp( 50/T ) T = years 7. In the past 50 years there have been two large earthquakes (M > 6) in the LA area. What is the probability of a large earthquake in the next 15 years, assuming the occurrence of earthquakes is a Poisson process? The mean return period is T = 50/2 = 25 years. P [one or more EQ in 15 years] = 1 P [no EQ in 15 years] = = 1 ((15/25) 0 /0!) exp( 15/25) = (In actuality, the occurrence of an earthquake increases the probability of earthquakes, for a year or two.) 8. Consider the following events (E: electrical power failure); (F : flood) with probabilities: P [E] = 0.2; P [F ] = 0.1; P [E F ] = Note that P [E] > P [E F ]. So, P [E F ] = P [E F ] P [F ] = (0.1)(0.1) = 0.01 If E and F were assumed to be independent, then the probability of E and F would be calculated as P [E F ] = P [E] P [F ] = (0.2)(0.1) = 0.02 Neglecting the conditional dependence of random events can lead to large errors.

11 Rules of Probability Assume that during rush-hour one out of ten-thousand drivers is impaired by alcohol. Also assume that breathalyzers have a 2% false-positive rate and a 0% false-negative rate. If a police officer stops a car at random and finds that the driver tests positively for alcohol, what is the likelihood that the officer has correctly identified an impaired driver? assign random variables to events: D =drunk, D =sober, B =breathalyzer positive, B =breathalyzer negative. Given: P [D] = , P [B D ] = 0.02, P [B D] = 0, Find: P [D B]. D and D are complementary, P [D ] = 1 P [D] = (B D) and (B D) are complementary, P (B D) = 1 P (B D) = 1 Bayes Theorem with the Theorem of Total Probability: P [D B] = P [B D] P [D] P [B] P [B D] P [D] = P [B D] P [D] + P [B D ] P [D ] (1)(0.0001) = (1)(0.0001) + (0.02)(0.9999) = (1)(0.0001) = percent This surprisingly-low probability is an example of the base rate fallacy of the false positive paradox which arises when the event being tested ( the driver is drunk in this example) occurs with very low probability. Seen another way, the test changes the probability of finding a drunk driver from one in ten-thousand to one in two-hundred. 10. Ang+Tang, v.1, example 2.27, p. 57. Consider emission standards for automobiles and industry in terms of three events: I: industry meets standards; A: automobiles meet standards; and R: there has been an acceptable reduction in pollution. Given: P [I] = 0.75; P [A] = 0.60; A and I are independent events; P [R AI] = 1; P [R AI ] = 0.80; P [R A I] = 0.80; and P [R A I ] = 0. Find: P [R]. The following set of events are CE and ME: AI, AI, A I, A I. So, from the Theorem of Total Probability, P [R] = P [R AI] P [AI] + P [R AI ] P [AI ] + P [R A I] P [A I] + P [R A I ] P [A I ]

12 12 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin Since A and I are independent, P [AI] = (0.60)(0.75) = 0.45; P [AI ] = (0.60)(0.25) = 0.15; P [A I] = (0.40)(0.75) = 0.30; P [A I ] = (0.40)(0.25) = So, P [R] = (1.0)(0.45) + (0.80)(0.15) + (0.80)(0.30) + (0)(0.1) = 0.81 If it turns out that pollution is not reduced (R ), what is the probability that it is entirely due to a failure to control automobile exhaust (A I)? P [A I R ] = P [R A I] P [A I] P [R ] = (1 0.8)(0.3) (1 0.81) = 0.32 If it turns out that pollution is not reduced (R ), what is the probability that automobile exhaust was not controlled (A )? The events I and I are complementary and the events A I and A I are mutually exclusive. P [A R ] = P [A I R ] + P [A I R ] = P [R A I] P [A I] + P [R A I ] P [A I ] P [R ] P [R ] (1 0.8)(0.3) (1 0)(0.10) = + = 0.84 (1 0.81) (1 0.81) If it turns out that pollution is not reduced (R ), what is the probability that industry emissions were not controlled (I )? The events A and A are complementary and the events AI and A I are mutually exclusive. P [I R ] = P [AI R ] + P [A I R ] = P [R AI ] P [AI ] + P [R A I ] P [A I ] P [R ] P [R ] (1 0.8)(0.15) (1 0)(0.10) = + = 0.68 (1 0.81) (1 0.81) So, if pollution is not reduced it is more likely that it has to do with cars not meeting standards than industry.

13 Rules of Probability Ang+Tang, v.1, example 2.25, p. 55 Consider damaging storms causing flooding and storm-damage in a county and within a city. Suppose that the probability of a storm in a county is 20% in any given year. During such a storm: the probability of a flood in the county is 25%; the probability of flood in the city during a flood in the county is 5%; and if there is no flood in the county the probability of storm damage in the city is 10%. Assume floods only happen during storms and if a flood happens in a city, property will be damaged with absolute certainty. If there is a flood in the county without a flood in the city, the probability of storm damage in the city is 15%. There can be no flood in the city without a flood in the county. What is the annual probability of storm damage in the city? In terms of events: S: storm in the county F : flood in the county (only happens if there is a storm in the county) W : flood in the city (only happens if there is a flood in the county) D: storm damage in the city the probabilities above (and their complements) can be summarized in the table below. storm flood in county flood in city damage in city P [S] = 0.20 P [F S] = 0.25 P [W F ] = 0.05 P [D W ] = 1 P [W F ] = 0.95 P [D SF W ] = 0.15 P [F S] = 0.75 P [W F ] = 0 P [D SF ] = 0.10 P [W F ] = 1 P [S ] = 0.80 P [F S ] = 0 P [D S ] = 0 P [F S ] = 1 P [W F ] = 0 The events (SF ), (SF ), and (S ) are CE and ME. So, from the Theorem of Total Probability, P [D] = P [D SF ] P [SF ] + P [D SF ] P [SF ] + P [D S ]P [S ] The conditional probabilities needed to solve this problem can be found from further application of the Theorem of Total Probability and the intersection of dependent events.

14 14 CEE 201L. Uncertainty, Design, and Optimization Duke University Spring 2016 H.P. Gavin If there is a storm, a flood in the county, and a flood in the city, damage in the city will certainly occur (P [D SF W ] = 1), and might occur even if there is not a flood in the city (P [D SF W ] = 0.15). P [D SF ] = P [D SF W ] P [W SF ] + P [D SF W ] P [W SF ] = (1.0)(0.05) + (0.15)(0.95) = P [SF ] = P [F S] P [S] = (0.25)(0.20) = 0.05 (intersection of dependent events)) P [D SF ] = 0.10 (given) P [SF ] = P [F S] P [S] = (0.75)(0.20) = 0.15 (intersection of dependent events) P [D S ] = 0 (given) So, P [D] = (0.1925)(0.05) + (0.10)(0.15) + (0)(0.8) = Note that while there is twice the likelihood of damage when there is a flood in the county as compared to no flood in the county (P [D SF ] 0.19 vs P [D SF ] = 0.1), the probability of damage and a flood in the county is less than the probability of damage and no flood in the county. P [D SF ] = P [D SF ] P [SF ] = (0.1925)(0.05) P [D SF ] = P [D SF ] P [SF ] = (0.10)(0.15) = W D P[SF]=0.05 SF P[SF ]=0.15 S P[S ]=0.8