Basi Properties of Probability Definitions: A random experiment is a proedure or an operation whose outome is unertain and annot be predited with ertainty in advane. The olletion of all possible outomes is alled the sample spae. We will typially use the letter S to denote a sample spae. An event is any subset of the sample spae. Events are usually denoted by apital letters other than S. Events are olletions (or sets) of outomes from the sample spae. Examples of random experiments: 1. Toss a oin three times and reord the results of eah toss in order. List all events in the sample spae. Two examples of events from this random experiment are A = the event that there are exatly 2 heads in the 3 tosses B = the event that there are no heads List the outomes ontained in eah of these events. 2. Toss a oin repeatedly and ount the number of tosses until the first heads. What is the sample spae? 3. Measure the lifetime of a light bulb in hours. What is the sample spae? An example of an event in this sample spae would be the event A that a light bulb lasts at least 500 hours. Write this event using set notation. The outomes in a sample spae are said to be equally likely if they will all our approximately equally often in the long run if the random experiment is repeated many, many times. In whih of the examples above does the sample spae onsist of equally likely outomes? 1
Notation of Set Theory First let us introdue some definitions and notation: Definitions: Let S be a sample spae and let A and B be any two events from S. Then The union of the events A and B, denoted A B, is the event onsisting of all outomes that belong to A or B or both. The intersetion of the events A and B, denoted A B or sometimes by the shorter AB, is the event onsisting of all outomes ommon to both A and B. The omplement of an event A, denoted A, is the olletion of all outomes that are not in A. The event A is a subset of B, denoted A B, if every outome in A is also ontained in B. The empty set or null set, denoted Ø, is the event whih onsists of no outomes. The events A and B are disjoint or mutually exlusive if A and B annot happen simultaneously. Thus A and B are disjoint if A B. Pr(A) or P(A) is used to denote the probability of event A. Example: Shuffle a standard dek of 52 ards and randomly selet one ard from the dek. Then the sample spae S onsists of eah of the 52 ards in the dek. Some possible events to onsider: A = the ard is a heart B = the ard is a fae ard C = the ard is the king of hearts D = the ard is blak Desribe the following related events: A = A B = A B = The union of A and B is an event onsisting of 22 outomes (all 13 of the hearts plus the king, queen, and jak from eah of the others suits). Note that C A and C B. List two pairs of events from above that are mutually exlusive. Probability as a Long-term Relative Frequeny The probability of a random event is the long run proportion (or relative frequeny) of times the event would our if the random proess were repeated over and over an extremely large number of times under idential onditions. The probability of an event an be approximated by simulating the proess a large number of times. Simulation leads to an empirial, or experimental, estimate of the probability. 2
Treating probabilities as long-term frequenies is known as the frequentist approah to probability. Using the frequentist approah, if a sample spae onsists of a finite number of possible outomes, say N, and all outomes are equally likely, then it is natural to assign equal probabilities to eah outome. That is, 1 1 P eah outome. total number of outomes N Furthermore, in this situation in whih all outomes are equally likely, if an event A onsists of M distint outomes, then the probability of the event A is given by P A number of outomes in A M. number of outomes in S N In the example of dealing ards on the previous page, sine we are shuffling the dek and then randomly seleting one ard, all 52 ards are equally likely. As a result, we an ompute the probability of any event in the sample spae S simply by ounting the number of outomes in the event and dividing by 52, the total number of outomes in the sample spae. For example, 3 P(the ard is a heart and a fae ard) = PAB. 52 Find the probability of eah of the other events desribed on the previous page. Note: Sine events are sets, it makes sense to perform set operations suh as omplement, intersetion, and union on them, but it makes no sense to perform arithmeti operations suh as addition or multipliation on events. On the other hand, probabilities are numbers, so it is legitimate to add, multiply, and divide probabilities but not to take omplements, intersetions, or unions of them. 3
Case Study: 100 Best Films In 1998, the Amerian Film Institute reated a list of the top 100 Amerian films ever made. The list is inluded below. Rank Title Year 1 Citizen Kane 1941 2 Casablana 1942 3 The Godfather 1972 4 Gone With The Wind 1939 5 Lawrene Of Arabia 1962 6 The Wizard Of Oz 1939 7 The Graduate 1967 8 On The Waterfront 1954 9 Shindler's List 1993 10 Singin' In The Rain 1952 11 It's A Wonderful Life 1946 12 Sunset Boulevard 1950 13 The Bridge On The River Kwai 1957 14 Some Like It Hot 1959 15 Star Wars 1977 16 All About Eve 1950 17 The Afrian Queen 1951 18 Psyho 1960 19 Chinatown 1974 20 One Flew Over The Cukoo's Nest 1975 21 The Grapes Of Wrath 1940 22 2001: A Spae Odyssey 1968 23 The Maltese Falon 1941 24 Raging Bull 1980 25 E.T The Extra-Terrestrial 1982 26 Dr. Strangelove 1964 27 Bonnie And Clyde 1967 28 Apoalypse Now 1979 29 Mr. Smith Goes To Washington 1939 30 The Treasure Of The Sierra Madre 1948 31 Annie Hall 1977 32 The Godfather Part Ii 1974 33 High Noon 1952 34 To Kill A Mokingbird 1962 35 It Happened One Night 1934 36 Midnight Cowboy 1969 37 The Best Years Of Our Lives 1946 38 Double Indemnity 1944 39 Dotor Zhivago 1965 40 North By Northwest 1959 41 West Side Story 1961 42 Rear Window 1954 43 King Kong 1933 44 The Birth Of A Nation 1915 45 A Streetar Named Desire 1951 46 A Clokwork Orange 1971 47 Taxi Driver 1976 48 Jaws 1975 49 Snow White And The Seven Dwarfs 1937 50 Buth Cassidy And The Sundane Kid 1969 Rank Title Year 51 The Philadelphia Story 1940 52 From Here To Eternity 1953 53 Amadeus 1984 54 All Quiet On The Western Front 1930 55 The Sound Of Musi 1965 56 M*A*S*H 1970 57 The Third Man 1949 58 Fantasia 1940 59 Rebel Without A Cause 1955 60 Raiders Of The Lost Ark 1981 61 Vertigo 1958 62 Tootsie 1982 63 Stageoah 1939 64 Close Enounters Of The Third Kind 1977 65 The Silene Of The Lambs 1991 66 Network 1976 67 The Manhurian Candidate 1962 68 An Amerian In Paris 1951 69 Shane 1953 70 The Frenh Connetion 1971 71 Forrest Gump 1994 72 Ben-Hur 1959 73 Wuthering Heights 1939 74 The Gold Rush 1925 75 Danes With Wolves 1990 76 City Lights 1931 77 Amerian Graffiti 1973 78 Roky 1976 79 The Deer Hunter 1978 80 The Wild Bunh 1969 81 Modern Times 1936 82 Giant 1956 83 Platoon 1986 84 Fargo 1996 85 Duk Soup 1933 86 Mutiny On The Bounty 1935 87 Frankenstein 1931 88 Easy Rider 1969 89 Patton 1970 90 The Jazz Singer 1927 91 My Fair Lady 1964 92 A Plae In The Sun 1951 93 The Apartment 1960 94 Goodfellas 1990 95 Pulp Fition 1994 96 The Searhers 1956 97 Bringing Up Baby 1938 98 Unforgiven 1992 99 Guess Who's Coming To Dinner 1967 100 Yankee Doodle Dandy 1942 Suppose that two people (we ll all them Allan and Beth) get together to wath a movie and, to avoid potentially endless debates about a seletion, deide to hoose a movie at random from the top 100 list. You will investigate the probability that it has already been seen by at least one of them. 4
Explorations Basi Probability Rules Suppose that one film is seleted at random from the list. Let A denote the event that Allan has seen the film and let B denote the event that Beth has seen the film. Note that the events A and B an be thought of as sets; for example, A is the set of all films that Allan has seen. Sine the movie is being seleted at random, eah of the 100 films is equally likely to be hosen; that is, eah has probability 1/100. The probabilities of various events an thus be alulated by ounting how many of the 100 films omprise the event of interest. For example, the following 2x2 table lassifies eah movie aording to whether it was seen by Allan and whether it was seen by Beth: Beth yes Beth no Total Allan yes 36 9 Allan no 16 39 Total 100 a) Translate the following events into set notation using the symbols (A and B, omplement, union, intersetion) defined on the previous page. Then give the probability of the event as determined from the table: Allan and Beth have both seen the film. Allan has seen the film and Beth has not. Beth has seen the film and Allan has not. Neither Allan nor Beth has seen the film. b) Fill in the marginal totals of the table. From these totals determine the probability that Allan has seen the film and also the probability that Beth has seen the film. (Remember that the film is hosen at random, so all 100 are equally likely.) Reord these probabilities along with the appropriate symbols below. ) Determine the probability that Allan has not seen the film. Do the same for Beth. Reord these, along with the appropriate symbols, below. 5
d) If you had not been given the table, but instead had merely been told that P A.45 and P B.52, would you have been able to alulate P A and PB? Explain how. One of the most basi probability rules is the omplement rule, whih asserts that the probability of the omplement of an event equals one minus the probability of the event: A A P 1 P. e) Add the ounts in the appropriate ells of the table to alulate the probability that either Allan or Beth (or both) have seen the movie. Also indiate the symbols used to represent this event. f) If you had not been given the table but instead had merely been told that P A.45 and P B.52, would you have been able to alulate P A B? Explain. g) One might naively think that P A B P A P B it is larger or smaller than PA B. Calulate this sum, and indiate whether and by how muh. Explain why this makes sense, and indiate how to adjust the right side of this expression to make the equality valid. 6
The addition rule asserts that the probability of the union of two events an be alulated by adding the individual event probabilities and then subtrating the probability of their intersetion: P AB P A P B P A B. This rule should make good intuitive sense. If we simply add the probabilities of events A and B, we are ounting all outomes that are in both A and B twie, so we need to subtrat off the probability of A B in order to eliminate this double ounting. h) Use this addition rule as a seond way to alulate the probability that Allan or Beth has seen the movie, verifying your answer to e). i) As a third way to alulate this probability, first identify (in words and in symbols) the omplement of the event that Allan or Beth has seen the movie. Then find the probability of this omplement P A B. Does this math your from the table. Then use the omplement rule to determine answers to e) and h)? j) Under what irumstanes is it valid to say that P A B P A P B? If AB, then it follows that PAB P A P B. This is known as the addition rule for disjoint events; it is a speial ase of the addition rule sine if A B, P A B P 0. 7
Conditional Probability We are often interested in the onditional probability of one event given the information that another event has ourred. You will find that it is straightforward and intuitive to derive a reasonable definition of onditional probability by examining data in a two-way table. Suppose again that one of the top 100 films is to be seleted at random, and onsider again the 2x2 table indiating how many of the top 100 films Allan and Beth have seen: Beth yes Beth no Total Allan yes 36 9 45 Allan no 16 39 55 Total 52 48 100 k) Reall the (unonditional) probability that Allan has seen the movie and the (unonditional) probability that Beth has seen it. Also reall the probability that they both have seen the film. Reord these, along with the appropriate symbols, below: l) Now suppose that one the film has been seleted, you learn the partial information that Allan has seen it. Given this information, determine the onditional probability that Beth has seen it by restriting your attention to the Allan yes row of the table and assuming that those films are equally likely. We use the notation PB A to denote the onditional probability of event B given event A. m) Determine how PB A relates to P A, P B, and P A B. [Hints: Atually, one of these three probabilities is irrelevant. Determine whih two are relevant and how they relate to PB A by following your alulation from the table in (l). 8
Definition: The onditional probability of event B given event A is defined as follows: P A B PBA, P A provided that P A 0. Note that when defining a onditional probability, it is essential to require that would not make sense to ondition on an event that is impossible in the first plae. P A is positive sine it n) Use the definition to alulate the onditional probability that Allan has seen the movie given that Beth has seen it. Does the knowledge that Beth has seen it inrease, derease, or not affet the (unonditional) probability that Allan has seen it? Explain. o) Does PB A PA B in this ase? To onvine yourself that these need not even be lose, onsider seleting one Amerian itizen at random. Let M be the event that the person is male, and let S be the event that the person is a U.S. Senator. Make an eduated guess as to the values of P SM. Are they lose? PM S and p) Use the definition of onditional probability to alulate PB A and P AB for the film example. Does the knowledge that Beth has not seen the film inrease, derease, or not affet the probability that Allan has seen it? Explain. Independene Two events are said to be independent if knowledge that one ours does not hange the probability of other s ourrene. In other words, the events are independent if the onditional probability of one given the other (e.g., PAB ) is the same as the unonditional probability of the one in the first plae (e.g., PA ). Thus, in symbols, events A and B are independent if P AB P A. This is equivalent to requiring that PB A PB. 9
q) Express this ondition for independene in terms of the probability of the intersetion of A and B. [Hint: Use the definition of onditional probability on either of the expressions above.] You should find that another equivalent definition for A and B to be independent is that PAB PA PB. Mathematiians typially take this as the definition of independene and then prove that this is equivalent to the other two onditions for probability given above. Definition: Two events A and B are independent if and only if P AB PAPB. The following theorem follows from the definition above and basi properties of onditional probabilities: Theorem: Let A and B be any two events. Then the following are equivalent: (That is, if any one of the following statements is true, then all four must be true.) 1. A and B are independent, 2. PAB PAPB, P AB P A, 3. 4. PB A PB. As a onsequene of this theorem, we only need to hek any one of the onditions above to hek for independene of two events. r) Are the events {Allan has seen the film} and {Beth has seen the film} independent? Defend your answer using any one of the equivalent definitions of independene. Then write a sentene or two explaining why your answer makes sense given the data in the table. Example: Randomly selet one ard from a standard dek of 52. Consider the following three events: A = the ard is a heart B = the ard is a fae ard C = the ard is the king of hearts 13 1 First note that there are 13 hearts, so PA. Similarly, there are 12 fae ards (4 jaks, 4 52 4 12 3. 52 13 queens, 4 kings), so PB Finally, there is only one king of hearts, so P C 1. 52 10
If we know that the event A has ourred, then we know that the ard is one of the 13 hearts. With 3 this knowledge that the ard is a heart, the probability of the event B beomes PB A. Note 13 3 that this is the same as the unonditional probability of B; that is, PB A PB. Thus A 13 and B are independent knowing that the ard is a heart has no effet at all on the probability that the ard is a fae ard. Similarly, knowing that the ard is a fae ard has no effet at all on the P A B P A ; verify this for yourself.) probability that it is a heart (that is, On the other hand, if we know that the event B has ourred, then we know that the ard is one of 1 the 12 fae ards, so PC B. This is not the same as the unonditional probability PC, 12 so B and C are not independent (they are dependent). In this ase, knowing that the ard is a fae ard substantially inreases the probability that it is the king of hearts knowing that on of the two events has ourred does have a strong impat on the probability of the other ourring so the are not independent. Finally, note that if we know that the event C has ourred, then the ard is a fae ard, so PB C 1 and PB C P B, so we have verified in a seond way that B and C are not independent. Axioms of Probability and Proofs of Basi Probability Rules In the explorations setion above, we used intuition to disover rules for probability that seem to make sense. In fat, eah of these rules an be proven using the axiomati approah to probability. In the axiomati approah, developed by the Russian mathematiian Kolmogorov (1903-1987), we begin with a small number of axioms (or assumptions). These axioms are assumed to be self-evident, and then, on the basis of a few definite rules of mathematial and logial manipulation, all other results are arefully proven or derived from these axioms. Probability theory is based on the following three axioms. Let S denote the sample spae of an experiment. Assoiated with eah event A in S is a number P(A), alled the probability of A whih satisfies the following axioms: Axiom 1: PA 0 for every event A. Axiom 2: PS 1. Axiom 3: If A and B are mutually exlusive events, then P AB P A PB. Axiom 3a: (Needed if sample spae is infinite.) If A, A, A, is an infinite sequene of mutually 1 2 3 exlusive events, then P Aj PAj. j1 j1 These axioms an be used to prove many of the other results disovered earlier in this hapter. Theorem 1: (Complement Rule) PA P A Proof: 1. 11
Theorem 2: P 0. Proof: Let A = S. Then A. Now apply Theorem 2.1 and Axiom 2 to obtain P P A 1P A 1P S 11 0. Theorem 3: If A and B are events and A B, Proof: Theorem 1 Axiom 2 then P A PB. Theorem 4: For every event A, P A Proof: 0 1. Theorem 5: (Addition Rule for Two Events) For any two events A and B, P A B P A P B P A B. Proof: 12
Corollary: (Bonferroni Inequality) For any two events A and B, P AB P A PB. Proof: By Axiom 1, PAB 0, so P AB 0. Using this inequality along with the result of Theorem 2.5, we have P AB P A P B P AB P A P B 0. Theorem 6: (Addition Rule for Three Events) For any three events A, B, and C, P ABC P A P B P C P AB P AC P BC P AB C. Proof: To prove this, write A BC AB C and then use Theorem 2.5 twie. The details are left as an exerise. Theorem 7: Let A and B be any two events. Then the following are equivalent: 1. A and B are independent, P AB P A P B, 2. 3. PAB PA 4. PB A PB,. Proof: Numbers (1) and (2) are equivalent by the definition of independene. We will prove here that (2) and (3) are also equivalent. The proof that (2) and (4) are equivalent is left as an exerise. Sine all of the other statements are equivalent to (2), all four are equivalent. We begin by showing that (2) implies (3). By the definition of onditional probability, PA B P A B P B. Assuming that (2) is true, this an be rewritten as PB P A P B P A B P A. Thus if (2) is true, then (3) must also be true. To prove that (2) and (3) are equivalent, we also need to prove the onverse of the statement above. That is, we need to prove that if (3) is true, then (2) must also be true. By the definition of onditional probability, P A B P A B P B. Assuming that (3) is true, P A B we an rewrite this as PA, so multiplying by PB P B yields the desired result: PAB PAPB. Thus (3) implies (2), ompleting the proof. Theorem 8: If A and B are independent events, then eah of the following pairs of events are also independent: 1. A and B 2. A and B 3. A and B 13
Proof: We will prove (1) here. The proofs of (2) and (3) are left as an exerise. Assume that A and B are independent. Note that the event A an A B be broken into two parts: the part of A that is inside of B, whih is denoted A B or just AB, and the part of A that is outside of B, AB AB whih is denoted A B or just AB. So A AB AB. Furthermore, the events AB and AB are learly mutually exlusive (an outome annot be both in B and not in B). Now, using Axiom 3 along with the fat that AB and AB are mutually exlusive, we obtain P A P ABAB P AB P AB. Sine A and B are independent, P AB P A PB P A P A PB P AB. Subtrating PAPB from both sides and then fatoring yields, so the equation above beomes PAB P A P A PB P A PB Finally, by the omplement rule (Theorem 2.1), 1 PB PB P AB P A PB. But this means that A and 1., so we have B are independent (by the definition of independene). Exerises 1. Suppose that you flip two fair oins. Is the sample spae of equally likely outomes properly represented as {HH, TH, HT, TT} or as {2 heads, 2 tails, 1 of eah}? Explain. 2. For eah of the following situations, list the sample spae (that is, list all possible outomes) and also indiate whether it seems reasonable to assume that all of the outomes are equally likely. If not, inlude a short explanation. a) whether or not you pass this ourse b) your grade in this ourse ) the olor of a randomly seleted M&M andy d) the outome of the roll of a fair die e) the sum of the outomes of independently rolling two fair die f) a tennis raquet landing with the label up or down when spun g) the last digit of the Soial Seurity Number of a randomly seleted Amerian h) the number of flips of a fair oin until the first heads appears 3. Suppose that you independently roll two fair four-sided die. (Eah die is equally likely to land on 1 or 2 or 3 or 4.) a) Using the notation (x,y) to mean that the first die lands on x and the seond on y, list all 16 outomes in the sample spae. b) List all of the outomes in the following events: o A = {the first die lands on 2} o B = {the sum of the die exeeds 5} o C = {the first die lands on a larger number than the seond die} o D = {the differene between the two die is one or less} 14
) Determine the probability of eah event listed in b). d) Now suppose that the two die were fair but six-sided. Realulate the probabilities of the events listed in b). (You need not list out all of the outomes in eah event.) 4. Identify whih of the following are legitimate uses of event/probability notation and whih are not. Give an explanation for the ones that are not. a) PA B d) P AB P A B b) PA PB e) PA B ) PA B P AB f) 5. Suppose that you roll two fair, six-sided die. Consider the events: A = {the first die lands on 2} B = {the sum of the die equals 7} C = {the first die lands on a larger number than the seond die} D = {the differene between the two die is one or less} E = {the sum of the die equals 11}. a) Identify all pairs of these events that are disjoint. Explain your answers. b) Identify all pairs of these events that are independent. Justify your answers with appropriate alulations. ) Identify one pair of these events with the property that learning that one has ourred makes the other more likely to have ourred. Justify your answer with appropriate alulations. d) Identify one pair of these events with the property that learning that one has ourred makes the other less likely to have ourred. Justify your answer with appropriate alulations. 6. Suppose that you hear a weather foreast announing that the probability of rain on Saturday is 50% and that the probability of rain on Sunday is 50%. Define the events A = {rain on Saturday} and U = {rain on Sunday}. a) If A and U are independent, what is the probability of rain on at least one day of the weekend? U A, what must be true of PU A b) If P.8 on at least one day of the weekend?? In this ase what is the probability of rain ) What is the largest possible value for P A U? What has to be true of this value? P U A to ahieve d) What is the smallest possible value for P A U? What has to be true of ahieve this value? 7. Given the following probabilities: PA 0.5 PB 0.6 PC 0.6 PAB 0.3 P AC 0.2 PBC 0.3 PABC 0.1 find the onditional probabilities P A B, P B C, P A B, and PB A C. P U A to 15
8. Let A and B be two events and let P A 0.4, PB p, and P A B 0.8 a. For what value of p will A and B be mutually exlusive? b. For what value of p will A and B be independent?. 9. A survey is onduted to determine the soures that people in a large metropolitan area use to get news. The survey indiates that 77% obtain news from television, 63% from newspapers, 47% from radio, 45% from television and newspapers, 29% from television and radio, 21% from newspapers and radio, and 6% from all three. a. Sketh a Venn diagram and fill in all of the appropriate probabilities. b. What proportion of people obtain news from television, but not newspapers?. What proportion of people do not obtain news from either television or radio? d. What proportion of people do not obtain news from any of these three soures? e. Given that radio is a news soure, what is the probability that a newspaper is also a news soure? f. Given that TV is a news soure, what is the probability that radio is not a news soure? g. Given that both newspaper and radio are news soures, what is the probability that TV is not a news soure? 10. Prove Theorem 2.6. Make sure that every step of your proof is learly explained and justified. Indiate all other theorems or axioms that you make use of in your proof. 11. Prove parts (2) and (3) of Theorem 2.8. Make sure that every step of your proof is learly explained and justified. Indiate all other theorems or axioms that you make use of in your proof. 12. Show that the 3 axioms of probability are satisfied by onditional probabilities. In other words, if P B prove that 0, a. PA B 0, b. PB B 1,. If A and P A A B P A B P A B. 1 2 1 2 1 2 Make sure that every step of your proof is learly explained and justified. Indiate all theorems or axioms that you make use of in your proof. A are mutually exlusive, then 16