DUBLIN INSTITUTE OF TECHNOLOGY KEVIN STREET, DUBLIN 8. Probability Based Learning: Introduction to Probability REVISION QUESTIONS *** SOLUTIONS ***

Transcription

1 DUBLIN INSTITUTE OF TECHNOLOGY KEVIN STREET, DUBLIN 8 Probability Based Learig: Itroductio to Probability REVISION QUESTIONS *** *** MACHINE LEARNING AT DIT Dr. Joh Kelleher Dr. Bria Mac Namee *** *** *** ***

2 *** *** 1. Give the joit distributio for X ad Y listed i Table 1 calculate the followig: (a) P (X = x 1 ) ( marks) = 0.26 (b) P (Y = y 2 ) ( marks) = 0.46 (c) P (Y = y 2 X = x 1 ) ( marks) From the product rule: P (a b) = P (a b) P (Y = y 2 X = x 1 ) = P (Y = y 2 X = x 1 ) = 0.14 P (b) P (Y =y2 X=x1) P (X=x 1) 0.26 Table 1: Joit Distributio for X ad Y X = x 1 X = x 2 Y = y Y = y Y = y Give that P (a b) = 0.5, P (a) = 0.3, P (b) = 0.4 calculate P (b a). P (b a) = P (a b) P (b) P (a) = = 0.67 (x marks) 3. Cosider the domai of dealig 5-card poker hads from a stadard deck of 52 cards, uder the assumptio that the dealer is fair. (a) Give that the umber of combiatios of r objects that ca be selected, without regard to order ad without repetitio, from distict objects is give by the equatio ( ) r =! ( r)!r!. How may atomic evets are there i the joit probability distributio (i.e., how may 5-card hads are there)? There are ( ) 52 5 = 52! (47!5! = 52?51?50?49?48 1?2?3?4?5 = 2, 598, 960 possible five-card hads *** *** 1 *** ***

3 *** *** (b) What is the probability of each atomic evet? By the fair-dealig assumptio, each of these is equally likely. Each had therefore occurs with probability 1/2,598,960. (c) What is the probability of beig dealt a royal straight flush (i.e. beig dealt a had cotaiig Ace, Kig, Quee, Jack ad 10 all from the oe suit)? There are four hads that are royal straight ushes (oe i each suit). These evets are are mutually exclusive, therefore the probability of a royal ush is just the sum of the probabilities of the atomic evets, i.e., 4 2,598,960 = 1 649,740. (d) What is the probability of beig dealt a four of a kid (i.e. four kigs, or four ies etc.? There are 13 possible kids ad for each, the fth card ca be oe of 48 possible other cards. The total probability is therefore Give the full joit distributio show i Table 2, calculate the followig: (a) P (toothache) Table 2: Full joit distributio for a detist visit toothache toothache catch catch catch catch cavity cavity ,598,960 = 1 4,165. This asks for the probability that T oothache is true. P (toothache) = = 0.2 (b) P(Cavity) This asks for the vector of probability values for the radom variable Cavity. It has two values, which we list i the order true, false. First add up P (cavity) = = 0.2. The we have textbfp (Cavity) = 0.2, 0.8. (c) P(T oothache cavity) This asks for the vector of probability values for T oothache, give that Cavity is true. textbfp (T oothache cavity) = , = 0.6, 0.4 (d) P(Cavity toothache catch) *** *** 2 *** ***

4 *** *** This asks for the vector of probability values for Cavity, give that either T oothache or Catch is true. Recall P (a b) = P (a b) P (b) P(Cavity toothache catch) = P (cavity (toothache cavity)) P (toothache catch), P ( cavity (toothache cavity)) P (toothache catch) First compute P (toothache catch) = = The P(Cavity toothache catch) = , = , After you yearly checkup, the doctor has bad ews ad good ews. The bad ews is that you tested positive for a serious disease ad that the test is 99% accurate (i.e., the probability of testig positive whe you do have the disease is 0.99, as is the probability of testig egative whe you do t have the disease). The good ews is that this is a rare disease, strikig oly 1 i 10,000 people of your age. Why is it good ews that the disease is rare? What are the chaces that you actually have the disease? We are give the followig iformatio: P (test disease) = 0.99 P ( test disease) = 0.99 P (disease) = ad the observatio test. What the patiet is cocered about is P (disease test). Roughly speakig, the reaso it is a good thig that the disease is rare is that P (disease test) is proportioal to P (disease), so a lower prior for disease will mea a lower value for P (disease test). Roughly speakig, if 10,000 people take the test, we expect 1 to actually have the disease, ad most likely test positive, while the rest do ot have the disease, but 1 percet of them (about 100 people) will test positive ayway, so P (disease test) will be about 1 i 100. More P (b a)p (a) precisely: P (a b) = P (a) P (test disease)p (disease) P (disease test) = P (test) P (test disease)p (disease) P (disease test) = P (test disease)p (disease)+p (test disease)p ( disease) ( )+( ) = The moral is that whe the disease is much rarer tha the test a ccuracy, a positive test result does ot mea the disease is likely. A false positive readig remais much more likely. 6. Suppose you are give a bag cotaiig ubiased cois. You are told that 1 of these cois are ormal, with heads o oe side ad tails o the other, whereas oe is is fake, with heads o both sides. (a) Suppose you reach ito the bag, pickig out a coi uiformly at radom, flip it, ad get a head. What is the (coditioal) probability that the coi you chose is the fake coi? *** *** 3 *** ***

5 *** *** There are ways to pick a coi, ad 2 outcomes for each flip (although with the fake coi, the results of the ip are idistiguishable), so there are total atomic evets, each equally likely. Of those, oly 2 pick the fake coi, ad 2 + ( 1) result i heads (the two heads results for the fake coi ad half the rest of the flips (-1)). So the probability of a fake coi give heads, P (f ake heads), is P (heads fake)p (fake) P (heads). P (heads fake) = 1 P (fake) = 1 P (heads) = totals umber of head results P (fake heads) = P (fake heads) = 1 2 P (fake heads) = total atomic evets ( 1) 2+( 1) 2+( 1) = 2+( 1) P (fake heads) = 2 +1) More formally: Let α be a ormalisig costat. P(F ake heads) = αp(heads F ake)p(f ake) P(F ake heads) = α P (heads fake), P (heads fake) P (fake), P ( fake) P(F ake heads) = α 1.0, 0.5 1, ( 1) P(F ake heads) = α 1, ( 1) Let us compute α 1 + ( 1) = 2+( 1) = +1 By defiitio α +1 = 1 α = +1 Pluggig α = back ito our equatio +1 P(F ake heads) = 1 P(F ake heads) = 2 +1, ( 1) +1, ( 1) (b) Suppose you cotiue flippig the coi for a total of k times after pickig it ad see k head. Nows what is the coditioal probability that you picked the fake coi? *** *** 4 *** ***

6 *** *** Now there are 2 k atomic evets: (i) there are ways to pick a coi, (ii) there are k flips of the picked coi. Each flip ca result i oe of 2 outcomes (although with the fake coi, the results of the ip are idistiguishable). As a result there are 2 k possible outcomes for k flips. (iii) so there are 2 k atomic evets i the domai (outcomes of flips ways of pickig a coi Of these 2 k atomic evets 2 k pick the fake coi, ad 2 k + ( 1) result i heads. So the probability of a fake coi give a ru of k heads, P (fake heads k 2 ), is k. Note this approaches 1 as k icreases, (2 k +( 1)) as expected. Doig it the formal way: P(F ake heads k ) = αp(heads k F ake)p(f ake) P(F ake heads k ) = α 1.0, 0.5 k 1, ( 1) P(F ake heads k ) = α 1, ( 1) 2 k Let us compute α 1 + ( 1) 2 k = 2k +( 1) 2 k By defiitio α = 2k 2 k +( 1) Pluggig α = P(F ake heads k ) = back ito our equatio 2k 2 k +( 1) 2 k 2 k +( 1), ( 1) 2 k +( 1 (c) Suppose you wated to decide whether the chose coi was fake by flippig it k times. The decisio procedure returs FAKE if all k flips come up heads, otherwise it returs NORMAL. What is the (ucoditioal) probability that this procedure makes a error? The procedure makes a error if ad oly if a fair coi is chose ad turs up heads k times i a row. The probability of this is: P ( fake heads k ) = P (heads k fake)p ( fake) P ( fake heads k ) = 0.5 k 1 P ( fake heads k ) = 1 2 k 7. Suppose you are a witess to a ighttime hit-ad-ru accidet ivolvig a taxi i Athes. All taxis i Athes are blue or gree. You swear, uder oath, that the taxi was blue. Extesive testig shows that, uder the dim lightig coditios, discrimiatio betwee blue ad gree is 75% reliable. (a) Is it possible to calculate the most likely color for the taxi? (Hit: distiguish carefully betwee the propositio that the taxi is blue ad the propositio that it appears blue.) *** *** 5 *** ***

7 *** *** The relevat aspect of the world ca be described by two radom variables: B meas the taxi was blue, ad LB meas the taxi looked blue. The iformatio o the reliability of color ideticatio ca be writte as P (LB B) = 0.75 ad P ( LB B) = 0.75 We eed to kow the probability that the taxi was blue, give that it looked blue: P (B LB) P (LB B)P (B) 0.75P (B) P ( B LB) P (LB B)P ( B) 0.25(1?P (B)) Thus we caot decide the probability without some iformatio about the prior probability of blue taxis, P (B). For example, if we kew that all taxis were blue, i.e., P (B) = 1, the obviously P(B LB) = 1. O the other had, if we adopt Laplaces Priciple of Idifferece, which states that propositios ca be deemed equally likely i the absece of ay differetiatig iformatio, the we have P (B) = 0.5 ad P (B LB) = Usually we will have some differetiatig iformatio, so this priciple does ot apply. (b) What about ow, give that 9 out of 10 Atheia taxis are gree? Give that 9 out of 10 taxis are gree, ad assumig the taxi i questio is draw radomly from the taxi populatio, we have P (B) = 0.1. Hece P (B LB) P ( B LB) P (B LB) = = P ( B LB) = = 0.75 *** *** 6 *** ***