Basic Probability. Probability: The part of Mathematics devoted to quantify uncertainty

Transcription

1 AMS 5 PROBABILITY

2 Basic Probability Probability: The part of Mathematics devoted to quantify uncertainty Frequency Theory Bayesian Theory Game: Playing Backgammon. The chance of getting (6,6) is 1/36. If we rolled the dice many times independently and under the same conditions, the (6,6) would turn up about 1/36 of the time. Frequency Theory

3 Basic Probability Therefore according to the Frequency Theory everyone should get the same answer when calculating chances and thus probability is Objective.

4 Basic Probability This interpretation is fine when you think about events that can be repeated. If in October 2003 you had been asked to calculate the chance that the Marlins won the World Series, it would have been very hard to think of that problem in terms of frequencies. Here is a Subjective (Bayesian) interpretation:

5 Properties Given the frequentist interpretation we observe that: This is because frequencies can not be negative and can not be above 100%. On the other hand, if you think about the subjective interpretation, you see that, unless you are willing to loose money for sure, you will never bet more than one unit. On the other hand, nobody will accept a negative bet from you, unless that person is willing to loose money for sure. So the subjective definition also implies that chance is between 0 and 1.

6 Properties From the frequentist perspective this is justified since, if A is an event, then every time A happens the opposite does not happen and vice-versa. So the proportion of times that the opposite of A happens is 1 minus the proportion of times A happens. From the subjective perspective, if you are willing to bet p on A happening to get a prize of 1, then you must be willing to bet 1-p on the opposite of A. That is, if you are willing to bet 30 cents on Florida, you must be willing to bet 70 cents on NY (this is not about your favorite team, this is about your rational quantification of each team's chances).

7 Example A box contains red marbles and blue marbles. One marble is drawn at random from the box. If it is red you win $1, if it is blue you win nothing. If you can choose between the following two boxes, which one you would prefer? 1. The box contains 3 red marbles and 2 blue ones. 2. The box contains 30 red marbles and 20 blue ones In both cases the answer is 3/5. This is because what counts is the ratio (number of red marbles)/(total number of marbles) which is the same for both boxes. This is a general rule when all possible outcomes are equally likely.

8 Events An event is a collection of possible outcomes in a specific situation. Duke wins the NCAA tournament. A dice is rolled and the outcome is greater than 4. In a sample of 20 students from this class the oldest student is less than 22 years of age. The temperature tomorrow is higher than 60 degrees. If all possible outcomes are equally likely then

9 Events You have an experiment you are thinking of performing, e.g. tossing a coin. The collection of all possible ways the experiment could come out is a set called the sample space, e.g. Ω = {heads, tails}. There is some particular event, i.e. some particular way the experiment could turn out, whose probability interests you, e.g. H={head}. Then the probability of getting head is: number of ways favorable to the event (H) 1 = total number of ways the experiment could come out 2

10 Probability vs. Statistics Inductive vs. deductive reasoning. Induction: : the process of reasoning from the part (sample) to the whole (population). Deduction: : the process of reasoning from the whole (population) to the part (sample). If you know the population fully and you choose a sampling method like random sampling, you can use probability to accurate predict what kind of sample you will get (Deduction( Deduction). If all you have is a sample and you try to infer from it back to the population that s statistics (Induction( Induction).

11 Probability vs. Statistics Probability (Deduction) POPULATION SAMPLE Statistics (Induction) Example (Probability): According to the latest census the 52.2% of the population of Boston are female. If we randomly select 100 Bostonians what is the probability of having less than 50 women? Example (Statistics): A random sample of 1000 Bostonians is chosen, and 550 of them are female. Estimate the percentage of the females in the whole population.

12 Conditional Probabilities Very often it happens that the occurrence of a certain event affects the chances of another even happening. How do we deal with that? We use conditional probabilities. Consider a deck of cards with 4 suits: clubs, diamonds, hearts and spades. Each suit has 13 cards: 2 through 10, jack, queen, king and ace. In total there are 52 cards. A deck of cards is shuffled and the top two cards are put over a table, face down. What are the chances that the second card is the queen of hearts? What are the chances that the second card is the queen of hearts given that the first card was the seven of clubs?

13 Conditional Probabilities To answer the first question think that shuffling the cards is like putting them in a random order. There are 52 possible positions for the queen of hearts and they are all equally likely. In particular the second position is one of those. So the chances are 1/52. To answer the second question we observe that we have only 51 positions left, since the seven of clubs is occupying one of them. So there is a chance of 1/51 that the second card will be the queen of hearts. The answer to the second question is a conditional probability. There is a condition on the event seven of clubs happening before we think of the event queen of hearts. In contrast the first question puts no condition on the first card.

14 Multiplication Rule Consider the following problem: a box has three tickets colored red, white and blue. We draw two tickets without replacement. What are the chances of drawing the red ticket and then the white? Since at the beginning there are three tickets and they all have the same chances, then the probability of drawing the red ticket is 1/3. Suppose the first ticket is drawn and turns out to be the red one, then there are two tickets left, both with the same chances of being drawn. So the probability of drawing the white one is 1/2. There are 3 ways of drawing the first ticket and 2 ways of drawing the second for each one of the ways of drawing the first. So there are 6 possible combinations of first and second tickets of which only one is red-white. Implying that the probability if 1/ = 6 3 2

15 Multiplication Rule Examples: 1. What is the probability that the first card taken from a deck of well shuffled cards will be the seven of clubs and the second will be the queen of hearts? 1/52 is the chance that the first one is the seven of clubs. 1/51 is the chance that the second will be the queen of hearts. So the probability is: =

16 Multiplication Rule 2. What is the probability that the first and the second cards are aces? There are 4 aces out 52 cards. So the probability of the first card being an ace is 4/52. If the first one is an ace, there are 3 aces left in the deck, so the chance that the second is also an ace is 3/51. So the probability is: = A coin is tossed twice. What are the chances that the result is a head followed by a tail? The chance of a head is 1/2. No matter what the first toss is, the probability of a tail in the second toss is 1/2. So the probability is: = 4 2 2

17 Independence Notice that in the last example is doesn't really matter that the first toss resulted in heads or tails. The results of the first toss do not affect the probability of the second toss. This is because the two tosses are independent. Example: A box contains five tickets labeled 1, 1, 2, 2 and 3. Two draws are made at random, with replacement, from the box. Suppose the first draw is a 1, what are the chances of getting a 2 in the second draw?

18 Independence When drawing with replacement the ticket that is drawn the first time is replaced back into the box. So the chances of getting a 2 in the second draw are the same for any outcome of the first draw. Suppose the draws are done without replacement. Then after the first draw there is one less ticket in the box and this affects the probability of drawing a 2 in the second draw. When two events are independent the probability that both will happen is the product of their unconditional probabilities. This is a special case of the multiplication rule.

19 Notation Consider an event A. The probability of A is denoted as P(A). Consider two events A and B. The conditional probability of A given B is denoted as P(A B). The multiplication rule can be written as: P(A & B) = P(A)P(B A)=P(B)P(A B). The events A and B are independent if: P(A B)=P(A) and P(B A)=P(B) When the events A and B are independent then the multiplication rule can be written as: P(A & B) = P(A)P(B).

20 Counting outcomes As we saw previously, when sampling at random, the probability of an event can be calculated by counting the number of possible outcomes that correspond to the event and dividing that number by the total number of outcomes. The former implies listing the ways something can happen. Let's consider the problem of rolling two dice. There are 6 possible outcomes for the first die and, for each of those, there are 6 possible outcomes of the second die to pair with the results of the first. So we have 6 x 6 = 36 possible pairs as outcomes of the experiment. We can use the following 6 x 6 table to answer a few questions regarding rolling two dice.

21 Counting outcomes Q: What is the probability that the sum will be equal to 2? A: There is only one way of getting 2 as the sum of the two dice, so the probability is 1/36. Q: What is the probability that the sum will be 7? A: There are 6 possible ways of getting a sum of 7, thus the probability is 6/36 = 1/6.

22 Counting outcomes With three dice life is even more complicated. To start with we have 6 possibilities for each die. That produces 6 x 6 x 6 = 6 3 = 216 possible outcomes. Suppose we want to know the chances that the sum of the three dice is 9. We can do that by obtaining the following combinations If we think of the chances that the sum will be equal to 10 we also get six possible combinations Nevertheless the chances of a 9 and a 10 are different since the listed combinations can be obtained in a different number of ways. In fact the following table counts the number of triplets that produce each outcome.

23 Counting outcomes So the probability that the sum will be 9 is 25/216 whilst the probability that sum will be 10 is 27/216.

24 Addition Rule We saw that the multiplicative rule is useful when looking at two events that occur jointly. So it is related to the problem of observing A and B. Let's consider the or case. Two dice are rolled and the sum of the two is observed. The events: the sum is greater than 6 and the sum is smaller than 3 are disjoint.

25 Addition Rule Q: What are the chances that a card from a well shuffled deck will be either hearts or spades? A: We either think that there are 26 out of the 52 cards that are either spades or hearts, and so the probability is 26/52 = 1/2. Or we can think that the chance of the card being hearts is 1/4 and the chance of it being spades is 1/4. Since the two events are disjoint, the chance of either spades or hearts is 1/4+1/4 = 1/2. Q: What are the chances of getting at least one 1 when two dice are rolled? A: Consider the events: 1 in the red die and one in the blue die. If we use the addition rule then the chances are 1/6+1/6 = 1/3 = 12/36. But if we observe the table we have that there are 11 ways of getting a 1. Thus the chances are just 11/36.

26 Addition Rule The addition rule does not apply in this case since the two events are not disjoint. The addition rule is counting the outcome (1, 1) twice. The right answer is obtained by subtracting the probability that the events happen simultaneously. In the dice example we have 1/6 + 1/6-1/36 = 11/36, which is the correct answer. The mathematical notation for this is: P(A or B) = P(A) + P(B) - P(A and B)

27 Independence vs. Disjoint Q: If two events are disjoint, are they also independent? A: No. Actually when two events are disjoint, the occurrence of one gives a lot of information about the occurrence of the other. In particular, the other can not happen! Remember that two events are independent when the probability of one is unaffected by the other event happening. Since this is not the case when events are disjoint, then they are not independent. Suppose A is an event and denote A c as the opposite of A. Then A and A c are disjoint but, for any value of P(A), P(A A c ) = 0, so they are not independent.

28 Examples Q1: What is the probability that in four rolls of a die at least one 1 will turn up? A: In one roll of a die there is 1/6 probability of getting a 1. In 4 rolls there is: = 6 3 Q2: What is the probability that in 24 rolls of a pair of dice, at least one double 1 will turn up? A: In one roll of a pair of dice there is 1/36 chance of getting a double 1. In 24 rolls the chances are: = 36 3 We are adding probabilities for non mutually exclusive events. To see how wrong our argument is, think of rolling one die 8 times, then we 1 8 > 1 6 would get probability of observing a 1!

29 What is the right calculation? Examples First problem: Think of the opposite event. The gambler looses if none of the four rolls come up 1. What are the chances of not getting a 1 in a specific roll? This can be calculated as 1-1/6 = 5/6. For the gambler to loose this has to happen the first and the second and the third and the fourth. The rolls are all independent, thus we use the multiplication rule and obtain (5/6) 4 = Thus the chance of winning is = Second problem: In a similar way we get the chances of the event which is the opposite of getting a double 1 in one particular roll: 35/36. We want this to happen 24 times, so we get (35/26) 24 = The chance of winning is =

30 Components in Series/Parallel A system of three components in series is such that the whole system works if all the components work. That corresponds to the figure If all three components are independent and they have probability P1, P2 and P3 of working properly, then the system will work properly if all three components work and that, using the multiplication rule, happens with probability P = P1 P2 P3 This can be generalized to an arbitrary number of components.

31 Components in Series/Parallel Suppose that a system is made of three component and it is such that it will work if any of the three components work properly. Then the components are in parallel. This corresponds to the figure below. To work out the probability that the system will function, consider the opposite event: the system will not function. This only happens if all three components do not function.

32 Components in Series/Parallel The probability that component 1 will not function is 1-P1. For component 2 we have 1-P2 and for component 3 we have 1-P3. Using the multiplicative rule we obtain that the system will not function with probability (1-P1)(1-P2)(1-P3). So the probability that the system will function is: P = 1-(1-P1)(1-P2)(1-P3) Again, this can be generalized to an arbitrary number of components. Suppose P1 = P2 = P3 = 1/3 then: Series: P = (1/3) 3 = 1 / 27. Parallel: P = 1-(1-1/3) 3 =19 / 27.

33 Summary of Probability Rules 0 P(A) 1. P(not A) = 1 P(A). P(A or B) = P(A) + P(B) P(AB) (addition rule). If A, B are disjoint then P(A or B) = P(A) + P(B). P(A and B) = P(A)P(B A)=P(B)P(A B) (multiplication rule). If A, B are independent then P(A and B) = P(A)P(B). From the multiplication rule we get the definition of the conditional probability: P(A and B) P(A B) = P(B)