Math 141 Lecture 1: Conditional Probability Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141
Last Time Definitions: Sample Space, Events
Last Time Definitions: Sample Space, Events Axioms or Rules of probability what are they?
Last Time Definitions: Sample Space, Events Axioms or Rules of probability what are they? Consequences: A B = P(A) P(B) P(A) = 1 P(A c ) P(A B) = P(A) + P(B) P(A B)
This Time Conditional Probability The Multiplication Rule Independent Events
Motivation A standard probability puzzle! Suppose I tell you that I have two children. You see me walking in the park with a girl, whom I introduce as my daughter. What is the probability that my other child is also a girl? The Real Question: How should you update your probability evaluations to account for new information?
Sample Space and Events Original Sample Space: Ω = {{BB}, {GB}, {BG}, {GG}}
Sample Space and Events Original Sample Space: Ω = {{BB}, {GB}, {BG}, {GG}} Event Probabilities: Assume that each of those four outcomes has probability 1/4.
Sample Space and Events Original Sample Space: Ω = {{BB}, {GB}, {BG}, {GG}} Event Probabilities: Assume that each of those four outcomes has probability 1/4. What event have we observed?
Sample Space and Events Original Sample Space: Ω = {{BB}, {GB}, {BG}, {GG}} Event Probabilities: Assume that each of those four outcomes has probability 1/4. What event have we observed? At least one child is female! {GB} {BG} {GG}
The New Sample Space The Conditional Sample Space: One event is ruled out, leaving: Ω C = {{GB}, {BG}, {GG}}
The New Sample Space The Conditional Sample Space: One event is ruled out, leaving: Ω C = {{GB}, {BG}, {GG}} If the three events are still equally likely, then each has probability 1/3.
The New Sample Space The Conditional Sample Space: One event is ruled out, leaving: Ω C = {{GB}, {BG}, {GG}} If the three events are still equally likely, then each has probability 1/3. Conditional probability The probability that both are girls, given we know at least one is a girl, is P({GG} {GB} {BG} {GG}) = 1/3
The New Sample Space The Conditional Sample Space: One event is ruled out, leaving: Ω C = {{GB}, {BG}, {GG}} If the three events are still equally likely, then each has probability 1/3. Conditional probability The probability that both are girls, given we know at least one is a girl, is P({GG} {GB} {BG} {GG}) = 1/3 The original (or marginal) probability of {GG} was 1/4.
The Formal Version Definition: Let A and B be events. Then the conditional probability of A, given we know B occurs is P(A B) = P(A B) P(B) if P(B) 0.
A Venn Diagram for Conditional Probability The definition P(A B) = P(A B)/P(B) restricts the sample space to B, and rescales to give P(B B) = 1: A A B B
The Puzzle Revisited Let A be the event {GG}, ie I have two daughters. Let B be the event that I have at least one daughter: Then A B is B = {GB} {BG} {GG}. {GG} ({GB} {BG} {GG}) = {GG}
The Computation P(A B) = = = = = P(A B) P(B) P({GG} ({GB} {BG} {GG})) P({GB} {BG} {GG}) P({GG}) P({GB} {BG} {GG}) P({GG}) P({GB}) + P({BG}) + P({GG}) 1/4 1/4 + 1/4 + 1/4 = 1/3
A Multiplication Rule Rearrange the definition of conditional probability: P(A B) = P(A B) P(B) multiplying both sides by P(B) we have P(A B) = P(A B) P(B) This gives us a tool for computing probabilities of more complicated events!
Application: Cards Suppose we have a well shuffled standard deck (52 cards): every card is equally likely to be in every position, and all possible orders of the cards are equally likely.
Application: Cards Suppose we have a well shuffled standard deck (52 cards): every card is equally likely to be in every position, and all possible orders of the cards are equally likely. What is the probability that the first card in the deck is an ace?
Application: Cards Suppose we have a well shuffled standard deck (52 cards): every card is equally likely to be in every position, and all possible orders of the cards are equally likely. What is the probability that the first card in the deck is an ace? What is the probability that the second card is an ace?
Application: Cards Suppose we have a well shuffled standard deck (52 cards): every card is equally likely to be in every position, and all possible orders of the cards are equally likely. What is the probability that the first card in the deck is an ace? What is the probability that the second card is an ace? What is the probability that the last card is an ace?
Application: Cards Suppose we have a well shuffled standard deck (52 cards): every card is equally likely to be in every position, and all possible orders of the cards are equally likely. What is the probability that the first card in the deck is an ace? What is the probability that the second card is an ace? What is the probability that the last card is an ace? What is the probability that the first two cards on the top of the deck are both aces?
The Probability of Two Aces at the Top of the Deck Let A 1 be the event that the first card is an ace, and A 2 be the event that the second card is an ace.
The Probability of Two Aces at the Top of the Deck Let A 1 be the event that the first card is an ace, and A 2 be the event that the second card is an ace. P(A 1 ) = 4/52 = 1/13
The Probability of Two Aces at the Top of the Deck Let A 1 be the event that the first card is an ace, and A 2 be the event that the second card is an ace. P(A 1 ) = 4/52 = 1/13 P(A 2 A 1 ) = 3/51 = 1/17
The Probability of Two Aces at the Top of the Deck Let A 1 be the event that the first card is an ace, and A 2 be the event that the second card is an ace. P(A 1 ) = 4/52 = 1/13 P(A 2 A 1 ) = 3/51 = 1/17 P(A 1 A 2 ) = P(A 2 A 1 ) P(A 1 ) or 4 3 52 51 = 1 13 17 = 1 221
More About Cards Suppose we have a well shuffled standard 52 card deck.
More About Cards Suppose we have a well shuffled standard 52 card deck. What is the probability that the first card is an ace given that the second card is an ace?
More About Cards Suppose we have a well shuffled standard 52 card deck. What is the probability that the first card is an ace given that the second card is an ace? What is the probability that the last card is an ace given that the first card is an ace?
More About Cards Suppose we have a well shuffled standard 52 card deck. What is the probability that the first card is an ace given that the second card is an ace? What is the probability that the last card is an ace given that the first card is an ace? What is the probability that the first card is an ace given the last card is an ace?
More About Cards Suppose we have a well shuffled standard 52 card deck. What is the probability that the first card is an ace given that the second card is an ace? What is the probability that the last card is an ace given that the first card is an ace? What is the probability that the first card is an ace given the last card is an ace? What is the probability that the second card is a king given the first is an ace?
Symmetry! P(A B) = P(A B) P(B) = P(B A) P(A)
Extensions of the Multiplication Rule P(A B) = P(A B) P(B) Assuming that P(A B C) > 0, P(A B C) = P(A B C) P(B C) = P(A B C) P(B C) P(C) The multiplication rule may be extended to arbitrary collections of sets with non-zero probability: P(A B C D) = etc. Question: Why do we need the condtion that the joint probability is positive?
Examples Question: What is the probability that the first three cards in a well shuffled deck are all Aces? Question: What is the probability that the first four cards in a well shuffled deck are all hearts?
Summary: Conditional Probability All probabilities are conditional probabilities! We always condition on the set of possible outcomes for an experiment or measurement. Sometimes new information allows us to revise our probabilities. The formula for conditional probability tells us how to do so. Sometimes we can use conditioning arguments to evaluate probabilities of complicated events!
Independent Events Independent events are events that contain no information about each other: knowing that one has occured does not help to predict whether the other will occur. Definition: Events A and B are independent if P(A B) = P(A) or P(A B) = P(A) P(B)
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently.
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently. P(H 1 H 2 ) = P(H 1 ) P(H 2 ) = 1 2 1 2 = 1 4
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently. P(H 1 H 2 ) = P(H 1 ) P(H 2 ) = 1 2 1 2 = 1 4 P(H 1 T 2 ) = P(H 1 ) P(T 2 ) = 1 4
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently. P(H 1 H 2 ) = P(H 1 ) P(H 2 ) = 1 2 1 2 = 1 4 P(H 1 T 2 ) = P(H 1 ) P(T 2 ) = 1 4 P(T 1 H 2 ) = P(T 1 ) P(H 2 ) = 1 4
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently. P(H 1 H 2 ) = P(H 1 ) P(H 2 ) = 1 2 1 2 = 1 4 P(H 1 T 2 ) = P(H 1 ) P(T 2 ) = 1 4 P(T 1 H 2 ) = P(T 1 ) P(H 2 ) = 1 4 P(T 1 T 2 ) = P(T 1 ) P(T 2 ) = 1 4
Example Toss a fair coin (P(H) = 1/2 = P(T )) twice, independently. P(H 1 H 2 ) = P(H 1 ) P(H 2 ) = 1 2 1 2 = 1 4 P(H 1 T 2 ) = P(H 1 ) P(T 2 ) = 1 4 P(T 1 H 2 ) = P(T 1 ) P(H 2 ) = 1 4 P(T 1 T 2 ) = P(T 1 ) P(T 2 ) = 1 4 P(H 1 T 1 ) =?
Important Point! In general, disjoint events are not independent: if P(A) 0, and P(A B) = 0, then P(A B) = P(A B) P(B) = 0 P(A) Similarly, independent events with non-zero probability are not disjoint. If A and B are disjoint, knowing that A has occurred tells you that B could not occur, which is informative!
Pairwise vs Complete Independence For 3 or more events, pairwise independence is not the same as complete (or mutual) independence. For complete independence, all possible combinations must be independent, including: P(A B C) = P(A) P(B A C) = P(B) P(C A B) = P(C) P(A B C) = P(A)P(B)P(C) In other words, no combination of events gives information about any other combination of events.
Example: pairwise independent but not mutually independent events Roll 2 fair dice independently, say one is red and the other blue.
Example: pairwise independent but not mutually independent events Roll 2 fair dice independently, say one is red and the other blue. Let R 1 be the event rolled 1 on the red die.
Example: pairwise independent but not mutually independent events Roll 2 fair dice independently, say one is red and the other blue. Let R 1 be the event rolled 1 on the red die. Let B 1 be the event rolled 1 on the blue die.
Example: pairwise independent but not mutually independent events Roll 2 fair dice independently, say one is red and the other blue. Let R 1 be the event rolled 1 on the red die. Let B 1 be the event rolled 1 on the blue die. Let S 7 be the event The sum of the two rolls is 7.
Example: pairwise independent but not mutually independent events Roll 2 fair dice independently, say one is red and the other blue. Let R 1 be the event rolled 1 on the red die. Let B 1 be the event rolled 1 on the blue die. Let S 7 be the event The sum of the two rolls is 7. What is P(R 1 B 1 S 7 )?
Our puzzle again Assuming that children are equally likely to be male or female, and that the sex of successive births are independent, then P(G 1 G 2 ) = P(G 1 ) P(G 2 ) = 1 2 1 2 = 1 4
Our puzzle again Assuming that children are equally likely to be male or female, and that the sex of successive births are independent, then P(G 1 G 2 ) = P(G 1 ) P(G 2 ) = 1 2 1 2 = 1 4 What is P(G 2 G 1 )?
Our puzzle again Assuming that children are equally likely to be male or female, and that the sex of successive births are independent, then P(G 1 G 2 ) = P(G 1 ) P(G 2 ) = 1 2 1 2 = 1 4 What is P(G 2 G 1 )? What is P(G 1 G 2 )?
Our puzzle again Assuming that children are equally likely to be male or female, and that the sex of successive births are independent, then P(G 1 G 2 ) = P(G 1 ) P(G 2 ) = 1 2 1 2 = 1 4 What is P(G 2 G 1 )? What is P(G 1 G 2 )? What is P(G 1 G 2 at least one girl )?
Summary Conditional Probability: P(A B) = P(A B) P(B)
Summary Conditional Probability: P(A B) = P(A B) P(B) The multiplication Rule: P(A B) = P(A B)P(B)
Summary Conditional Probability: P(A B) = P(A B) P(B) The multiplication Rule: P(A B) = P(A B)P(B) Independent events: P(A B) = P(A) or P(A B) = P(A)P(B)