# Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

Save this PDF as:

Size: px
Start display at page:

Download "Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11"

## Transcription

1 CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According to clinical trials, the test has the following properties:. When applied to an affected person, the test comes up positive in 90% of cases, and negative in 0% (these are called false negatives ). 2. When applied to a healthy person, the test comes up negative in 80% of cases, and positive in 20% (these are called false positives ). Suppose that the incidence of the condition in the US population is 5%. When a random person is tested and the test comes up positive, what is the probability that the person actually has the condition? (Note that this is presumably not the same as the simple probability that a random person has the condition, which is just 20.) This is an example of a conditional probability: we are interested in the probability that a person has the condition (event A) given that he/she tests positive (event B). Let s write this as Pr[A B]. How should we define Pr[A B]? Well, since event B is guaranteed to happen, we should look not at the whole sample space Ω, but at the smaller sample space consisting only of the sample points in B. What should the conditional probabilities of these sample points be? If they all simply inherit their probabilities from Ω, then the sum of these probabilities will be ω B Pr[ω] =, which in general is less than. So we should normalize the probability of each sample point by. I.e., for each sample point ω B, the new probability becomes Pr[ω B] = Pr[ω]. Now it is clear how to define Pr[A B]: namely, we just sum up these normalized probabilities over all sample points that lie in both A and B: Pr[A B] := ω A B Pr[ω B] = ω A B Pr[ω] =. Definition. (conditional probability): For events A, B in the same probability space, such that > 0, the conditional probability of A given B is Pr[A B] :=. Let s go back to our medical testing example. The sample space here consists of all people in the US denote their number by N (so N 250 million). The population consists of four disjoint subsets: CS 70, Fall 2009, Note

2 T P: the true positives (90% of N 20 = 9N 200 of them); FP: the false positives (20% of 9N 20 = 9N 00 of them); T N: the true negatives (80% of 9N 20 = 76N 00 of them); FN: the false negatives (0% of N 20 = N 200 of them). Now let A be the event that a person chosen at random is affected, and B the event that he/she tests positive. Note that B is the union of the disjoint sets T P and FP, so Thus we have B = TP + FP = 9N N 00 = 47N 200. Pr[A] = 20 and = Now when we condition on the event B, we focus in on the smaller sample space consisting only of those 47N 200 individuals who test positive. To compute Pr[A B], we need to figure out (the part of A that lies in B). But A B is just the set of people who are both affected and test positive, i.e., A B = T P. So we have T P = N = Finally, we conclude from the definition of conditional probability that = 9/200 47/200 = This seems bad: if a person tests positive, there s only about a 9% chance that he/she actually has the condition! This sounds worse than the original claims made by the pharmaceutical company, but in fact it s just another view of the same data. [Incidentally, note that Pr[B A] = 9/200 /20 = 9 0 ; so Pr[A B] and Pr[B A] can be very different. Of course, Pr[B A] is just the probability that a person tests positive given that he/she has the condition, which we knew from the start was 90%.] To complete the picture, what s the (unconditional) probability that the test gives a correct result (positive or negative) when applied to a random person? Call this event C. Then Pr[C] = T P + TN N = = So the test is about 80% effective overall, a more impressive statistic. But how impressive is it? Suppose we ignore the test and just pronounce everybody to be healthy. Then we would be correct on 95% of the population (the healthy ones), and wrong on the affected 5%. I.e., this trivial test is 95% effective! So we might ask if it is worth running the test at all. What do you think? Here are a couple more examples of conditional probabilities, based on some of our sample spaces from the previous lecture note.. Balls and bins. Suppose we toss m = 3 balls into n = 3 bins; this is a uniform sample space with 3 3 = 27 points. We already know that the probability the first bin is empty is ( 3 )3 = ( 2 3 )3 = What is the probability of this event given that the second bin is empty? Call these events A,B CS 70, Fall 2009, Note 2

3 respectively. To compute Pr[A B] we need to figure out. But A B is the event that both the first two bins are empty, i.e., all three balls fall in the third bin. So = 27 (why?). Therefore, = /27 8/27 = 8. Not surprisingly, 8 8 is quite a bit less than 27 : knowing that bin 2 is empty makes it significantly less likely that bin will be empty. 2. Dice. Roll two fair dice. Let A be the event that their sum is even, and B the event that the first die is even. By symmetry it s easy to see that Pr[A] = 2 and = 2. Moreover, a little counting gives us that = 4. What is Pr[A B]? Well, = /4 /2 = 2. In this case, Pr[A], i.e., conditioning on B does not change the probability of A. Bayesian Inference The medical test problem is a canonical example of an inference problem: given a noisy observation (the result of the test), we want to figure out the likelihood of something not directly observable (whether a person is healthy). To bring out the common structure of such inference problems, let us redo the calculations in the medical test example but only in terms of events without explicitly mentioning the sample points of the underlying sample space. Recall: A is the event the person is affected, B is the event that the test is positive. What are we given? Pr[A] = 0.05, (5% of the U.S. population is affected.) Pr[B A] = 0.9 (90% of the affected people test positive) Pr[B A] = 0.2 (20% of healthy people test positive) We want to calculate Pr[A B]. We can proceed as follows: = Pr[B A]Pr[A] () and = + = Pr[B A]Pr[A] + Pr[B A]( Pr[A]) (2) Combining equations () and (2), we have expressed Pr[A B] in terms of Pr[A], Pr[B A] and Pr[B A]: Pr[B A] Pr[A] Pr[B A] Pr[A] + Pr[B A]( Pr[A]) (3) This equation is useful for many inference problems. We are given Pr[A], which is the (unconditional) probability that the event of interest A happens. We are given Pr[B A] and Pr[B A], which quantify how noisy CS 70, Fall 2009, Note 3

4 the observation is. (If Pr[B A] = and Pr[B A] = 0, for example, the observation is completely noiseless.) Now we want to calculate Pr[A B], the probability that the event of interest happens given we made the observation. Equation (3) allows us to do just that. Equation (3) is at the heart of a subject called Bayesian inference, used extensively in fields such as machine learning, communications and signal processing. The equation can be interpreted as a way to update knowledge after making an observation. In this interpretation, Pr[A] can be thought of as a prior probability: our assessment of the likelihood of an event of interest A before making an observation. It reflects our prior knowledge. Pr[A B] can be interpreted as the posterior probability of A after the observation. It reflects our new knowledge. Of course, equations (), (2) and (3 are derived from the basic axioms of probability and the definition of conditional probability, and are therefore true with or without the above Bayesian inference interpretation. However, this interpretation is very useful when we apply probability theory to study inference problems. Bayes Rule and Total Probability Rule Equations () and (2) are very useful in their own right. The first is called Bayes Rule and the second is called the Total Probability Rule. Bayes Rule is useful when one wants to calculate Pr[A B] but one is given Pr[B A] instead, i.e. it allows us to flip things around. The Total Probability Rule is an application of the strategy of dividing into cases we learnt in Notes 2 to calculating probabilities. We want to calculate the probability of an event B. There are two possibilities: either an event A happens or A does not happen. If A happens the probability that B happens is Pr[B A]. If A does not happen, the probability that B happens is Pr[B A]. If we know or can easily calculate these two probabilities and also Pr[A], then the total probability rule yields the probability of event B. Independent events Definition.2 (independence): Two events A, B in the same probability space are independent if Pr[A B] = Pr[A]. The intuition behind this definition is the following. Suppose that > 0. Then we have = Pr[A] = Pr[A]. Thus independence has the natural meaning that the probability of A is not affected by whether or not B occurs. (By a symmetrical argument, we also have Pr[B A] = provided Pr[A] > 0.) For events A, B such that > 0, the condition Pr[A] is actually equivalent to the definition of independence. Examples: In the balls and bins example above, events A,B are not independent. In the dice example, events A, B are independent. The above definition generalizes to any finite set of events: Definition.3 (mutual independence): Events A,...,A n are mutually independent if for every subset I {,...,n}, Pr[ i I A i ] = i I Pr[A i ]. Note that we need this property to hold for every subset I. For mutually independent events A,...,A n, it is not hard to check from the definition of conditional probability that, for any i n and any subset I {,...,n} \ {i}, we have Pr[A i j I A j ] = Pr[A i ]. CS 70, Fall 2009, Note 4

5 Note that the independence of every pair of events (so-called pairwise independence) does not necessarily imply mutual independence. For example, it is possible to construct three events A, B,C such that each pair is independent but the triple A, B,C is not mutually independent. Combinations of events In most applications of probability in Computer Science, we are interested in things like Pr[ n i= A i ] and Pr[ n i= A i ], where the A i are simple events (i.e., we know, or can easily compute, the Pr[A i ]). The intersection i A i corresponds to the logical AND of the events A i, while the union i A i corresponds to their logical OR. As an example, if A i denotes the event that a failure of type i happens in a certain system, then i A i is the event that the system fails. In general, computing the probabilities of such combinations can be very difficult. In this section, we discuss some situations where it can be done. Intersections of events From the definition of conditional probability, we immediately have the following product rule (sometimes also called the chain rule) for computing the probability of an intersection of events. Theorem.: [Product Rule] For any events A, B, we have More generally, for any events A,...,A n, = Pr[A]Pr[B A]. Pr[ n i= A i ] = Pr[A ] Pr[A 2 A ] Pr[A 3 A A 2 ] Pr[A n n i= A i]. Proof: The first assertion follows directly from the definition of Pr[B A] (and is in fact a special case of the second assertion with n = 2). To prove the second assertion, we will use induction on n (the number of events). The base case is n =, and corresponds to the statement that Pr[A] = Pr[A], which is trivially true. For the inductive step, let n > and assume (the inductive hypothesis) that Pr[ n i= A i] = Pr[A ] Pr[A 2 A ] Pr[A n n 2 i= A i]. Now we can apply the definition of conditional probability to the two events A n and n i= A i to deduce that Pr[ n i= A i ] = Pr[A n ( n i= A i)] = Pr[A n n i= A i] Pr[ n i= A i] = Pr[A n n i= A i] Pr[A ] Pr[A 2 A ] Pr[A n n 2 i= A i], where in the last line we have used the inductive hypothesis. This completes the proof by induction. The product rule is particularly useful when we can view our sample space as a sequence of choices. The next few examples illustrate this point.. Coin tosses. Toss a fair coin three times. Let A be the event that all three tosses are heads. Then A = A A 2 A 3, where A i is the event that the ith toss comes up heads. We have Pr[A] = Pr[A ] Pr[A 2 A ] Pr[A 3 A A 2 ] = Pr[A ] Pr[A 2 ] Pr[A 3 ] = = 8. CS 70, Fall 2009, Note 5

6 The second line here follows from the fact that the tosses are mutually independent. Of course, we already know that Pr[A] = 8 from our definition of the probability space in the previous lecture note. The above is really a check that the space behaves as we expect. If the coin is biased with heads probability p, we get, again using independence, Pr[A] = Pr[A ] Pr[A 2 ] Pr[A 3 ] = p 3. And more generally, the probability of any sequence of n tosses containing r heads and n r tails is p r ( p) n r. This is in fact the reason we defined the probability space this way in the previous lecture note: we defined the sample point probabilities so that the coin tosses would behave independently. 2. Balls and bins. Let A be the event that bin is empty. We saw in the previous lecture note (by counting) that Pr[A] = ( n )m, where m is the number of balls and n is the number of bins. The product rule gives us a different way to compute the same probability. We can write A = m i= A i, where A i is the event that ball i misses bin. Clearly Pr[A i ] = n for each i. Also, the A i are mutually independent since ball i chooses its bin regardless of the choices made by any of the other balls. So ( Pr[A] = Pr[A ] Pr[A m ] = m. n) 3. Card shuffling. We can look at the sample space as a sequence of choices as follows. First the top card is chosen uniformly from all 52 cards, i.e., each card with probability 52. Then (conditional on the first card), the second card is chosen uniformly from the remaining 5 cards, each with probability 5. Then (conditional on the first two cards), the third card is chosen uniformly from the remaining 50, and so on. The probability of any given permutation, by the product rule, is therefore = 52!. Reassuringly, this is in agreement with our definition of the probability space in the previous lecture note, based on counting permutations. 4. Poker hands. Again we can view the sample space as a sequence of choices. First we choose one of the cards (note that it is not the first card, since the cards in our hand have no ordering) uniformly from all 52 cards. Then we choose another card from the remaining 5, and so on. For any given poker hand, the probability of choosing it is (by the product rule): = ( 52 ), 5 just as before. Where do the numerators 5, 4, 3, 2, come from? Well, for the given hand the first card we choose can be any of the five in the hand: i.e., five choices out of 52. The second can be any of the remaining four in the hand: four choices out of 5. And so on. This arises because the order of the cards in the hand is irrelevant. Let s use this view to compute the probability of a flush in a different way. Clearly this is 4 Pr[A], where A is the probability of a Hearts flush. And we can write A = 5 i= A i, where A i is the event that the ith card we pick is a Heart. So we have Pr[A] = Pr[A ] Pr[A 2 A ] Pr[A 5 4 i= A i ]. Strictly speaking, we should really also have checked from our original definition of the probability space that Pr[A ], Pr[A 2 A ] and Pr[A 3 A A 2 ] are all equal to 2. CS 70, Fall 2009, Note 6

7 Clearly Pr[A ] = 3 52 = 4. What about Pr[A 2 A ]? Well, since we are conditioning on A (the first card is a Heart), there are only 5 remaining possibilities for the second card, 2 of which are Hearts. So Pr[A 2 A ] = 2 5. Similarly, Pr[A 3 A A 2 ] = 50, and so on. So we get 4 Pr[A] = , which is exactly the same fraction we computed in the previous lecture note. So now we have two methods of computing probabilities in many of our sample spaces. It is useful to keep these different methods around, both as a check on your answers and because in some cases one of the methods is easier to use than the other. 5. Monty Hall. Recall that we defined the probability of a sample point by multiplying the probabilities of the sequence of choices it corresponds to; thus, e.g., Pr[(,,2)] = = 8. The reason we defined it this way is that we knew (from our model of the problem) the probabilities for each choice conditional on the previous one. Thus, e.g., the 2 in the above product is the probability that Carol opens door 2 conditional on the prize door being door and the contestant initially choosing door. In fact, we used these conditional probabilities to define the probabilities of our sample points. Unions of events You are in Las Vegas, and you spy a new game with the following rules. You pick a number between and 6. Then three dice are thrown. You win if and only if your number comes up on at least one of the dice. The casino claims that your odds of winning are 50%, using the following argument. Let A be the event that you win. We can write A = A A 2 A 3, where A i is the event that your number comes up on die i. Clearly Pr[A i ] = 6 for each i. Therefore, Pr[A] = Pr[A A 2 A 3 ] = Pr[A ] + Pr[A 2 ] + Pr[A 3 ] = 3 6 = 2. Is this calculation correct? Well, suppose instead that the casino rolled six dice, and again you win iff your number comes up at least once. Then the analogous calculation would say that you win with probability 6 6 =, i.e., certainly! The situation becomes even more ridiculous when the number of dice gets bigger than 6. The problem is that the events A i are not disjoint: i.e., there are some sample points that lie in more than one of the A i. (We could get really lucky and our number could come up on two of the dice, or all three.) So if we add up the Pr[A i ] we are counting some sample points more than once. Fortunately, there is a formula for this, known as the Principle of Inclusion/Exclusion: Theorem.2: [Inclusion/Exclusion] For events A,...,A n in some probability space, we have Pr[ n i= A i ] = n i= Pr[A i ] Pr[A i A j ] + Pr[A i A j A k ] ± Pr[ n i= A i ]. {i, j} {i, j,k} [In the above summations, {i, j} denotes all unordered pairs with i j, {i, j, k} denotes all unordered triples of distinct elements, and so on.] CS 70, Fall 2009, Note 7

8 I.e., to compute Pr[ i A i ], we start by summing the event probabilities Pr[A i ], then we subtract the probabilities of all pairwise intersections, then we add back in the probabilities of all three-way intersections, and so on. We won t prove this formula here; but you might like to verify it for the special case n = 3 by drawing a Venn diagram and checking that every sample point in A A 2 A 3 is counted exactly once by the formula. You might also like to prove the formula for general n by induction (in similar fashion to the proof of the Product Rule above). Taking the formula on faith, what is the probability we get lucky in the new game in Vegas? Pr[A A 2 A 3 ] = Pr[A ] + Pr[A 2 ] + Pr[A 3 ] Pr[A A 2 ] Pr[A A 3 ] Pr[A 2 A 3 ] + Pr[A A 2 A 3 ]. Now the nice thing here is that the events A i are mutually independent (the outcome of any die does not depend on that of the others), so Pr[A i A j ] = Pr[A i ]Pr[A j ] = ( 6 )2 = 36, and similarly Pr[A A 2 A 3 ] = ( 6 )3 = 26. So we get Pr[A A2 A3] = ( 3 ) ( ) = So your odds are quite a bit worse than the casino is claiming! When n is large (i.e., we are interested in the union of many events), the Inclusion/Exclusion formula is essentially useless because it involves computing the probability of the intersection of every non-empty subset of the events: and there are 2 n of these! Sometimes we can just look at the first few terms of it and forget the rest: note that successive terms actually give us an overestimate and then an underestimate of the answer, and these estimates both get better as we go along. However, in many situations we can get a long way by just looking at the first term:. Disjoint events. If the events A i are all disjoint (i.e., no pair of them contain a common sample point such events are also called mutually exclusive), then Pr[ n i= A i ] = n i= Pr[A i ]. [Note that we have already used this fact several times in our examples, e.g., in claiming that the probability of a flush is four times the probability of a Hearts flush clearly flushes in different suits are disjoint events.] 2. Union bound. Always, it is the case that Pr[ n i= A i ] n i= Pr[A i ]. This merely says that adding up the Pr[A i ] can only overestimate the probability of the union. Crude as it may seem, in the next lecture note we ll see how to use the union bound effectively in a Computer Science example. CS 70, Fall 2009, Note 8

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice,

### Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22

CS 70 Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice, roulette

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### 6.3 Conditional Probability and Independence

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

### Topic 6: Conditional Probability and Independence

Topic 6: September 15-20, 2011 One of the most important concepts in the theory of probability is based on the question: How do we modify the probability of an event in light of the fact that something

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 13. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 3 Random Variables: Distribution and Expectation Random Variables Question: The homeworks of 20 students are collected

### Chapter 3: The basic concepts of probability

Chapter 3: The basic concepts of probability Experiment: a measurement process that produces quantifiable results (e.g. throwing two dice, dealing cards, at poker, measuring heights of people, recording

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Probabilistic Strategies: Solutions

Probability Victor Xu Probabilistic Strategies: Solutions Western PA ARML Practice April 3, 2016 1 Problems 1. You roll two 6-sided dice. What s the probability of rolling at least one 6? There is a 1

### Finite and discrete probability distributions

8 Finite and discrete probability distributions To understand the algorithmic aspects of number theory and algebra, and applications such as cryptography, a firm grasp of the basics of probability theory

### PROBABILITY NOTIONS. Summary. 1. Random experiment

PROBABILITY NOTIONS Summary 1. Random experiment... 1 2. Sample space... 2 3. Event... 2 4. Probability calculation... 3 4.1. Fundamental sample space... 3 4.2. Calculation of probability... 3 4.3. Non

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2 Proofs Intuitively, the concept of proof should already be familiar We all like to assert things, and few of us

### Basics of Probability

Basics of Probability August 27 and September 1, 2009 1 Introduction A phenomena is called random if the exact outcome is uncertain. The mathematical study of randomness is called the theory of probability.

### Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80)

Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you

### Introduction to Probability

Massachusetts Institute of Technology Course Notes 0 6.04J/8.06J, Fall 0: Mathematics for Computer Science November 4 Professor Albert Meyer and Dr. Radhika Nagpal revised November 6, 00, 57 minutes Introduction

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics.

Elementary probability theory The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics. 1.1 Experiments, outcomes and sample

### Question: What is the probability that a five-card poker hand contains a flush, that is, five cards of the same suit?

ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the

### Topic 1 Probability spaces

CSE 103: Probability and statistics Fall 2010 Topic 1 Probability spaces 1.1 Definition In order to properly understand a statement like the chance of getting a flush in five-card poker is about 0.2%,

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### A set is a Many that allows itself to be thought of as a One. (Georg Cantor)

Chapter 4 Set Theory A set is a Many that allows itself to be thought of as a One. (Georg Cantor) In the previous chapters, we have often encountered sets, for example, prime numbers form a set, domains

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University

Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments

### Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to

### A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes.

Chapter 7 Probability 7.1 Experiments, Sample Spaces, and Events A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes. Each outcome

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Jan 17 Homework Solutions Math 151, Winter 2012. Chapter 2 Problems (pages 50-54)

Jan 17 Homework Solutions Math 11, Winter 01 Chapter Problems (pages 0- Problem In an experiment, a die is rolled continually until a 6 appears, at which point the experiment stops. What is the sample

### Chapter 6 Finite sets and infinite sets. Copyright 2013, 2005, 2001 Pearson Education, Inc. Section 3.1, Slide 1

Chapter 6 Finite sets and infinite sets Copyright 013, 005, 001 Pearson Education, Inc. Section 3.1, Slide 1 Section 6. PROPERTIES OF THE NATURE NUMBERS 013 Pearson Education, Inc.1 Slide Recall that denotes

### Math/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

### Combinatorics: The Fine Art of Counting

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

### Basic Probability Concepts

page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes

### Basic Probability. Probability: The part of Mathematics devoted to quantify uncertainty

AMS 5 PROBABILITY Basic Probability Probability: The part of Mathematics devoted to quantify uncertainty Frequency Theory Bayesian Theory Game: Playing Backgammon. The chance of getting (6,6) is 1/36.

### Finite Sets. Theorem 5.1. Two non-empty finite sets have the same cardinality if and only if they are equivalent.

MATH 337 Cardinality Dr. Neal, WKU We now shall prove that the rational numbers are a countable set while R is uncountable. This result shows that there are two different magnitudes of infinity. But we

### INTRODUCTION TO PROOFS: HOMEWORK SOLUTIONS

INTRODUCTION TO PROOFS: HOMEWORK SOLUTIONS STEVEN HEILMAN Contents 1. Homework 1 1 2. Homework 2 6 3. Homework 3 10 4. Homework 4 16 5. Homework 5 19 6. Homework 6 21 7. Homework 7 25 8. Homework 8 28

### For the purposes of this course, the natural numbers are the positive integers. We denote by N, the set of natural numbers.

Lecture 1: Induction and the Natural numbers Math 1a is a somewhat unusual course. It is a proof-based treatment of Calculus, for all of you who have already demonstrated a strong grounding in Calculus

### Probability Review. ICPSR Applied Bayesian Modeling

Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

### 33 Probability: Some Basic Terms

33 Probability: Some Basic Terms In this and the coming sections we discuss the fundamental concepts of probability at a level at which no previous exposure to the topic is assumed. Probability has been

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### (Refer Slide Time: 1:41)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture # 10 Sets Today we shall learn about sets. You must

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### Sets and functions. {x R : x > 0}.

Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### Probability. A random sample is selected in such a way that every different sample of size n has an equal chance of selection.

1 3.1 Sample Spaces and Tree Diagrams Probability This section introduces terminology and some techniques which will eventually lead us to the basic concept of the probability of an event. The Rare Event

### STAT 270 Probability Basics

STAT 270 Probability Basics Richard Lockhart Simon Fraser University Spring 2015 Surrey 1/28 Purposes of These Notes Jargon: experiment, sample space, outcome, event. Set theory ideas and notation: intersection,

### 6.042/18.062J Mathematics for Computer Science. Expected Value I

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

### Massachusetts Institute of Technology

n (i) m m (ii) n m ( (iii) n n n n (iv) m m Massachusetts Institute of Technology 6.0/6.: Probabilistic Systems Analysis (Quiz Solutions Spring 009) Question Multiple Choice Questions: CLEARLY circle the

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### Chapter 3. Cartesian Products and Relations. 3.1 Cartesian Products

Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing

### Worked examples Basic Concepts of Probability Theory

Worked examples Basic Concepts of Probability Theory Example 1 A regular tetrahedron is a body that has four faces and, if is tossed, the probability that it lands on any face is 1/4. Suppose that one

### + Section 6.2 and 6.3

Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities

### Notes on counting finite sets

Notes on counting finite sets Murray Eisenberg February 26, 2009 Contents 0 Introduction 2 1 What is a finite set? 2 2 Counting unions and cartesian products 4 2.1 Sum rules......................................

### JANUARY TERM 2012 FUN and GAMES WITH DISCRETE MATHEMATICS Module #9 (Geometric Series and Probability)

JANUARY TERM 2012 FUN and GAMES WITH DISCRETE MATHEMATICS Module #9 (Geometric Series and Probability) Author: Daniel Cooney Reviewer: Rachel Zax Last modified: January 4, 2012 Reading from Meyer, Mathematics

### Mathematical Induction

Mathematical Induction (Handout March 8, 01) The Principle of Mathematical Induction provides a means to prove infinitely many statements all at once The principle is logical rather than strictly mathematical,

### Chapter 1 LOGIC AND PROOF

Chapter 1 LOGIC AND PROOF To be able to understand mathematics and mathematical arguments, it is necessary to have a solid understanding of logic and the way in which known facts can be combined to prove

### Events. Independence. Coin Tossing. Random Phenomena

Random Phenomena Events A random phenomenon is a situation in which we know what outcomes could happen, but we don t know which particular outcome did or will happen For any random phenomenon, each attempt,

### Elements of probability theory

2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted

### Introduction to Discrete Probability. Terminology. Probability definition. 22c:19, section 6.x Hantao Zhang

Introduction to Discrete Probability 22c:19, section 6.x Hantao Zhang 1 Terminology Experiment A repeatable procedure that yields one of a given set of outcomes Rolling a die, for example Sample space

### Discrete Mathematics Lecture 5. Harper Langston New York University

Discrete Mathematics Lecture 5 Harper Langston New York University Empty Set S = {x R, x 2 = -1} X = {1, 3}, Y = {2, 4}, C = X Y (X and Y are disjoint) Empty set has no elements Empty set is a subset of

### INCIDENCE-BETWEENNESS GEOMETRY

INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full

### Week 2: Conditional Probability and Bayes formula

Week 2: Conditional Probability and Bayes formula We ask the following question: suppose we know that a certain event B has occurred. How does this impact the probability of some other A. This question

### Bayesian Tutorial (Sheet Updated 20 March)

Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that

### A Simple Example. Sample Space and Event. Tree Diagram. Tree Diagram. Probability. Probability - 1. Probability and Counting Rules

Probability and Counting Rules researcher claims that 10% of a large population have disease H. random sample of 100 people is taken from this population and examined. If 20 people in this random sample

### Chapter 4: Probabilities and Proportions

Stats 11 (Fall 2004) Lecture Note Introduction to Statistical Methods for Business and Economics Instructor: Hongquan Xu Chapter 4: Probabilities and Proportions Section 4.1 Introduction In the real world,

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence

### 3. Recurrence Recursive Definitions. To construct a recursively defined function:

3. RECURRENCE 10 3. Recurrence 3.1. Recursive Definitions. To construct a recursively defined function: 1. Initial Condition(s) (or basis): Prescribe initial value(s) of the function.. Recursion: Use a

### Discrete Mathematics & Mathematical Reasoning Chapter 6: Counting

Discrete Mathematics & Mathematical Reasoning Chapter 6: Counting Colin Stirling Informatics Slides originally by Kousha Etessami Colin Stirling (Informatics) Discrete Mathematics (Chapter 6) Today 1 /

### Mathematical Induction

Mathematical Induction Victor Adamchik Fall of 2005 Lecture 1 (out of three) Plan 1. The Principle of Mathematical Induction 2. Induction Examples The Principle of Mathematical Induction Suppose we have

### Sampling Distributions and the Central Limit Theorem

135 Part 2 / Basic Tools of Research: Sampling, Measurement, Distributions, and Descriptive Statistics Chapter 10 Sampling Distributions and the Central Limit Theorem In the previous chapter we explained

### Assigning Probabilities

What is a Probability? Probabilities are numbers between 0 and 1 that indicate the likelihood of an event. Generally, the statement that the probability of hitting a target- that is being fired at- is

### 6.2 Unions and Intersections

22 CHAPTER 6. PROBABILITY 6.2 Unions and Intersections The probability of a union of events Exercise 6.2- If you roll two dice, what is the probability of an even sum or a sum of 8 or more? Exercise 6.2-2

### Probability Life is full of uncertainty. Probability is the best way we currently have to quantify it.

Probability Life is full of uncertainty. Probability is the best way we currently have to quantify it. Probability Life is full of uncertainty. Probability is the best way we currently have to quantify

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 1

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 1 Course Outline CS70 is a course on Discrete Mathematics and Probability for EECS Students. The purpose of the course

### CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE

CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE With the notion of bijection at hand, it is easy to formalize the idea that two finite sets have the same number of elements: we just need to verify

### Lecture 4: Probability Spaces

EE5110: Probability Foundations for Electrical Engineers July-November 2015 Lecture 4: Probability Spaces Lecturer: Dr. Krishna Jagannathan Scribe: Jainam Doshi, Arjun Nadh and Ajay M 4.1 Introduction

### Basic concepts in probability. Sue Gordon

Mathematics Learning Centre Basic concepts in probability Sue Gordon c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Set Notation You may omit this section if you are

### Introduction to mathematical arguments

Introduction to mathematical arguments (background handout for courses requiring proofs) by Michael Hutchings A mathematical proof is an argument which convinces other people that something is true. Math

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Lecture 2 : Basics of Probability Theory

Lecture 2 : Basics of Probability Theory When an experiment is performed, the realization of the experiment is an outcome in the sample space. If the experiment is performed a number of times, different

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### 21-110: Problem Solving in Recreational Mathematics

-0: Problem Solving in Recreational Mathematics Homework assignment solutions Problem. An urn contains five red balls and three yellow balls. Two balls are drawn from the urn at random, without replacement.

### Induction and the Division Theorem 2.1 The Method of Induction

2 Induction and the Division Theorem 2.1 The Method of Induction In the Introduction we discussed a mathematical problem whose solution required the verification of an infinite family of statements. We

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### Lecture 3. Mathematical Induction

Lecture 3 Mathematical Induction Induction is a fundamental reasoning process in which general conclusion is based on particular cases It contrasts with deduction, the reasoning process in which conclusion

### The Casino Lab STATION 1: CRAPS

The Casino Lab Casinos rely on the laws of probability and expected values of random variables to guarantee them profits on a daily basis. Some individuals will walk away very wealthy, while others will

### 2.1.1 Examples of Sets and their Elements

Chapter 2 Set Theory 2.1 Sets The most basic object in Mathematics is called a set. As rudimentary as it is, the exact, formal definition of a set is highly complex. For our purposes, we will simply define

### Probabilities. Probability of a event. From Random Variables to Events. From Random Variables to Events. Probability Theory I

Victor Adamchi Danny Sleator Great Theoretical Ideas In Computer Science Probability Theory I CS 5-25 Spring 200 Lecture Feb. 6, 200 Carnegie Mellon University We will consider chance experiments with

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 5

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 5 Modular Arithmetic One way to think of modular arithmetic is that it limits numbers to a predefined range {0,1,...,N

### Test of Hypotheses. Since the Neyman-Pearson approach involves two statistical hypotheses, one has to decide which one

Test of Hypotheses Hypothesis, Test Statistic, and Rejection Region Imagine that you play a repeated Bernoulli game: you win \$1 if head and lose \$1 if tail. After 10 plays, you lost \$2 in net (4 heads

### 8 Divisibility and prime numbers

8 Divisibility and prime numbers 8.1 Divisibility In this short section we extend the concept of a multiple from the natural numbers to the integers. We also summarize several other terms that express

### Section 2.1. Tree Diagrams

Section 2.1 Tree Diagrams Example 2.1 Problem For the resistors of Example 1.16, we used A to denote the event that a randomly chosen resistor is within 50 Ω of the nominal value. This could mean acceptable.

### A Foundation for Geometry

MODULE 4 A Foundation for Geometry There is no royal road to geometry Euclid 1. Points and Lines We are ready (finally!) to talk about geometry. Our first task is to say something about points and lines.

### MAS113 Introduction to Probability and Statistics

MAS113 Introduction to Probability and Statistics 1 Introduction 1.1 Studying probability theory There are (at least) two ways to think about the study of probability theory: 1. Probability theory is a

### ORDERS OF ELEMENTS IN A GROUP

ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

### Introduction to Probability

3 Introduction to Probability Given a fair coin, what can we expect to be the frequency of tails in a sequence of 10 coin tosses? Tossing a coin is an example of a chance experiment, namely a process which