# Elements of probability theory

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted with certainty. 2.1 SAMPLE SPACES A situation whose outcomes occur randomly is called an experiment. The set of all possible outcomes of an experiment is called the sample space corresponding to the experiment, and is denoted by. A generic element of is called a sample point, or simply a point, and is denoted by! 2. Example 2.1 A coin is tossed twice and the sequence of heads (H) and tails (T) is recorded. The possible outcomes of this experiment are HH, HT, TH and TT. Hence, the sample space corresponding to this experiment consists of the four points = fhh;ht;th;ttg: A sample space is called nite if it is empty or contains a nite number of points, otherwise is called in nite. A sample space is called countable if its points can be indexed by the set of positive integers. A sample space that is nite or countable is called discrete. Example 2.2 A coin is tossed until H is recorded. The sample space corrsponding to this experiment is = fh;th;tth;ttth;tttth;:::g: Thus contains countably many points. 2 Not all sample spaces are discrete. For example, the sample space consisting of all positive real numbers is not discrete, neither is the sample space consisting of all real numbers in the interval [0; 1]. 2

2 8 2.2 RELATIONS AMONG EVENTS A subset of points in a sample space is called an event in. An event occurs if and only if one of its points occurs. Viewed as an event, is called the sure event. In general, events will be de ned by certain conditions on the points that compose them. Because events are just subsets of points in, concepts and results from point set theory apply to events. In particular, if A and B are events in, A implies B, written A µ B, if and only if all points in A also belong to B. The events A and B are identical, written A = B, if and only if A µ B and B µ A, that is, A and B contain exactly the same points. Other usual operations and relations between sets are listed below (union) A [ B = f! 2 :! 2 A or! 2 Bg, (intersection) A \ B = f! 2 :! 2 A and! 2 Bg, (complement) A c = f! 2 :! 62 Ag, (impossible event) ; = c, (di erence) A B = A \ B c, (symmetric di erence) A B = (A B) [ (B A). Instead of A \ B we also write AB. Operations and relations between sets are easy to visualize using Venn diagrams. If A \ B = ;, then A andb are called disjoint or mutually exclusive events. Intersections and unions of a countable collection A 1 ;A 2 ;A 3 ;::: of events are denoted by T 1 A i and S 1 A i, respectively. If A is an arbitrary family of subsets in, we write [ A2A A = f! 2 :! 2 A for some A 2 Ag and \ A2A A = f! 2 :! 2 A for all A 2 Ag Some basic relationships between events are: A [ ; = A, A [ =, A [ A = A, [ ; =, A \ ; = ;, A \ = A, A \ A = A, \ ; = ;. and

3 ELEMENTS OF PROBABILITY THEORY 9 (commutative law) A [ B = B [ A, A \ B = B \ A, (distributive law) A [ (B \ C) = (A [ B) \ (A [ C), A \ (B [ C) = (A \ B) [ (A \ C), (associative law) (A [ B) [ C = A [ (B [ C), (A \ B) \ C = A \ (B \ C), (De Morgan's laws) (A [ B) c = A c \ B c, (A \ B) c = A c [ B c. De Morgan's laws show that complementation, union and intersection are not independent operations. The commutative, distributive and associative laws and De Morgan's laws can easily be extended to a countable collection of events A 1 ;A 2 ;A 3 ;:::. The characteristic function (cf) of an event A µ is a function 1 A ( ) de ned for all! 2 by the relation ½ 1; if! 2 A, 1 A (!) = 0; otherwise. We also write 1(! 2 A) or, more compactly, 1(A). There is a one-to-one correspondence between sets and their cf's, and all properties of sets and set operations can be expressed in terms of cf's. For example, if C = A c then 1 C = 1 1 A, if C = A [ B then 1 C = max(1 A ; 1 B ), and if C = A \ B then 1 C = 1 A 1 B. 2.3 PROBABILITY How can we attach probabilities to events? The rst and easiest case is an experiment with a nite sample space consisting of N points. Suppose that, because of the nature of the experiment (e.g. tossing a fair coin), all points in are equiprobable, that is, equally likely, and let A be some event in. We de ne the probability of A, written P(A), as the ratio P(A) = N(A) N ; (2.1) where N(A) denotes the number of points in A. For any A µ, we have 0 P(A) 1; P( ) = N( ) N = 1; P(;) = N(;) N = 0: Further, if A and B are disjoint events in, then P(A [ B) = N(A [ B) N = N(A) N + N(B) N = P(A) + P(B):

4 10 Despite its simplicity, formula (2.1) can lead to non trivial calculations. In order to use it in a given problem, we need to determine: (i) the number N of all equiprobable outcomes, and (ii) the number of all those outcomes leading to the occurrence of A. A second case is whena basic experimentcan be repeated inexactly the same conditions any number n of times. We call this situation the case of independent trials under identical conditions. In this case, we can give a precise meaning to the concept of probability. In each trial a particular event A may or may not occur. Let n(a) be the number of trials in which A occurs. The relative frequency of the event A in the given series of n trials is de ned as f n (A) = n(a) n : It is an empirical fact that the f n (A) observed for di erent series of trials are virtually the same for large n, clustering about a constant value P(A), called the probability of A. Roughly speaking, the probability of A equals the fraction of trials leading to the occurrence of A in a large series of trials. 2.4 COMBINATORIAL RESULTS Whenever equal probabilities are assigned to the elements of a nite sample space, computation of probabilities of events reduces to counting the points comprising the events. Theorem 2.1 Given n elements a 1 ;:::;a n and m elements b 1 ;:::;b m there are exactly nm distinct ordered pairs (a i ;b j ) containing one element of each kind. Thus, if one experiment has n possible outcomes and another experiment has m possible outcomes, there are nm possible outcomes for the two expriments. More generally we have: Theorem 2.2 Given n 1 elements a 1 ;:::;a n1, n 2 elements b 1 ;:::;b n2, etc., up to n r elements x 1 ;:::;x nr, there are n 1 n 2 n r distinct ordered pairs (a i1 ;b i2 ;:::;x ir ) containing one element of each kind. Thus, if there are r experiments, where the rst has n 1 possible outcomes, the second n 2,..., and the rth n r possible outcomes, there are a total of n 1 n 2 n r possible outcomes for the r experiments. A permutation is an ordered arrangement of objects. An ordered sample of size r is a permutation of r objects obtained from a set ofn elements. Two possible ways for obtaining samples are: sampling with replacement and sampling without replacement. Notice that only samples of size r n without replacement are possible.

5 ELEMENTS OF PROBABILITY THEORY 11 Theorem 2.3 Given a set of n elements and sample size r, there are n r di erent ordered samples with replacement, and n(n 1)(n 2) (n r + 1) = di erent ordered samples without replacement. n! (n r)! Theorem 2.3 implies that the number of permutations or orderings of n elements is equal to n!. A combination is a set of elements without repetitions and without regard to ordering. For example, fa; bg and fb; ag are di erent permutations but only one combination. Thus, a combination is the number of unordered samples of a given size drawn without replacement from a nite set of objects. Theorem 2.4 The number of possible combinations of n objects taken r at a time (r n), is equal to µ Cr n n! n = r!(n r)! = : r Proof. Since the number of ordered samples is equal to the number of unordered samples times the number of ways to order each sample we have that n! (n r)! = Cn r r!; from which C n r = n! r!(n r)! : 2 The number Cr n is called binomial coe±cient, since it occurs in the binomial expansion nx µ n (a + b) n = a n r b r : r r=0 More generally, simple induction gives the following: Theorem 2.5 Given a set of n elements, let n 1 ;:::;n k be positive integers such that P k n i = n. Then there are µ n n! = (2.2) n 1 n 2 :::n k n 1!n 2!:::n k! ways of partitioning the set into k unorderd samples without replacement of size n 1 ;:::;n k respectively. The numbers (2.2) are called multinomial coe±cients.

6 FINITE PROBABILITY SPACES The de nition of probability in terms of equiprobable events is circular. On the other hand, de ning probabilities as limits of relative frequencies inindependent trials under identical conditions is far too restrictive. To avoid these problems we shall now present a purely axiomatic treatment of probabilities. De nition 2.1 A sample space is called a nite probability space if is nite and for every event A µ, there is de ned a real number P(A), called the probability of the event A, such that: A.1: P(A) 0; A.2: P( ) = 1; A.3: if A 1 and A 2 are mutually exclusive events in, then P(A [ B) = P(A) + P(B): It follows from De nition 2.1 that, for any subset A and B of, 0 P(A) 1; (2.3) 2 Further P(A c ) = 1 P(A); (2.4) P(;) = 0; (2.5) A µ B ) P(A) P(B): (2.6) P(A [ B) = P(A) + P(B) P(AB) (Covering theorem): (2.7) This implies the following upper bound on P(A [ B) P(A [ B) P(A) + P(B); with equality if and only if A and B are disjoint. Notice that B = AB[A c B, where AB and A c B are mutually exclusive events. Hence P(B) = P(AB) + P(A c B) and therefore P(B) P(AB) = P(A c B). Substituting in (2.7) gives P(A [ B) = P(A) + P(A c B) (Addition law): (2.8) Also notice that, by De Morgan's law and the Covering theorem, This implies 1 P(AB) = P((AB) c ) = P(A c [ B c ) P(A c ) + P(B c ): P(AB) 1 P(A c ) P(B c ) (Bonferroni inequality); (2.9)

7 ELEMENTS OF PROBABILITY THEORY 13 with equality if and only if A c and B c are disjoint. More generally, if A 1 ;:::;A n is a nite collection of events in, then A 1 ;A c 1A 2 ;:::;A c 1A c 2;:::;A c n 1A n form a partition of S n A i, and so n[ P( A i ) = P(A 1 ) + P(A c 1A 2 ) + + P(A c 1A c 2 A c n 1A n ): This result generalizes the Addition law (2.8). Since for all n 1, we also have A c 1A c 2 A c n 1A n µ A n n[ P( A i ) nx P(A i ): This result generalizes the Covering theorem (2.7). Finally, the generalization of the Bonferroni inequality (2.9) is n\ P( A i ) 1 nx P(A c i): 2.6 MEASURABLE SPACES AND MEASURES For in nite sample spaces, some modi cations of the axioms A.1-A.3 and some additional concepts of set theory are required. The reason is that some subsets of an in nite sample space may be so irrregular that it is not possible to assign a probability to them. A set whose elements are sets of will be called a class of sets in. When a set operation performed on sets in a class A gives as a result sets which also belong to A, we say that A is closed under the given operation. De nition 2.2 A nonempty class A of subsets of is called a eld or an algebra on if it contains and is closed under complementation and nite unions, that is, 2 A; A 2 A ) A c 2 A; (2.10) n[ A i 2 A; i = 1;:::;n ) A i 2 A: (2.11) By De Morgan's laws, (2.10) and (2.11) together imply A i 2 A; i = 1;:::;n ) n\ A i 2 A: 2

8 14 Thus, all standardset operations (union, intersection andcomplementation) can be performed any nite number of times on the elements of a eld A without obtaining a set not in A. De nition 2.3 A eld A on is a ¾- eld or a ¾-algebra if it is closed under countable unions, that is, if A i 2 A; i = 1; 2;::: ) A i 2 A: (2.12) By De Morgan's law, (2.10) and (2.12) together imply A i 2 A; i = 1; 2;::: ) 1\ A i 2 A: Thus, all standard set operations can be performed any countable number of times on the elements of a ¾- eld A without obtaining a set not in A. If A is a class of subsets of, the smallest eld (¾- eld) containing A is called the eld (¾- eld) generated by A. It can be veri ed that the eld (¾- eld) generated by A is equal to the intersection of all eld (¾- elds) containing A. If A is a ¾- eld on, the pair ( ;A) is called a measurable space. A subset A of is said to be measurable if A 2 A. Given a space, it is generally possible to de ne many ¾- elds on. To distinguish between them, the members of a given ¾- eld A on will be called A-measurable sets. Example 2.3 An important ¾- eld on the real line < is that generated by the class of all bounded semi-closed intervals of the form (a;b], 1 < a < b < 1. This ¾- eld is called the Borel eld on < and denoted by B. Its elements are called the Borel sets. Since B is a ¾- eld, repeated nite and countable set theoretic operations on its elements will never lead outside B. The measurable space (<;B) is called the Borel line. Notice that B would equivalently be generated by all the open half-lines of <, all the open intervals of <, or all the closed intervals of <. 2 A set function is a function de ned on a class of sets. De nition 2.4 A measure ¹ on a measurable space ( ;A) is a nonnegative set function¹de ned for all sets ofa and satisfying: M.1: ¹(;) = 0; M.2: (Countable additivity) if fa i g is any countable sequence of disjoint A- measurable sets, then ¹( A i ) = nx ¹(A i ): 2

9 ELEMENTS OF PROBABILITY THEORY 15 Clearly, countable additivity implies nite additivity, that is, if A 1 ;:::;A n is a nite collection of disjoint measurable sets, then n[ ¹( A i ) = nx ¹(A i ): Example 2.4 Let f be a nonnegative function of the points of a set. Let the ¾- eld A consist of all countable subsets of. A measure ¹ on ( ;A) is then de ned as nx ¹(;) = 0; ¹(f! 1 ;:::;! n g) = f(! i ): If f = 1, then ¹ is called counting measure. 2 It is easy to verify that if ¹ is a measure on ( ;A), then it is monotone, that is, ¹(A) ¹(B) whenever A;B 2 A and A ½ B. De nition 2.5 A measure ¹ on ( ;A) is called nite if ¹( ) < 1. It is called ¾- nite if there exists a sequence fa i g of sets in A such that S 1 A i = and ¹(A i ) < 1, n = 1; 2;:::. 2 Example 2.5 An important ¾- nite measure is the one de ned on the Borel line (<;B) by ¹((a;b]) = b a, the length of the interval (a;b]. Such a measure is called Lebesgue measure. It is easy to verify that every countable set is a Borel set of measure zero. 2 De nition 2.6 If ¹ is a measure on ( ;A), the triple ( ;A;¹) is called a measure space. 2 A measure space ( ;A;¹) is called complete if it contains all subsets of sets of measure zero, that is, if A 2 A, B ½ A, and ¹(A) = 0, then B 2 A. It can be shown that each measure space can be completed by the addition of subsets of sets of measure zero. If ¹ is a ¾- nite measure de ned on ( ;A) and F(A) is the ¾- eld generated by A, then it can be shown that there exists a unique measure ¹ on ( ;F(A)) such that ¹ (A) = ¹(A) for all A 2 A. Further, ¹ is also ¾- nite. Such a measure is called the extension of ¹. De nition 2.7 A measure space ( ;A;P) is a probability space if P is a ¾- nite measure with P( ) = PROBABILITY SPACES From De nition 2.7, a probability space is a triple ( ;A;P), where is the sample space associated with an experiment, A is a ¾- eld on, and the probability measure P is a real valued function de ned for all sets in A and satisfying:

10 16 P.1: P(A) 0 for all A 2 A; P.2: P( ) = 1; P.3: (Countable additivity): If fa i g is a countable sequence of disjoint subsets in A, then nx P( A i ) = P(A i ): If ( ;A;P) is a probability space, then the sets in A are interpreted as possible events associated with an experiment. For any A 2 A, the real number P(A) is called the probability of the event A. A support of P is any set A 2 A for which P(A) = 1. If is a nite sample space and A is the set of all the events in (the collection of all subsets of ), then properties P.1{P.3 are equivalent to A.1{A.3 that de ne a nite probability space. As a consequence of properties P.1{P.3, relationships (2.3){(2.9) hold for any A; B 2 A. Further, their generalizations hold for any nite collection of events in A. Notice that the Covering theorem (2.7) can be shown to hold for any countable collection of events in A. Further, if fa i g is a countable collection of events in A such that A 1 µ A 2 µ, then P( A i ) = lim n!1 P(A i): 2.8 CONDITIONAL PROBABILITY Let ( ;A;P) be a probability space and let B 2 A be an event such that P(B) > 0. If we know that B occurred, then the relevant sample space becomes B rather than. This justi es de ning the conditional probability of A given B as P(AjB) = P(AB) (2.13) P(B) if P(B) > 0, and P(A jb) = 0 if P(B) = 0. It is easy to verify that the function P( jb) de ned on A is a probability measure on ( ;A), that is, it satis es P.1{P.3. We call P( jb) the conditional probability measure given B. Notice that (2.13) can equivalently be written as P(AB) = P(AjB)P(B): This result, called the Multiplication law, provides a convenient way of nding P(AB) whenever P(AjB) and P(B) are easy to nd. The Multiplication law can be generalized to a nite collection of events A 1 ;:::;A n in A P(A 1 A n ) = P(A n ja n 1 A 1 )P(A n 1 A 1 ) = P(A n ja n 1 A 1 )P(A n 1 ja n 2 A 1 )P(A n 2 A 1 );

11 ELEMENTS OF PROBABILITY THEORY 17 and so on. Thus P(A 1 A n ) = P(A 1 )P(A 2 ja 1 )P(A 3 ja 2 A 1 ) P(A n ja n 1 A 1 ): Now consider a countable collection fb i g of disjoint events in A such that P(B i ) > 0 for every i and S 1 B i =. Clearly, For any A 2 A, where we used the fact that 1X P( B i ) = P(B i ) = 1: P(A) = P(A \ ( B i )) + P(A \ ( B i ) c ) = P(A \ ( B i )); P(( B i ) c ) = 1 P( B i ) = 0: Thus, by the Morgan's laws, 1X P(A) = P( AB i ) = P(AB i ); since AB i \ AB j = ; for all i 6= j. Therefore P(A) = 1X P(AjB i )P(B i ); which is called the Law of total probabilities. Now let A 2 A be such that P(A) > 0, and consider computing the conditional probability P(B j ja) given knowledge of fp(ajb i )g and fp(b i )g. By the de nition of conditional probability and the Multiplication law, P(B j ja) = P(B ja) P(A) = P(AjB j)p(b j ) P(A) for any xed j = 1; 2;:::. Therefore, by the Law of total probabilities, P(B j ja) = P(A jb j) P(B j ) P 1 P(AjB i)p(b i ) ; which is called Bayes rule.

12 INDEPENDENCE Let A;B 2 A be two events with non-zero probability. If knowing that B occurred gives no information about whether or not A occurred, then the probability assigned to A should not be modi ed by the knowledge that B occurred. Hence P(AjB) = P(A); and so P(AB) = P(A)P(B): (2.14) Two events A;B 2 A are said to be (pairwise) independent if (2.14) holds. Notice that this de nition of independence is symmetric in A and B, and also covers the case when P(A) = 0 or P(B) = 0. It is easy to show that if A and B are independent, then A and B c as well as A c and B c are independent. Three events A;B;C 2 A are said to be (mutually) independent if they are pairwise independent and P(ABC) = P(A)P(B) P(C): This condition is necessary, for pairwise independence does not ensure that, for example, P((AB)C) = P(AB)P(C). It is easy to verify that if A;B and C are independent events, then A[B and C are independent, and A \ B and C are independent. More generally, a family A of events are (mutually) independent if, for every nite collection A 1 ;:::;A n of events in A, REFERENCES n\ P( A i ) = ny P(A i ): (2.15) Billingsley P. (1979) Probability and Measure, Wiley, New York. Feller, W. (1968) An Introduction to Probability Theory and Its Applications (3rd ed.), Vol. 1, Wiley, New York. Halmos, P.R. (1974)Measure Theory, Springer, New York. Kolmogorov, A.N. and Fomin S.V. (1970) Introductory Real Analysis, Dover, New York. Loµeve, M. (1977)Probability Theory (4th ed.), Vol. 1, Springer, New York. Royden H.L. (1968)Real Analysis (2nd ed.), MacMillan, New York.

### Introduction to Probability

3 Introduction to Probability Given a fair coin, what can we expect to be the frequency of tails in a sequence of 10 coin tosses? Tossing a coin is an example of a chance experiment, namely a process which

### Sigma Field Notes for Economics 770 : Econometric Theory

1 Sigma Field Notes for Economics 770 : Econometric Theory Jonathan B. Hill Dept. of Economics, University of North Carolina 0. DEFINITIONS AND CONVENTIONS The following is a small list of repeatedly used

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Basic Probability Concepts

page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### This chapter describes set theory, a mathematical theory that underlies all of modern mathematics.

Appendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern mathematics. A.1 Basic Definitions Definition A.1.1. A set is an unordered collection of elements.

### Lecture 2 : Basics of Probability Theory

Lecture 2 : Basics of Probability Theory When an experiment is performed, the realization of the experiment is an outcome in the sample space. If the experiment is performed a number of times, different

### Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University

Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments

### Discrete Mathematics Lecture 5. Harper Langston New York University

Discrete Mathematics Lecture 5 Harper Langston New York University Empty Set S = {x R, x 2 = -1} X = {1, 3}, Y = {2, 4}, C = X Y (X and Y are disjoint) Empty set has no elements Empty set is a subset of

### Math 507/420 Homework Assignment #1: Due in class on Friday, September 20. SOLUTIONS

Math 507/420 Homework Assignment #1: Due in class on Friday, September 20. SOLUTIONS 1. Show that a nonempty collection A of subsets is an algebra iff 1) for all A, B A, A B A and 2) for all A A, A c A.

### Probability - Part I. Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty.

Probability - Part I Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty. Definition : The sample space (denoted S) of a random experiment

### Basics of Probability

Basics of Probability August 27 and September 1, 2009 1 Introduction A phenomena is called random if the exact outcome is uncertain. The mathematical study of randomness is called the theory of probability.

### Chapter 3: The basic concepts of probability

Chapter 3: The basic concepts of probability Experiment: a measurement process that produces quantifiable results (e.g. throwing two dice, dealing cards, at poker, measuring heights of people, recording

### STAT 270 Probability Basics

STAT 270 Probability Basics Richard Lockhart Simon Fraser University Spring 2015 Surrey 1/28 Purposes of These Notes Jargon: experiment, sample space, outcome, event. Set theory ideas and notation: intersection,

### Measure Theory. Jesus Fernandez-Villaverde University of Pennsylvania

Measure Theory Jesus Fernandez-Villaverde University of Pennsylvania 1 Why Bother with Measure Theory? Kolmogorov (1933). Foundation of modern probability. Deals easily with: 1. Continuous versus discrete

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Chap2: The Real Number System (See Royden pp40)

Chap2: The Real Number System (See Royden pp40) 1 Open and Closed Sets of Real Numbers The simplest sets of real numbers are the intervals. We define the open interval (a, b) to be the set (a, b) = {x

### IAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION

IAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION 1 WHAT IS STATISTICS? Statistics is a science of collecting data, organizing and describing it and drawing conclusions from it. That is, statistics

### Discrete Mathematics & Mathematical Reasoning Chapter 6: Counting

Discrete Mathematics & Mathematical Reasoning Chapter 6: Counting Colin Stirling Informatics Slides originally by Kousha Etessami Colin Stirling (Informatics) Discrete Mathematics (Chapter 6) Today 1 /

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### Math212a1010 Lebesgue measure.

Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.

### Chapter 1. Probability Theory: Introduction. Definitions Algebra. RS Chapter 1 Random Variables

Chapter 1 Probability Theory: Introduction Definitions Algebra Definitions: Semiring A collection of sets F is called a semiring if it satisfies: Φ F. If A, B F, then A B F. If A, B F, then there exists

### Finite Sets. Theorem 5.1. Two non-empty finite sets have the same cardinality if and only if they are equivalent.

MATH 337 Cardinality Dr. Neal, WKU We now shall prove that the rational numbers are a countable set while R is uncountable. This result shows that there are two different magnitudes of infinity. But we

### Lecture 2: Probability

Lecture 2: Probability Assist. Prof. Dr. Emel YAVUZ DUMAN MCB1007 Introduction to Probability and Statistics İstanbul Kültür University Outline 1 Introduction 2 Sample Spaces 3 Event 4 The Probability

### Basic concepts in probability. Sue Gordon

Mathematics Learning Centre Basic concepts in probability Sue Gordon c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Set Notation You may omit this section if you are

### Math/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### NOTES ON MEASURE THEORY. M. Papadimitrakis Department of Mathematics University of Crete. Autumn of 2004

NOTES ON MEASURE THEORY M. Papadimitrakis Department of Mathematics University of Crete Autumn of 2004 2 Contents 1 σ-algebras 7 1.1 σ-algebras............................... 7 1.2 Generated σ-algebras.........................

### Random Variables and Measurable Functions.

Chapter 3 Random Variables and Measurable Functions. 3.1 Measurability Definition 42 (Measurable function) Let f be a function from a measurable space (Ω, F) into the real numbers. We say that the function

### LECTURE NOTES IN MEASURE THEORY. Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12)

1 LECTURE NOTES IN MEASURE THEORY Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12) 2 PREFACE These are lecture notes on integration theory for a eight-week

### PROBABILITY NOTIONS. Summary. 1. Random experiment

PROBABILITY NOTIONS Summary 1. Random experiment... 1 2. Sample space... 2 3. Event... 2 4. Probability calculation... 3 4.1. Fundamental sample space... 3 4.2. Calculation of probability... 3 4.3. Non

### Assigning Probabilities

What is a Probability? Probabilities are numbers between 0 and 1 that indicate the likelihood of an event. Generally, the statement that the probability of hitting a target- that is being fired at- is

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### Notes on counting finite sets

Notes on counting finite sets Murray Eisenberg February 26, 2009 Contents 0 Introduction 2 1 What is a finite set? 2 2 Counting unions and cartesian products 4 2.1 Sum rules......................................

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### Lesson 3 Chapter 2: Introduction to Probability

Lesson 3 Chapter 2: Introduction to Probability Department of Statistics The Pennsylvania State University 1 2 The Probability Mass Function and Probability Sampling Counting Techniques 3 4 The Law of

### Chapter 1. Sigma-Algebras. 1.1 Definition

Chapter 1 Sigma-Algebras 1.1 Definition Consider a set X. A σ algebra F of subsets of X is a collection F of subsets of X satisfying the following conditions: (a) F (b) if B F then its complement B c is

### Sections 2.1, 2.2 and 2.4

SETS Sections 2.1, 2.2 and 2.4 Chapter Summary Sets The Language of Sets Set Operations Set Identities Introduction Sets are one of the basic building blocks for the types of objects considered in discrete

### 5. Probability Calculus

5. Probability Calculus So far we have concentrated on descriptive statistics (deskriptiivinen eli kuvaileva tilastotiede), that is methods for organizing and summarizing data. As was already indicated

### The set consisting of all natural numbers that are in A and are in B is the set f1; 3; 5g;

Chapter 5 Set Theory 5.1 Sets and Operations on Sets Preview Activity 1 (Set Operations) Before beginning this section, it would be a good idea to review sets and set notation, including the roster method

### Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics

Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights

### Discrete Mathematics

Discrete Mathematics Chih-Wei Yi Dept. of Computer Science National Chiao Tung University March 16, 2009 2.1 Sets 2.1 Sets 2.1 Sets Basic Notations for Sets For sets, we ll use variables S, T, U,. We can

### CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE

CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE With the notion of bijection at hand, it is easy to formalize the idea that two finite sets have the same number of elements: we just need to verify

### Statistics in Geophysics: Introduction and Probability Theory

Statistics in Geophysics: Introduction and Steffen Unkel Department of Statistics Ludwig-Maximilians-University Munich, Germany Winter Term 2013/14 1/32 What is Statistics? Introduction Statistics is the

### Set theory, and set operations

1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Extension of measure

1 Extension of measure Sayan Mukherjee Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard.

### k, then n = p2α 1 1 pα k

Powers of Integers An integer n is a perfect square if n = m for some integer m. Taking into account the prime factorization, if m = p α 1 1 pα k k, then n = pα 1 1 p α k k. That is, n is a perfect square

### Probability and Statistics Vocabulary List (Definitions for Middle School Teachers)

Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..

Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,

### STAT 571 Assignment 1 solutions

STAT 571 Assignment 1 solutions 1. If Ω is a set and C a collection of subsets of Ω, let A be the intersection of all σ-algebras that contain C. rove that A is the σ-algebra generated by C. Solution: Let

### Events. Independence. Coin Tossing. Random Phenomena

Random Phenomena Events A random phenomenon is a situation in which we know what outcomes could happen, but we don t know which particular outcome did or will happen For any random phenomenon, each attempt,

### Module 6: Basic Counting

Module 6: Basic Counting Theme 1: Basic Counting Principle We start with two basic counting principles, namely, the sum rule and the multiplication rule. The Sum Rule: If there are n 1 different objects

### So let us begin our quest to find the holy grail of real analysis.

1 Section 5.2 The Complete Ordered Field: Purpose of Section We present an axiomatic description of the real numbers as a complete ordered field. The axioms which describe the arithmetic of the real numbers

### 33 Probability: Some Basic Terms

33 Probability: Some Basic Terms In this and the coming sections we discuss the fundamental concepts of probability at a level at which no previous exposure to the topic is assumed. Probability has been

### Chapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum a-posteriori Hypotheses

Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum a-posteriori Hypotheses ML:IV-1 Statistical Learning STEIN 2005-2015 Area Overview Mathematics Statistics...... Stochastics

### Chapter 1. Open Sets, Closed Sets, and Borel Sets

1.4. Borel Sets 1 Chapter 1. Open Sets, Closed Sets, and Borel Sets Section 1.4. Borel Sets Note. Recall that a set of real numbers is open if and only if it is a countable disjoint union of open intervals.

### Problem Set. Problem Set #2. Math 5322, Fall December 3, 2001 ANSWERS

Problem Set Problem Set #2 Math 5322, Fall 2001 December 3, 2001 ANSWERS i Problem 1. [Problem 18, page 32] Let A P(X) be an algebra, A σ the collection of countable unions of sets in A, and A σδ the collection

### Probability and Counting

Probability and Counting Basic Counting Principles Permutations and Combinations Sample Spaces, Events, Probability Union, Intersection, Complements; Odds Conditional Probability, Independence Bayes Formula

### Sets, Relations and Functions

Sets, Relations and Functions Eric Pacuit Department of Philosophy University of Maryland, College Park pacuit.org epacuit@umd.edu ugust 26, 2014 These notes provide a very brief background in discrete

### Question: What is the probability that a five-card poker hand contains a flush, that is, five cards of the same suit?

ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the

### Structure of Measurable Sets

Structure of Measurable Sets In these notes we discuss the structure of Lebesgue measurable subsets of R from several different points of view. Along the way, we will see several alternative characterizations

### Solutions to Tutorial 2 (Week 3)

THE UIVERSITY OF SYDEY SCHOOL OF MATHEMATICS AD STATISTICS Solutions to Tutorial 2 (Week 3) MATH3969: Measure Theory and Fourier Analysis (Advanced) Semester 2, 2016 Web Page: http://sydney.edu.au/science/maths/u/ug/sm/math3969/

### POSITIVE INTEGERS, INTEGERS AND RATIONAL NUMBERS OBTAINED FROM THE AXIOMS OF THE REAL NUMBER SYSTEM

MAT 1011 TECHNICAL ENGLISH I 03.11.2016 Dokuz Eylül University Faculty of Science Department of Mathematics Instructor: Engin Mermut Course assistant: Zübeyir Türkoğlu web: http://kisi.deu.edu.tr/engin.mermut/

### Chapter 4. Probability and Probability Distributions

Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

### Sets. A set is a collection of (mathematical) objects, with the collection treated as a single mathematical object.

Sets 1 Sets Informally: A set is a collection of (mathematical) objects, with the collection treated as a single mathematical object. Examples: real numbers, complex numbers, C integers, All students in

### REAL ANALYSIS I HOMEWORK 2

REAL ANALYSIS I HOMEWORK 2 CİHAN BAHRAN The questions are from Stein and Shakarchi s text, Chapter 1. 1. Prove that the Cantor set C constructed in the text is totally disconnected and perfect. In other

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### Measure Theory and Probability

Chapter 2 Measure Theory and Probability 2.1 Introduction In advanced probability courses, a random variable (r.v.) is defined rigorously through measure theory, which is an in-depth mathematics discipline

### A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes.

Chapter 7 Probability 7.1 Experiments, Sample Spaces, and Events A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes. Each outcome

### Combinatorial Proofs

Combinatorial Proofs Two Counting Principles Some proofs concerning finite sets involve counting the number of elements of the sets, so we will look at the basics of counting. Addition Principle: If A

### Sets and set operations

CS 441 Discrete Mathematics for CS Lecture 7 Sets and set operations Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square asic discrete structures Discrete math = study of the discrete structures used

### Logic & Discrete Math in Software Engineering (CAS 701) Dr. Borzoo Bonakdarpour

Logic & Discrete Math in Software Engineering (CAS 701) Background Dr. Borzoo Bonakdarpour Department of Computing and Software McMaster University Dr. Borzoo Bonakdarpour Logic & Discrete Math in SE (CAS

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### AXIOMATIC PROBABILITY AND POINT SETS

AXIOMATIC PROBABILITY AND POINT SETS The axioms of Kolmogorov. Let S denote a sample space with a probability measure P defined over it, such that probability of any event A S is given by P (A). Then,

### Math 117 Chapter 7 Sets and Probability

Math 117 Chapter 7 and Probability Flathead Valley Community College Page 1 of 15 1. A set is a well-defined collection of specific objects. Each item in the set is called an element or a member. Curly

### + Section 6.2 and 6.3

Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities

### MA651 Topology. Lecture 6. Separation Axioms.

MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples

### PART I. THE REAL NUMBERS

PART I. THE REAL NUMBERS This material assumes that you are already familiar with the real number system and the representation of the real numbers as points on the real line. I.1. THE NATURAL NUMBERS

### 6.3 Conditional Probability and Independence

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

### Probability OPRE 6301

Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.

### Definition of Random Variable A random variable is a function from a sample space S into the real numbers.

.4 Random Variable Motivation example In an opinion poll, we might decide to ask 50 people whether they agree or disagree with a certain issue. If we record a for agree and 0 for disagree, the sample space

### A set is a Many that allows itself to be thought of as a One. (Georg Cantor)

Chapter 4 Set Theory A set is a Many that allows itself to be thought of as a One. (Georg Cantor) In the previous chapters, we have often encountered sets, for example, prime numbers form a set, domains

### Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab

Lecture 2 Probability BMIR Lecture Series in Probability and Statistics Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab 2.1 1 Sample Spaces and Events Random

### Sets and Cardinality Notes for C. F. Miller

Sets and Cardinality Notes for 620-111 C. F. Miller Semester 1, 2000 Abstract These lecture notes were compiled in the Department of Mathematics and Statistics in the University of Melbourne for the use

### I. WHAT IS PROBABILITY?

C HAPTER 3 PROBABILITY Random Experiments I. WHAT IS PROBABILITY? The weatherman on 0 o clock news program states that there is a 20% chance that it will snow tomorrow, a 65% chance that it will rain and

### Sets, Venn Diagrams & Counting

MT 142 College Mathematics Sets, Venn Diagrams & Counting Module SC Terri Miller revised January 5, 2011 What is a set? Sets set is a collection of objects. The objects in the set are called elements of

### A1. Basic Reviews PERMUTATIONS and COMBINATIONS... or HOW TO COUNT

A1. Basic Reviews Appendix / A1. Basic Reviews / Perms & Combos-1 PERMUTATIONS and COMBINATIONS... or HOW TO COUNT Question 1: Suppose we wish to arrange n 5 people {a, b, c, d, e}, standing side by side,

### Statistical Inference. Prof. Kate Calder. If the coin is fair (chance of heads = chance of tails) then

Probability Statistical Inference Question: How often would this method give the correct answer if I used it many times? Answer: Use laws of probability. 1 Example: Tossing a coin If the coin is fair (chance

### 4. Joint Distributions

Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose

### Elements of probability theory

The role of probability theory in statistics We collect data so as to provide evidentiary support for answers we give to our many questions about the world (and in our particular case, about the business

### Notes on Chapter 1, Section 2 Arithmetic and Divisibility

Notes on Chapter 1, Section 2 Arithmetic and Divisibility August 16, 2006 1 Arithmetic Properties of the Integers Recall that the set of integers is the set Z = f0; 1; 1; 2; 2; 3; 3; : : :g. The integers

### Sets and functions. {x R : x > 0}.

Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.

### God created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886)

Chapter 2 Numbers God created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886) God created the integers and the rest is the work

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics.

Elementary probability theory The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics. 1.1 Experiments, outcomes and sample

### ) + ˆf (n) sin( 2πnt. = 2 u x 2, t > 0, 0 < x < 1. u(0, t) = u(1, t) = 0, t 0. (x, 0) = 0 0 < x < 1.

Introduction to Fourier analysis This semester, we re going to study various aspects of Fourier analysis. In particular, we ll spend some time reviewing and strengthening the results from Math 425 on Fourier