Elements of probability theory


 Melvin Strickland
 2 years ago
 Views:
Transcription
1 2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted with certainty. 2.1 SAMPLE SPACES A situation whose outcomes occur randomly is called an experiment. The set of all possible outcomes of an experiment is called the sample space corresponding to the experiment, and is denoted by. A generic element of is called a sample point, or simply a point, and is denoted by! 2. Example 2.1 A coin is tossed twice and the sequence of heads (H) and tails (T) is recorded. The possible outcomes of this experiment are HH, HT, TH and TT. Hence, the sample space corresponding to this experiment consists of the four points = fhh;ht;th;ttg: A sample space is called nite if it is empty or contains a nite number of points, otherwise is called in nite. A sample space is called countable if its points can be indexed by the set of positive integers. A sample space that is nite or countable is called discrete. Example 2.2 A coin is tossed until H is recorded. The sample space corrsponding to this experiment is = fh;th;tth;ttth;tttth;:::g: Thus contains countably many points. 2 Not all sample spaces are discrete. For example, the sample space consisting of all positive real numbers is not discrete, neither is the sample space consisting of all real numbers in the interval [0; 1]. 2
2 8 2.2 RELATIONS AMONG EVENTS A subset of points in a sample space is called an event in. An event occurs if and only if one of its points occurs. Viewed as an event, is called the sure event. In general, events will be de ned by certain conditions on the points that compose them. Because events are just subsets of points in, concepts and results from point set theory apply to events. In particular, if A and B are events in, A implies B, written A µ B, if and only if all points in A also belong to B. The events A and B are identical, written A = B, if and only if A µ B and B µ A, that is, A and B contain exactly the same points. Other usual operations and relations between sets are listed below (union) A [ B = f! 2 :! 2 A or! 2 Bg, (intersection) A \ B = f! 2 :! 2 A and! 2 Bg, (complement) A c = f! 2 :! 62 Ag, (impossible event) ; = c, (di erence) A B = A \ B c, (symmetric di erence) A B = (A B) [ (B A). Instead of A \ B we also write AB. Operations and relations between sets are easy to visualize using Venn diagrams. If A \ B = ;, then A andb are called disjoint or mutually exclusive events. Intersections and unions of a countable collection A 1 ;A 2 ;A 3 ;::: of events are denoted by T 1 A i and S 1 A i, respectively. If A is an arbitrary family of subsets in, we write [ A2A A = f! 2 :! 2 A for some A 2 Ag and \ A2A A = f! 2 :! 2 A for all A 2 Ag Some basic relationships between events are: A [ ; = A, A [ =, A [ A = A, [ ; =, A \ ; = ;, A \ = A, A \ A = A, \ ; = ;. and
3 ELEMENTS OF PROBABILITY THEORY 9 (commutative law) A [ B = B [ A, A \ B = B \ A, (distributive law) A [ (B \ C) = (A [ B) \ (A [ C), A \ (B [ C) = (A \ B) [ (A \ C), (associative law) (A [ B) [ C = A [ (B [ C), (A \ B) \ C = A \ (B \ C), (De Morgan's laws) (A [ B) c = A c \ B c, (A \ B) c = A c [ B c. De Morgan's laws show that complementation, union and intersection are not independent operations. The commutative, distributive and associative laws and De Morgan's laws can easily be extended to a countable collection of events A 1 ;A 2 ;A 3 ;:::. The characteristic function (cf) of an event A µ is a function 1 A ( ) de ned for all! 2 by the relation ½ 1; if! 2 A, 1 A (!) = 0; otherwise. We also write 1(! 2 A) or, more compactly, 1(A). There is a onetoone correspondence between sets and their cf's, and all properties of sets and set operations can be expressed in terms of cf's. For example, if C = A c then 1 C = 1 1 A, if C = A [ B then 1 C = max(1 A ; 1 B ), and if C = A \ B then 1 C = 1 A 1 B. 2.3 PROBABILITY How can we attach probabilities to events? The rst and easiest case is an experiment with a nite sample space consisting of N points. Suppose that, because of the nature of the experiment (e.g. tossing a fair coin), all points in are equiprobable, that is, equally likely, and let A be some event in. We de ne the probability of A, written P(A), as the ratio P(A) = N(A) N ; (2.1) where N(A) denotes the number of points in A. For any A µ, we have 0 P(A) 1; P( ) = N( ) N = 1; P(;) = N(;) N = 0: Further, if A and B are disjoint events in, then P(A [ B) = N(A [ B) N = N(A) N + N(B) N = P(A) + P(B):
4 10 Despite its simplicity, formula (2.1) can lead to non trivial calculations. In order to use it in a given problem, we need to determine: (i) the number N of all equiprobable outcomes, and (ii) the number of all those outcomes leading to the occurrence of A. A second case is whena basic experimentcan be repeated inexactly the same conditions any number n of times. We call this situation the case of independent trials under identical conditions. In this case, we can give a precise meaning to the concept of probability. In each trial a particular event A may or may not occur. Let n(a) be the number of trials in which A occurs. The relative frequency of the event A in the given series of n trials is de ned as f n (A) = n(a) n : It is an empirical fact that the f n (A) observed for di erent series of trials are virtually the same for large n, clustering about a constant value P(A), called the probability of A. Roughly speaking, the probability of A equals the fraction of trials leading to the occurrence of A in a large series of trials. 2.4 COMBINATORIAL RESULTS Whenever equal probabilities are assigned to the elements of a nite sample space, computation of probabilities of events reduces to counting the points comprising the events. Theorem 2.1 Given n elements a 1 ;:::;a n and m elements b 1 ;:::;b m there are exactly nm distinct ordered pairs (a i ;b j ) containing one element of each kind. Thus, if one experiment has n possible outcomes and another experiment has m possible outcomes, there are nm possible outcomes for the two expriments. More generally we have: Theorem 2.2 Given n 1 elements a 1 ;:::;a n1, n 2 elements b 1 ;:::;b n2, etc., up to n r elements x 1 ;:::;x nr, there are n 1 n 2 n r distinct ordered pairs (a i1 ;b i2 ;:::;x ir ) containing one element of each kind. Thus, if there are r experiments, where the rst has n 1 possible outcomes, the second n 2,..., and the rth n r possible outcomes, there are a total of n 1 n 2 n r possible outcomes for the r experiments. A permutation is an ordered arrangement of objects. An ordered sample of size r is a permutation of r objects obtained from a set ofn elements. Two possible ways for obtaining samples are: sampling with replacement and sampling without replacement. Notice that only samples of size r n without replacement are possible.
5 ELEMENTS OF PROBABILITY THEORY 11 Theorem 2.3 Given a set of n elements and sample size r, there are n r di erent ordered samples with replacement, and n(n 1)(n 2) (n r + 1) = di erent ordered samples without replacement. n! (n r)! Theorem 2.3 implies that the number of permutations or orderings of n elements is equal to n!. A combination is a set of elements without repetitions and without regard to ordering. For example, fa; bg and fb; ag are di erent permutations but only one combination. Thus, a combination is the number of unordered samples of a given size drawn without replacement from a nite set of objects. Theorem 2.4 The number of possible combinations of n objects taken r at a time (r n), is equal to µ Cr n n! n = r!(n r)! = : r Proof. Since the number of ordered samples is equal to the number of unordered samples times the number of ways to order each sample we have that n! (n r)! = Cn r r!; from which C n r = n! r!(n r)! : 2 The number Cr n is called binomial coe±cient, since it occurs in the binomial expansion nx µ n (a + b) n = a n r b r : r r=0 More generally, simple induction gives the following: Theorem 2.5 Given a set of n elements, let n 1 ;:::;n k be positive integers such that P k n i = n. Then there are µ n n! = (2.2) n 1 n 2 :::n k n 1!n 2!:::n k! ways of partitioning the set into k unorderd samples without replacement of size n 1 ;:::;n k respectively. The numbers (2.2) are called multinomial coe±cients.
6 FINITE PROBABILITY SPACES The de nition of probability in terms of equiprobable events is circular. On the other hand, de ning probabilities as limits of relative frequencies inindependent trials under identical conditions is far too restrictive. To avoid these problems we shall now present a purely axiomatic treatment of probabilities. De nition 2.1 A sample space is called a nite probability space if is nite and for every event A µ, there is de ned a real number P(A), called the probability of the event A, such that: A.1: P(A) 0; A.2: P( ) = 1; A.3: if A 1 and A 2 are mutually exclusive events in, then P(A [ B) = P(A) + P(B): It follows from De nition 2.1 that, for any subset A and B of, 0 P(A) 1; (2.3) 2 Further P(A c ) = 1 P(A); (2.4) P(;) = 0; (2.5) A µ B ) P(A) P(B): (2.6) P(A [ B) = P(A) + P(B) P(AB) (Covering theorem): (2.7) This implies the following upper bound on P(A [ B) P(A [ B) P(A) + P(B); with equality if and only if A and B are disjoint. Notice that B = AB[A c B, where AB and A c B are mutually exclusive events. Hence P(B) = P(AB) + P(A c B) and therefore P(B) P(AB) = P(A c B). Substituting in (2.7) gives P(A [ B) = P(A) + P(A c B) (Addition law): (2.8) Also notice that, by De Morgan's law and the Covering theorem, This implies 1 P(AB) = P((AB) c ) = P(A c [ B c ) P(A c ) + P(B c ): P(AB) 1 P(A c ) P(B c ) (Bonferroni inequality); (2.9)
7 ELEMENTS OF PROBABILITY THEORY 13 with equality if and only if A c and B c are disjoint. More generally, if A 1 ;:::;A n is a nite collection of events in, then A 1 ;A c 1A 2 ;:::;A c 1A c 2;:::;A c n 1A n form a partition of S n A i, and so n[ P( A i ) = P(A 1 ) + P(A c 1A 2 ) + + P(A c 1A c 2 A c n 1A n ): This result generalizes the Addition law (2.8). Since for all n 1, we also have A c 1A c 2 A c n 1A n µ A n n[ P( A i ) nx P(A i ): This result generalizes the Covering theorem (2.7). Finally, the generalization of the Bonferroni inequality (2.9) is n\ P( A i ) 1 nx P(A c i): 2.6 MEASURABLE SPACES AND MEASURES For in nite sample spaces, some modi cations of the axioms A.1A.3 and some additional concepts of set theory are required. The reason is that some subsets of an in nite sample space may be so irrregular that it is not possible to assign a probability to them. A set whose elements are sets of will be called a class of sets in. When a set operation performed on sets in a class A gives as a result sets which also belong to A, we say that A is closed under the given operation. De nition 2.2 A nonempty class A of subsets of is called a eld or an algebra on if it contains and is closed under complementation and nite unions, that is, 2 A; A 2 A ) A c 2 A; (2.10) n[ A i 2 A; i = 1;:::;n ) A i 2 A: (2.11) By De Morgan's laws, (2.10) and (2.11) together imply A i 2 A; i = 1;:::;n ) n\ A i 2 A: 2
8 14 Thus, all standardset operations (union, intersection andcomplementation) can be performed any nite number of times on the elements of a eld A without obtaining a set not in A. De nition 2.3 A eld A on is a ¾ eld or a ¾algebra if it is closed under countable unions, that is, if A i 2 A; i = 1; 2;::: ) A i 2 A: (2.12) By De Morgan's law, (2.10) and (2.12) together imply A i 2 A; i = 1; 2;::: ) 1\ A i 2 A: Thus, all standard set operations can be performed any countable number of times on the elements of a ¾ eld A without obtaining a set not in A. If A is a class of subsets of, the smallest eld (¾ eld) containing A is called the eld (¾ eld) generated by A. It can be veri ed that the eld (¾ eld) generated by A is equal to the intersection of all eld (¾ elds) containing A. If A is a ¾ eld on, the pair ( ;A) is called a measurable space. A subset A of is said to be measurable if A 2 A. Given a space, it is generally possible to de ne many ¾ elds on. To distinguish between them, the members of a given ¾ eld A on will be called Ameasurable sets. Example 2.3 An important ¾ eld on the real line < is that generated by the class of all bounded semiclosed intervals of the form (a;b], 1 < a < b < 1. This ¾ eld is called the Borel eld on < and denoted by B. Its elements are called the Borel sets. Since B is a ¾ eld, repeated nite and countable set theoretic operations on its elements will never lead outside B. The measurable space (<;B) is called the Borel line. Notice that B would equivalently be generated by all the open halflines of <, all the open intervals of <, or all the closed intervals of <. 2 A set function is a function de ned on a class of sets. De nition 2.4 A measure ¹ on a measurable space ( ;A) is a nonnegative set function¹de ned for all sets ofa and satisfying: M.1: ¹(;) = 0; M.2: (Countable additivity) if fa i g is any countable sequence of disjoint A measurable sets, then ¹( A i ) = nx ¹(A i ): 2
9 ELEMENTS OF PROBABILITY THEORY 15 Clearly, countable additivity implies nite additivity, that is, if A 1 ;:::;A n is a nite collection of disjoint measurable sets, then n[ ¹( A i ) = nx ¹(A i ): Example 2.4 Let f be a nonnegative function of the points of a set. Let the ¾ eld A consist of all countable subsets of. A measure ¹ on ( ;A) is then de ned as nx ¹(;) = 0; ¹(f! 1 ;:::;! n g) = f(! i ): If f = 1, then ¹ is called counting measure. 2 It is easy to verify that if ¹ is a measure on ( ;A), then it is monotone, that is, ¹(A) ¹(B) whenever A;B 2 A and A ½ B. De nition 2.5 A measure ¹ on ( ;A) is called nite if ¹( ) < 1. It is called ¾ nite if there exists a sequence fa i g of sets in A such that S 1 A i = and ¹(A i ) < 1, n = 1; 2;:::. 2 Example 2.5 An important ¾ nite measure is the one de ned on the Borel line (<;B) by ¹((a;b]) = b a, the length of the interval (a;b]. Such a measure is called Lebesgue measure. It is easy to verify that every countable set is a Borel set of measure zero. 2 De nition 2.6 If ¹ is a measure on ( ;A), the triple ( ;A;¹) is called a measure space. 2 A measure space ( ;A;¹) is called complete if it contains all subsets of sets of measure zero, that is, if A 2 A, B ½ A, and ¹(A) = 0, then B 2 A. It can be shown that each measure space can be completed by the addition of subsets of sets of measure zero. If ¹ is a ¾ nite measure de ned on ( ;A) and F(A) is the ¾ eld generated by A, then it can be shown that there exists a unique measure ¹ on ( ;F(A)) such that ¹ (A) = ¹(A) for all A 2 A. Further, ¹ is also ¾ nite. Such a measure is called the extension of ¹. De nition 2.7 A measure space ( ;A;P) is a probability space if P is a ¾ nite measure with P( ) = PROBABILITY SPACES From De nition 2.7, a probability space is a triple ( ;A;P), where is the sample space associated with an experiment, A is a ¾ eld on, and the probability measure P is a real valued function de ned for all sets in A and satisfying:
10 16 P.1: P(A) 0 for all A 2 A; P.2: P( ) = 1; P.3: (Countable additivity): If fa i g is a countable sequence of disjoint subsets in A, then nx P( A i ) = P(A i ): If ( ;A;P) is a probability space, then the sets in A are interpreted as possible events associated with an experiment. For any A 2 A, the real number P(A) is called the probability of the event A. A support of P is any set A 2 A for which P(A) = 1. If is a nite sample space and A is the set of all the events in (the collection of all subsets of ), then properties P.1{P.3 are equivalent to A.1{A.3 that de ne a nite probability space. As a consequence of properties P.1{P.3, relationships (2.3){(2.9) hold for any A; B 2 A. Further, their generalizations hold for any nite collection of events in A. Notice that the Covering theorem (2.7) can be shown to hold for any countable collection of events in A. Further, if fa i g is a countable collection of events in A such that A 1 µ A 2 µ, then P( A i ) = lim n!1 P(A i): 2.8 CONDITIONAL PROBABILITY Let ( ;A;P) be a probability space and let B 2 A be an event such that P(B) > 0. If we know that B occurred, then the relevant sample space becomes B rather than. This justi es de ning the conditional probability of A given B as P(AjB) = P(AB) (2.13) P(B) if P(B) > 0, and P(A jb) = 0 if P(B) = 0. It is easy to verify that the function P( jb) de ned on A is a probability measure on ( ;A), that is, it satis es P.1{P.3. We call P( jb) the conditional probability measure given B. Notice that (2.13) can equivalently be written as P(AB) = P(AjB)P(B): This result, called the Multiplication law, provides a convenient way of nding P(AB) whenever P(AjB) and P(B) are easy to nd. The Multiplication law can be generalized to a nite collection of events A 1 ;:::;A n in A P(A 1 A n ) = P(A n ja n 1 A 1 )P(A n 1 A 1 ) = P(A n ja n 1 A 1 )P(A n 1 ja n 2 A 1 )P(A n 2 A 1 );
11 ELEMENTS OF PROBABILITY THEORY 17 and so on. Thus P(A 1 A n ) = P(A 1 )P(A 2 ja 1 )P(A 3 ja 2 A 1 ) P(A n ja n 1 A 1 ): Now consider a countable collection fb i g of disjoint events in A such that P(B i ) > 0 for every i and S 1 B i =. Clearly, For any A 2 A, where we used the fact that 1X P( B i ) = P(B i ) = 1: P(A) = P(A \ ( B i )) + P(A \ ( B i ) c ) = P(A \ ( B i )); P(( B i ) c ) = 1 P( B i ) = 0: Thus, by the Morgan's laws, 1X P(A) = P( AB i ) = P(AB i ); since AB i \ AB j = ; for all i 6= j. Therefore P(A) = 1X P(AjB i )P(B i ); which is called the Law of total probabilities. Now let A 2 A be such that P(A) > 0, and consider computing the conditional probability P(B j ja) given knowledge of fp(ajb i )g and fp(b i )g. By the de nition of conditional probability and the Multiplication law, P(B j ja) = P(B ja) P(A) = P(AjB j)p(b j ) P(A) for any xed j = 1; 2;:::. Therefore, by the Law of total probabilities, P(B j ja) = P(A jb j) P(B j ) P 1 P(AjB i)p(b i ) ; which is called Bayes rule.
12 INDEPENDENCE Let A;B 2 A be two events with nonzero probability. If knowing that B occurred gives no information about whether or not A occurred, then the probability assigned to A should not be modi ed by the knowledge that B occurred. Hence P(AjB) = P(A); and so P(AB) = P(A)P(B): (2.14) Two events A;B 2 A are said to be (pairwise) independent if (2.14) holds. Notice that this de nition of independence is symmetric in A and B, and also covers the case when P(A) = 0 or P(B) = 0. It is easy to show that if A and B are independent, then A and B c as well as A c and B c are independent. Three events A;B;C 2 A are said to be (mutually) independent if they are pairwise independent and P(ABC) = P(A)P(B) P(C): This condition is necessary, for pairwise independence does not ensure that, for example, P((AB)C) = P(AB)P(C). It is easy to verify that if A;B and C are independent events, then A[B and C are independent, and A \ B and C are independent. More generally, a family A of events are (mutually) independent if, for every nite collection A 1 ;:::;A n of events in A, REFERENCES n\ P( A i ) = ny P(A i ): (2.15) Billingsley P. (1979) Probability and Measure, Wiley, New York. Feller, W. (1968) An Introduction to Probability Theory and Its Applications (3rd ed.), Vol. 1, Wiley, New York. Halmos, P.R. (1974)Measure Theory, Springer, New York. Kolmogorov, A.N. and Fomin S.V. (1970) Introductory Real Analysis, Dover, New York. Loµeve, M. (1977)Probability Theory (4th ed.), Vol. 1, Springer, New York. Royden H.L. (1968)Real Analysis (2nd ed.), MacMillan, New York.
Introduction to Probability
3 Introduction to Probability Given a fair coin, what can we expect to be the frequency of tails in a sequence of 10 coin tosses? Tossing a coin is an example of a chance experiment, namely a process which
More informationSigma Field Notes for Economics 770 : Econometric Theory
1 Sigma Field Notes for Economics 770 : Econometric Theory Jonathan B. Hill Dept. of Economics, University of North Carolina 0. DEFINITIONS AND CONVENTIONS The following is a small list of repeatedly used
More informationP (A) = lim P (A) = N(A)/N,
1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or nondeterministic experiments. Suppose an experiment can be repeated any number of times, so that we
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationBasic Probability Concepts
page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes
More informationLecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett
Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.
More informationThis chapter describes set theory, a mathematical theory that underlies all of modern mathematics.
Appendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern mathematics. A.1 Basic Definitions Definition A.1.1. A set is an unordered collection of elements.
More informationLecture 2 : Basics of Probability Theory
Lecture 2 : Basics of Probability Theory When an experiment is performed, the realization of the experiment is an outcome in the sample space. If the experiment is performed a number of times, different
More informationLecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University
Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments
More informationDiscrete Mathematics Lecture 5. Harper Langston New York University
Discrete Mathematics Lecture 5 Harper Langston New York University Empty Set S = {x R, x 2 = 1} X = {1, 3}, Y = {2, 4}, C = X Y (X and Y are disjoint) Empty set has no elements Empty set is a subset of
More informationMath 507/420 Homework Assignment #1: Due in class on Friday, September 20. SOLUTIONS
Math 507/420 Homework Assignment #1: Due in class on Friday, September 20. SOLUTIONS 1. Show that a nonempty collection A of subsets is an algebra iff 1) for all A, B A, A B A and 2) for all A A, A c A.
More informationProbability  Part I. Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty.
Probability  Part I Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty. Definition : The sample space (denoted S) of a random experiment
More informationBasics of Probability
Basics of Probability August 27 and September 1, 2009 1 Introduction A phenomena is called random if the exact outcome is uncertain. The mathematical study of randomness is called the theory of probability.
More informationChapter 3: The basic concepts of probability
Chapter 3: The basic concepts of probability Experiment: a measurement process that produces quantifiable results (e.g. throwing two dice, dealing cards, at poker, measuring heights of people, recording
More informationSTAT 270 Probability Basics
STAT 270 Probability Basics Richard Lockhart Simon Fraser University Spring 2015 Surrey 1/28 Purposes of These Notes Jargon: experiment, sample space, outcome, event. Set theory ideas and notation: intersection,
More informationMeasure Theory. Jesus FernandezVillaverde University of Pennsylvania
Measure Theory Jesus FernandezVillaverde University of Pennsylvania 1 Why Bother with Measure Theory? Kolmogorov (1933). Foundation of modern probability. Deals easily with: 1. Continuous versus discrete
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by MenGen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationChap2: The Real Number System (See Royden pp40)
Chap2: The Real Number System (See Royden pp40) 1 Open and Closed Sets of Real Numbers The simplest sets of real numbers are the intervals. We define the open interval (a, b) to be the set (a, b) = {x
More informationIAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION
IAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION 1 WHAT IS STATISTICS? Statistics is a science of collecting data, organizing and describing it and drawing conclusions from it. That is, statistics
More informationDiscrete Mathematics & Mathematical Reasoning Chapter 6: Counting
Discrete Mathematics & Mathematical Reasoning Chapter 6: Counting Colin Stirling Informatics Slides originally by Kousha Etessami Colin Stirling (Informatics) Discrete Mathematics (Chapter 6) Today 1 /
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
More informationMath212a1010 Lebesgue measure.
Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.
More informationChapter 1. Probability Theory: Introduction. Definitions Algebra. RS Chapter 1 Random Variables
Chapter 1 Probability Theory: Introduction Definitions Algebra Definitions: Semiring A collection of sets F is called a semiring if it satisfies: Φ F. If A, B F, then A B F. If A, B F, then there exists
More informationFinite Sets. Theorem 5.1. Two nonempty finite sets have the same cardinality if and only if they are equivalent.
MATH 337 Cardinality Dr. Neal, WKU We now shall prove that the rational numbers are a countable set while R is uncountable. This result shows that there are two different magnitudes of infinity. But we
More informationLecture 2: Probability
Lecture 2: Probability Assist. Prof. Dr. Emel YAVUZ DUMAN MCB1007 Introduction to Probability and Statistics İstanbul Kültür University Outline 1 Introduction 2 Sample Spaces 3 Event 4 The Probability
More informationBasic concepts in probability. Sue Gordon
Mathematics Learning Centre Basic concepts in probability Sue Gordon c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Set Notation You may omit this section if you are
More informationMath/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability
Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock
More informationMAT2400 Analysis I. A brief introduction to proofs, sets, and functions
MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take
More informationNOTES ON MEASURE THEORY. M. Papadimitrakis Department of Mathematics University of Crete. Autumn of 2004
NOTES ON MEASURE THEORY M. Papadimitrakis Department of Mathematics University of Crete Autumn of 2004 2 Contents 1 σalgebras 7 1.1 σalgebras............................... 7 1.2 Generated σalgebras.........................
More informationRandom Variables and Measurable Functions.
Chapter 3 Random Variables and Measurable Functions. 3.1 Measurability Definition 42 (Measurable function) Let f be a function from a measurable space (Ω, F) into the real numbers. We say that the function
More informationLECTURE NOTES IN MEASURE THEORY. Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12)
1 LECTURE NOTES IN MEASURE THEORY Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12) 2 PREFACE These are lecture notes on integration theory for a eightweek
More informationPROBABILITY NOTIONS. Summary. 1. Random experiment
PROBABILITY NOTIONS Summary 1. Random experiment... 1 2. Sample space... 2 3. Event... 2 4. Probability calculation... 3 4.1. Fundamental sample space... 3 4.2. Calculation of probability... 3 4.3. Non
More informationAssigning Probabilities
What is a Probability? Probabilities are numbers between 0 and 1 that indicate the likelihood of an event. Generally, the statement that the probability of hitting a target that is being fired at is
More informationMATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:
MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,
More informationNotes on counting finite sets
Notes on counting finite sets Murray Eisenberg February 26, 2009 Contents 0 Introduction 2 1 What is a finite set? 2 2 Counting unions and cartesian products 4 2.1 Sum rules......................................
More informationRi and. i=1. S i N. and. R R i
The subset R of R n is a closed rectangle if there are n nonempty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an
More informationLesson 3 Chapter 2: Introduction to Probability
Lesson 3 Chapter 2: Introduction to Probability Department of Statistics The Pennsylvania State University 1 2 The Probability Mass Function and Probability Sampling Counting Techniques 3 4 The Law of
More informationChapter 1. SigmaAlgebras. 1.1 Definition
Chapter 1 SigmaAlgebras 1.1 Definition Consider a set X. A σ algebra F of subsets of X is a collection F of subsets of X satisfying the following conditions: (a) F (b) if B F then its complement B c is
More informationSections 2.1, 2.2 and 2.4
SETS Sections 2.1, 2.2 and 2.4 Chapter Summary Sets The Language of Sets Set Operations Set Identities Introduction Sets are one of the basic building blocks for the types of objects considered in discrete
More information5. Probability Calculus
5. Probability Calculus So far we have concentrated on descriptive statistics (deskriptiivinen eli kuvaileva tilastotiede), that is methods for organizing and summarizing data. As was already indicated
More informationThe set consisting of all natural numbers that are in A and are in B is the set f1; 3; 5g;
Chapter 5 Set Theory 5.1 Sets and Operations on Sets Preview Activity 1 (Set Operations) Before beginning this section, it would be a good idea to review sets and set notation, including the roster method
More informationUndergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics
Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights
More informationDiscrete Mathematics
Discrete Mathematics ChihWei Yi Dept. of Computer Science National Chiao Tung University March 16, 2009 2.1 Sets 2.1 Sets 2.1 Sets Basic Notations for Sets For sets, we ll use variables S, T, U,. We can
More informationCARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE
CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE With the notion of bijection at hand, it is easy to formalize the idea that two finite sets have the same number of elements: we just need to verify
More informationStatistics in Geophysics: Introduction and Probability Theory
Statistics in Geophysics: Introduction and Steffen Unkel Department of Statistics LudwigMaximiliansUniversity Munich, Germany Winter Term 2013/14 1/32 What is Statistics? Introduction Statistics is the
More informationSet theory, and set operations
1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we
More informationProbabilities and Random Variables
Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so
More informationExtension of measure
1 Extension of measure Sayan Mukherjee Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard.
More informationk, then n = p2α 1 1 pα k
Powers of Integers An integer n is a perfect square if n = m for some integer m. Taking into account the prime factorization, if m = p α 1 1 pα k k, then n = pα 1 1 p α k k. That is, n is a perfect square
More informationProbability and Statistics Vocabulary List (Definitions for Middle School Teachers)
Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence
More informationPOWER SETS AND RELATIONS
POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty
More informationProbability Theory. Florian Herzog. A random variable is neither random nor variable. GianCarlo Rota, M.I.T..
Probability Theory A random variable is neither random nor variable. GianCarlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,
More informationSTAT 571 Assignment 1 solutions
STAT 571 Assignment 1 solutions 1. If Ω is a set and C a collection of subsets of Ω, let A be the intersection of all σalgebras that contain C. rove that A is the σalgebra generated by C. Solution: Let
More informationEvents. Independence. Coin Tossing. Random Phenomena
Random Phenomena Events A random phenomenon is a situation in which we know what outcomes could happen, but we don t know which particular outcome did or will happen For any random phenomenon, each attempt,
More informationModule 6: Basic Counting
Module 6: Basic Counting Theme 1: Basic Counting Principle We start with two basic counting principles, namely, the sum rule and the multiplication rule. The Sum Rule: If there are n 1 different objects
More informationSo let us begin our quest to find the holy grail of real analysis.
1 Section 5.2 The Complete Ordered Field: Purpose of Section We present an axiomatic description of the real numbers as a complete ordered field. The axioms which describe the arithmetic of the real numbers
More information33 Probability: Some Basic Terms
33 Probability: Some Basic Terms In this and the coming sections we discuss the fundamental concepts of probability at a level at which no previous exposure to the topic is assumed. Probability has been
More informationChapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum aposteriori Hypotheses
Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum aposteriori Hypotheses ML:IV1 Statistical Learning STEIN 20052015 Area Overview Mathematics Statistics...... Stochastics
More informationChapter 1. Open Sets, Closed Sets, and Borel Sets
1.4. Borel Sets 1 Chapter 1. Open Sets, Closed Sets, and Borel Sets Section 1.4. Borel Sets Note. Recall that a set of real numbers is open if and only if it is a countable disjoint union of open intervals.
More informationProblem Set. Problem Set #2. Math 5322, Fall December 3, 2001 ANSWERS
Problem Set Problem Set #2 Math 5322, Fall 2001 December 3, 2001 ANSWERS i Problem 1. [Problem 18, page 32] Let A P(X) be an algebra, A σ the collection of countable unions of sets in A, and A σδ the collection
More informationProbability and Counting
Probability and Counting Basic Counting Principles Permutations and Combinations Sample Spaces, Events, Probability Union, Intersection, Complements; Odds Conditional Probability, Independence Bayes Formula
More informationSets, Relations and Functions
Sets, Relations and Functions Eric Pacuit Department of Philosophy University of Maryland, College Park pacuit.org epacuit@umd.edu ugust 26, 2014 These notes provide a very brief background in discrete
More informationQuestion: What is the probability that a fivecard poker hand contains a flush, that is, five cards of the same suit?
ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the
More informationStructure of Measurable Sets
Structure of Measurable Sets In these notes we discuss the structure of Lebesgue measurable subsets of R from several different points of view. Along the way, we will see several alternative characterizations
More informationSolutions to Tutorial 2 (Week 3)
THE UIVERSITY OF SYDEY SCHOOL OF MATHEMATICS AD STATISTICS Solutions to Tutorial 2 (Week 3) MATH3969: Measure Theory and Fourier Analysis (Advanced) Semester 2, 2016 Web Page: http://sydney.edu.au/science/maths/u/ug/sm/math3969/
More informationPOSITIVE INTEGERS, INTEGERS AND RATIONAL NUMBERS OBTAINED FROM THE AXIOMS OF THE REAL NUMBER SYSTEM
MAT 1011 TECHNICAL ENGLISH I 03.11.2016 Dokuz Eylül University Faculty of Science Department of Mathematics Instructor: Engin Mermut Course assistant: Zübeyir Türkoğlu web: http://kisi.deu.edu.tr/engin.mermut/
More informationChapter 4. Probability and Probability Distributions
Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the
More informationSets. A set is a collection of (mathematical) objects, with the collection treated as a single mathematical object.
Sets 1 Sets Informally: A set is a collection of (mathematical) objects, with the collection treated as a single mathematical object. Examples: real numbers, complex numbers, C integers, All students in
More informationREAL ANALYSIS I HOMEWORK 2
REAL ANALYSIS I HOMEWORK 2 CİHAN BAHRAN The questions are from Stein and Shakarchi s text, Chapter 1. 1. Prove that the Cantor set C constructed in the text is totally disconnected and perfect. In other
More informationElementary Number Theory We begin with a bit of elementary number theory, which is concerned
CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,
More informationMeasure Theory and Probability
Chapter 2 Measure Theory and Probability 2.1 Introduction In advanced probability courses, a random variable (r.v.) is defined rigorously through measure theory, which is an indepth mathematics discipline
More informationA (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes.
Chapter 7 Probability 7.1 Experiments, Sample Spaces, and Events A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes. Each outcome
More informationCombinatorial Proofs
Combinatorial Proofs Two Counting Principles Some proofs concerning finite sets involve counting the number of elements of the sets, so we will look at the basics of counting. Addition Principle: If A
More informationSets and set operations
CS 441 Discrete Mathematics for CS Lecture 7 Sets and set operations Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square asic discrete structures Discrete math = study of the discrete structures used
More informationLogic & Discrete Math in Software Engineering (CAS 701) Dr. Borzoo Bonakdarpour
Logic & Discrete Math in Software Engineering (CAS 701) Background Dr. Borzoo Bonakdarpour Department of Computing and Software McMaster University Dr. Borzoo Bonakdarpour Logic & Discrete Math in SE (CAS
More information1.3 Induction and Other Proof Techniques
4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.
More informationAXIOMATIC PROBABILITY AND POINT SETS
AXIOMATIC PROBABILITY AND POINT SETS The axioms of Kolmogorov. Let S denote a sample space with a probability measure P defined over it, such that probability of any event A S is given by P (A). Then,
More informationMath 117 Chapter 7 Sets and Probability
Math 117 Chapter 7 and Probability Flathead Valley Community College Page 1 of 15 1. A set is a welldefined collection of specific objects. Each item in the set is called an element or a member. Curly
More information+ Section 6.2 and 6.3
Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities
More informationMA651 Topology. Lecture 6. Separation Axioms.
MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples
More informationPART I. THE REAL NUMBERS
PART I. THE REAL NUMBERS This material assumes that you are already familiar with the real number system and the representation of the real numbers as points on the real line. I.1. THE NATURAL NUMBERS
More information6.3 Conditional Probability and Independence
222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted
More informationProbability OPRE 6301
Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.
More informationDefinition of Random Variable A random variable is a function from a sample space S into the real numbers.
.4 Random Variable Motivation example In an opinion poll, we might decide to ask 50 people whether they agree or disagree with a certain issue. If we record a for agree and 0 for disagree, the sample space
More informationA set is a Many that allows itself to be thought of as a One. (Georg Cantor)
Chapter 4 Set Theory A set is a Many that allows itself to be thought of as a One. (Georg Cantor) In the previous chapters, we have often encountered sets, for example, prime numbers form a set, domains
More informationChingHan Hsu, BMES, National Tsing Hua University c 2014 by ChingHan Hsu, Ph.D., BMIR Lab
Lecture 2 Probability BMIR Lecture Series in Probability and Statistics ChingHan Hsu, BMES, National Tsing Hua University c 2014 by ChingHan Hsu, Ph.D., BMIR Lab 2.1 1 Sample Spaces and Events Random
More informationSets and Cardinality Notes for C. F. Miller
Sets and Cardinality Notes for 620111 C. F. Miller Semester 1, 2000 Abstract These lecture notes were compiled in the Department of Mathematics and Statistics in the University of Melbourne for the use
More informationI. WHAT IS PROBABILITY?
C HAPTER 3 PROBABILITY Random Experiments I. WHAT IS PROBABILITY? The weatherman on 0 o clock news program states that there is a 20% chance that it will snow tomorrow, a 65% chance that it will rain and
More informationSets, Venn Diagrams & Counting
MT 142 College Mathematics Sets, Venn Diagrams & Counting Module SC Terri Miller revised January 5, 2011 What is a set? Sets set is a collection of objects. The objects in the set are called elements of
More informationA1. Basic Reviews PERMUTATIONS and COMBINATIONS... or HOW TO COUNT
A1. Basic Reviews Appendix / A1. Basic Reviews / Perms & Combos1 PERMUTATIONS and COMBINATIONS... or HOW TO COUNT Question 1: Suppose we wish to arrange n 5 people {a, b, c, d, e}, standing side by side,
More informationStatistical Inference. Prof. Kate Calder. If the coin is fair (chance of heads = chance of tails) then
Probability Statistical Inference Question: How often would this method give the correct answer if I used it many times? Answer: Use laws of probability. 1 Example: Tossing a coin If the coin is fair (chance
More information4. Joint Distributions
Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose
More informationElements of probability theory
The role of probability theory in statistics We collect data so as to provide evidentiary support for answers we give to our many questions about the world (and in our particular case, about the business
More informationNotes on Chapter 1, Section 2 Arithmetic and Divisibility
Notes on Chapter 1, Section 2 Arithmetic and Divisibility August 16, 2006 1 Arithmetic Properties of the Integers Recall that the set of integers is the set Z = f0; 1; 1; 2; 2; 3; 3; : : :g. The integers
More informationSets and functions. {x R : x > 0}.
Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.
More informationGod created the integers and the rest is the work of man. (Leopold Kronecker, in an afterdinner speech at a conference, Berlin, 1886)
Chapter 2 Numbers God created the integers and the rest is the work of man. (Leopold Kronecker, in an afterdinner speech at a conference, Berlin, 1886) God created the integers and the rest is the work
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationThe concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics.
Elementary probability theory The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics. 1.1 Experiments, outcomes and sample
More information) + ˆf (n) sin( 2πnt. = 2 u x 2, t > 0, 0 < x < 1. u(0, t) = u(1, t) = 0, t 0. (x, 0) = 0 0 < x < 1.
Introduction to Fourier analysis This semester, we re going to study various aspects of Fourier analysis. In particular, we ll spend some time reviewing and strengthening the results from Math 425 on Fourier
More information