Elements of probability theory
|
|
- Melvin Strickland
- 7 years ago
- Views:
Transcription
1 2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted with certainty. 2.1 SAMPLE SPACES A situation whose outcomes occur randomly is called an experiment. The set of all possible outcomes of an experiment is called the sample space corresponding to the experiment, and is denoted by. A generic element of is called a sample point, or simply a point, and is denoted by! 2. Example 2.1 A coin is tossed twice and the sequence of heads (H) and tails (T) is recorded. The possible outcomes of this experiment are HH, HT, TH and TT. Hence, the sample space corresponding to this experiment consists of the four points = fhh;ht;th;ttg: A sample space is called nite if it is empty or contains a nite number of points, otherwise is called in nite. A sample space is called countable if its points can be indexed by the set of positive integers. A sample space that is nite or countable is called discrete. Example 2.2 A coin is tossed until H is recorded. The sample space corrsponding to this experiment is = fh;th;tth;ttth;tttth;:::g: Thus contains countably many points. 2 Not all sample spaces are discrete. For example, the sample space consisting of all positive real numbers is not discrete, neither is the sample space consisting of all real numbers in the interval [0; 1]. 2
2 8 2.2 RELATIONS AMONG EVENTS A subset of points in a sample space is called an event in. An event occurs if and only if one of its points occurs. Viewed as an event, is called the sure event. In general, events will be de ned by certain conditions on the points that compose them. Because events are just subsets of points in, concepts and results from point set theory apply to events. In particular, if A and B are events in, A implies B, written A µ B, if and only if all points in A also belong to B. The events A and B are identical, written A = B, if and only if A µ B and B µ A, that is, A and B contain exactly the same points. Other usual operations and relations between sets are listed below (union) A [ B = f! 2 :! 2 A or! 2 Bg, (intersection) A \ B = f! 2 :! 2 A and! 2 Bg, (complement) A c = f! 2 :! 62 Ag, (impossible event) ; = c, (di erence) A B = A \ B c, (symmetric di erence) A B = (A B) [ (B A). Instead of A \ B we also write AB. Operations and relations between sets are easy to visualize using Venn diagrams. If A \ B = ;, then A andb are called disjoint or mutually exclusive events. Intersections and unions of a countable collection A 1 ;A 2 ;A 3 ;::: of events are denoted by T 1 A i and S 1 A i, respectively. If A is an arbitrary family of subsets in, we write [ A2A A = f! 2 :! 2 A for some A 2 Ag and \ A2A A = f! 2 :! 2 A for all A 2 Ag Some basic relationships between events are: A [ ; = A, A [ =, A [ A = A, [ ; =, A \ ; = ;, A \ = A, A \ A = A, \ ; = ;. and
3 ELEMENTS OF PROBABILITY THEORY 9 (commutative law) A [ B = B [ A, A \ B = B \ A, (distributive law) A [ (B \ C) = (A [ B) \ (A [ C), A \ (B [ C) = (A \ B) [ (A \ C), (associative law) (A [ B) [ C = A [ (B [ C), (A \ B) \ C = A \ (B \ C), (De Morgan's laws) (A [ B) c = A c \ B c, (A \ B) c = A c [ B c. De Morgan's laws show that complementation, union and intersection are not independent operations. The commutative, distributive and associative laws and De Morgan's laws can easily be extended to a countable collection of events A 1 ;A 2 ;A 3 ;:::. The characteristic function (cf) of an event A µ is a function 1 A ( ) de ned for all! 2 by the relation ½ 1; if! 2 A, 1 A (!) = 0; otherwise. We also write 1(! 2 A) or, more compactly, 1(A). There is a one-to-one correspondence between sets and their cf's, and all properties of sets and set operations can be expressed in terms of cf's. For example, if C = A c then 1 C = 1 1 A, if C = A [ B then 1 C = max(1 A ; 1 B ), and if C = A \ B then 1 C = 1 A 1 B. 2.3 PROBABILITY How can we attach probabilities to events? The rst and easiest case is an experiment with a nite sample space consisting of N points. Suppose that, because of the nature of the experiment (e.g. tossing a fair coin), all points in are equiprobable, that is, equally likely, and let A be some event in. We de ne the probability of A, written P(A), as the ratio P(A) = N(A) N ; (2.1) where N(A) denotes the number of points in A. For any A µ, we have 0 P(A) 1; P( ) = N( ) N = 1; P(;) = N(;) N = 0: Further, if A and B are disjoint events in, then P(A [ B) = N(A [ B) N = N(A) N + N(B) N = P(A) + P(B):
4 10 Despite its simplicity, formula (2.1) can lead to non trivial calculations. In order to use it in a given problem, we need to determine: (i) the number N of all equiprobable outcomes, and (ii) the number of all those outcomes leading to the occurrence of A. A second case is whena basic experimentcan be repeated inexactly the same conditions any number n of times. We call this situation the case of independent trials under identical conditions. In this case, we can give a precise meaning to the concept of probability. In each trial a particular event A may or may not occur. Let n(a) be the number of trials in which A occurs. The relative frequency of the event A in the given series of n trials is de ned as f n (A) = n(a) n : It is an empirical fact that the f n (A) observed for di erent series of trials are virtually the same for large n, clustering about a constant value P(A), called the probability of A. Roughly speaking, the probability of A equals the fraction of trials leading to the occurrence of A in a large series of trials. 2.4 COMBINATORIAL RESULTS Whenever equal probabilities are assigned to the elements of a nite sample space, computation of probabilities of events reduces to counting the points comprising the events. Theorem 2.1 Given n elements a 1 ;:::;a n and m elements b 1 ;:::;b m there are exactly nm distinct ordered pairs (a i ;b j ) containing one element of each kind. Thus, if one experiment has n possible outcomes and another experiment has m possible outcomes, there are nm possible outcomes for the two expriments. More generally we have: Theorem 2.2 Given n 1 elements a 1 ;:::;a n1, n 2 elements b 1 ;:::;b n2, etc., up to n r elements x 1 ;:::;x nr, there are n 1 n 2 n r distinct ordered pairs (a i1 ;b i2 ;:::;x ir ) containing one element of each kind. Thus, if there are r experiments, where the rst has n 1 possible outcomes, the second n 2,..., and the rth n r possible outcomes, there are a total of n 1 n 2 n r possible outcomes for the r experiments. A permutation is an ordered arrangement of objects. An ordered sample of size r is a permutation of r objects obtained from a set ofn elements. Two possible ways for obtaining samples are: sampling with replacement and sampling without replacement. Notice that only samples of size r n without replacement are possible.
5 ELEMENTS OF PROBABILITY THEORY 11 Theorem 2.3 Given a set of n elements and sample size r, there are n r di erent ordered samples with replacement, and n(n 1)(n 2) (n r + 1) = di erent ordered samples without replacement. n! (n r)! Theorem 2.3 implies that the number of permutations or orderings of n elements is equal to n!. A combination is a set of elements without repetitions and without regard to ordering. For example, fa; bg and fb; ag are di erent permutations but only one combination. Thus, a combination is the number of unordered samples of a given size drawn without replacement from a nite set of objects. Theorem 2.4 The number of possible combinations of n objects taken r at a time (r n), is equal to µ Cr n n! n = r!(n r)! = : r Proof. Since the number of ordered samples is equal to the number of unordered samples times the number of ways to order each sample we have that n! (n r)! = Cn r r!; from which C n r = n! r!(n r)! : 2 The number Cr n is called binomial coe±cient, since it occurs in the binomial expansion nx µ n (a + b) n = a n r b r : r r=0 More generally, simple induction gives the following: Theorem 2.5 Given a set of n elements, let n 1 ;:::;n k be positive integers such that P k n i = n. Then there are µ n n! = (2.2) n 1 n 2 :::n k n 1!n 2!:::n k! ways of partitioning the set into k unorderd samples without replacement of size n 1 ;:::;n k respectively. The numbers (2.2) are called multinomial coe±cients.
6 FINITE PROBABILITY SPACES The de nition of probability in terms of equiprobable events is circular. On the other hand, de ning probabilities as limits of relative frequencies inindependent trials under identical conditions is far too restrictive. To avoid these problems we shall now present a purely axiomatic treatment of probabilities. De nition 2.1 A sample space is called a nite probability space if is nite and for every event A µ, there is de ned a real number P(A), called the probability of the event A, such that: A.1: P(A) 0; A.2: P( ) = 1; A.3: if A 1 and A 2 are mutually exclusive events in, then P(A [ B) = P(A) + P(B): It follows from De nition 2.1 that, for any subset A and B of, 0 P(A) 1; (2.3) 2 Further P(A c ) = 1 P(A); (2.4) P(;) = 0; (2.5) A µ B ) P(A) P(B): (2.6) P(A [ B) = P(A) + P(B) P(AB) (Covering theorem): (2.7) This implies the following upper bound on P(A [ B) P(A [ B) P(A) + P(B); with equality if and only if A and B are disjoint. Notice that B = AB[A c B, where AB and A c B are mutually exclusive events. Hence P(B) = P(AB) + P(A c B) and therefore P(B) P(AB) = P(A c B). Substituting in (2.7) gives P(A [ B) = P(A) + P(A c B) (Addition law): (2.8) Also notice that, by De Morgan's law and the Covering theorem, This implies 1 P(AB) = P((AB) c ) = P(A c [ B c ) P(A c ) + P(B c ): P(AB) 1 P(A c ) P(B c ) (Bonferroni inequality); (2.9)
7 ELEMENTS OF PROBABILITY THEORY 13 with equality if and only if A c and B c are disjoint. More generally, if A 1 ;:::;A n is a nite collection of events in, then A 1 ;A c 1A 2 ;:::;A c 1A c 2;:::;A c n 1A n form a partition of S n A i, and so n[ P( A i ) = P(A 1 ) + P(A c 1A 2 ) + + P(A c 1A c 2 A c n 1A n ): This result generalizes the Addition law (2.8). Since for all n 1, we also have A c 1A c 2 A c n 1A n µ A n n[ P( A i ) nx P(A i ): This result generalizes the Covering theorem (2.7). Finally, the generalization of the Bonferroni inequality (2.9) is n\ P( A i ) 1 nx P(A c i): 2.6 MEASURABLE SPACES AND MEASURES For in nite sample spaces, some modi cations of the axioms A.1-A.3 and some additional concepts of set theory are required. The reason is that some subsets of an in nite sample space may be so irrregular that it is not possible to assign a probability to them. A set whose elements are sets of will be called a class of sets in. When a set operation performed on sets in a class A gives as a result sets which also belong to A, we say that A is closed under the given operation. De nition 2.2 A nonempty class A of subsets of is called a eld or an algebra on if it contains and is closed under complementation and nite unions, that is, 2 A; A 2 A ) A c 2 A; (2.10) n[ A i 2 A; i = 1;:::;n ) A i 2 A: (2.11) By De Morgan's laws, (2.10) and (2.11) together imply A i 2 A; i = 1;:::;n ) n\ A i 2 A: 2
8 14 Thus, all standardset operations (union, intersection andcomplementation) can be performed any nite number of times on the elements of a eld A without obtaining a set not in A. De nition 2.3 A eld A on is a ¾- eld or a ¾-algebra if it is closed under countable unions, that is, if A i 2 A; i = 1; 2;::: ) A i 2 A: (2.12) By De Morgan's law, (2.10) and (2.12) together imply A i 2 A; i = 1; 2;::: ) 1\ A i 2 A: Thus, all standard set operations can be performed any countable number of times on the elements of a ¾- eld A without obtaining a set not in A. If A is a class of subsets of, the smallest eld (¾- eld) containing A is called the eld (¾- eld) generated by A. It can be veri ed that the eld (¾- eld) generated by A is equal to the intersection of all eld (¾- elds) containing A. If A is a ¾- eld on, the pair ( ;A) is called a measurable space. A subset A of is said to be measurable if A 2 A. Given a space, it is generally possible to de ne many ¾- elds on. To distinguish between them, the members of a given ¾- eld A on will be called A-measurable sets. Example 2.3 An important ¾- eld on the real line < is that generated by the class of all bounded semi-closed intervals of the form (a;b], 1 < a < b < 1. This ¾- eld is called the Borel eld on < and denoted by B. Its elements are called the Borel sets. Since B is a ¾- eld, repeated nite and countable set theoretic operations on its elements will never lead outside B. The measurable space (<;B) is called the Borel line. Notice that B would equivalently be generated by all the open half-lines of <, all the open intervals of <, or all the closed intervals of <. 2 A set function is a function de ned on a class of sets. De nition 2.4 A measure ¹ on a measurable space ( ;A) is a nonnegative set function¹de ned for all sets ofa and satisfying: M.1: ¹(;) = 0; M.2: (Countable additivity) if fa i g is any countable sequence of disjoint A- measurable sets, then ¹( A i ) = nx ¹(A i ): 2
9 ELEMENTS OF PROBABILITY THEORY 15 Clearly, countable additivity implies nite additivity, that is, if A 1 ;:::;A n is a nite collection of disjoint measurable sets, then n[ ¹( A i ) = nx ¹(A i ): Example 2.4 Let f be a nonnegative function of the points of a set. Let the ¾- eld A consist of all countable subsets of. A measure ¹ on ( ;A) is then de ned as nx ¹(;) = 0; ¹(f! 1 ;:::;! n g) = f(! i ): If f = 1, then ¹ is called counting measure. 2 It is easy to verify that if ¹ is a measure on ( ;A), then it is monotone, that is, ¹(A) ¹(B) whenever A;B 2 A and A ½ B. De nition 2.5 A measure ¹ on ( ;A) is called nite if ¹( ) < 1. It is called ¾- nite if there exists a sequence fa i g of sets in A such that S 1 A i = and ¹(A i ) < 1, n = 1; 2;:::. 2 Example 2.5 An important ¾- nite measure is the one de ned on the Borel line (<;B) by ¹((a;b]) = b a, the length of the interval (a;b]. Such a measure is called Lebesgue measure. It is easy to verify that every countable set is a Borel set of measure zero. 2 De nition 2.6 If ¹ is a measure on ( ;A), the triple ( ;A;¹) is called a measure space. 2 A measure space ( ;A;¹) is called complete if it contains all subsets of sets of measure zero, that is, if A 2 A, B ½ A, and ¹(A) = 0, then B 2 A. It can be shown that each measure space can be completed by the addition of subsets of sets of measure zero. If ¹ is a ¾- nite measure de ned on ( ;A) and F(A) is the ¾- eld generated by A, then it can be shown that there exists a unique measure ¹ on ( ;F(A)) such that ¹ (A) = ¹(A) for all A 2 A. Further, ¹ is also ¾- nite. Such a measure is called the extension of ¹. De nition 2.7 A measure space ( ;A;P) is a probability space if P is a ¾- nite measure with P( ) = PROBABILITY SPACES From De nition 2.7, a probability space is a triple ( ;A;P), where is the sample space associated with an experiment, A is a ¾- eld on, and the probability measure P is a real valued function de ned for all sets in A and satisfying:
10 16 P.1: P(A) 0 for all A 2 A; P.2: P( ) = 1; P.3: (Countable additivity): If fa i g is a countable sequence of disjoint subsets in A, then nx P( A i ) = P(A i ): If ( ;A;P) is a probability space, then the sets in A are interpreted as possible events associated with an experiment. For any A 2 A, the real number P(A) is called the probability of the event A. A support of P is any set A 2 A for which P(A) = 1. If is a nite sample space and A is the set of all the events in (the collection of all subsets of ), then properties P.1{P.3 are equivalent to A.1{A.3 that de ne a nite probability space. As a consequence of properties P.1{P.3, relationships (2.3){(2.9) hold for any A; B 2 A. Further, their generalizations hold for any nite collection of events in A. Notice that the Covering theorem (2.7) can be shown to hold for any countable collection of events in A. Further, if fa i g is a countable collection of events in A such that A 1 µ A 2 µ, then P( A i ) = lim n!1 P(A i): 2.8 CONDITIONAL PROBABILITY Let ( ;A;P) be a probability space and let B 2 A be an event such that P(B) > 0. If we know that B occurred, then the relevant sample space becomes B rather than. This justi es de ning the conditional probability of A given B as P(AjB) = P(AB) (2.13) P(B) if P(B) > 0, and P(A jb) = 0 if P(B) = 0. It is easy to verify that the function P( jb) de ned on A is a probability measure on ( ;A), that is, it satis es P.1{P.3. We call P( jb) the conditional probability measure given B. Notice that (2.13) can equivalently be written as P(AB) = P(AjB)P(B): This result, called the Multiplication law, provides a convenient way of nding P(AB) whenever P(AjB) and P(B) are easy to nd. The Multiplication law can be generalized to a nite collection of events A 1 ;:::;A n in A P(A 1 A n ) = P(A n ja n 1 A 1 )P(A n 1 A 1 ) = P(A n ja n 1 A 1 )P(A n 1 ja n 2 A 1 )P(A n 2 A 1 );
11 ELEMENTS OF PROBABILITY THEORY 17 and so on. Thus P(A 1 A n ) = P(A 1 )P(A 2 ja 1 )P(A 3 ja 2 A 1 ) P(A n ja n 1 A 1 ): Now consider a countable collection fb i g of disjoint events in A such that P(B i ) > 0 for every i and S 1 B i =. Clearly, For any A 2 A, where we used the fact that 1X P( B i ) = P(B i ) = 1: P(A) = P(A \ ( B i )) + P(A \ ( B i ) c ) = P(A \ ( B i )); P(( B i ) c ) = 1 P( B i ) = 0: Thus, by the Morgan's laws, 1X P(A) = P( AB i ) = P(AB i ); since AB i \ AB j = ; for all i 6= j. Therefore P(A) = 1X P(AjB i )P(B i ); which is called the Law of total probabilities. Now let A 2 A be such that P(A) > 0, and consider computing the conditional probability P(B j ja) given knowledge of fp(ajb i )g and fp(b i )g. By the de nition of conditional probability and the Multiplication law, P(B j ja) = P(B ja) P(A) = P(AjB j)p(b j ) P(A) for any xed j = 1; 2;:::. Therefore, by the Law of total probabilities, P(B j ja) = P(A jb j) P(B j ) P 1 P(AjB i)p(b i ) ; which is called Bayes rule.
12 INDEPENDENCE Let A;B 2 A be two events with non-zero probability. If knowing that B occurred gives no information about whether or not A occurred, then the probability assigned to A should not be modi ed by the knowledge that B occurred. Hence P(AjB) = P(A); and so P(AB) = P(A)P(B): (2.14) Two events A;B 2 A are said to be (pairwise) independent if (2.14) holds. Notice that this de nition of independence is symmetric in A and B, and also covers the case when P(A) = 0 or P(B) = 0. It is easy to show that if A and B are independent, then A and B c as well as A c and B c are independent. Three events A;B;C 2 A are said to be (mutually) independent if they are pairwise independent and P(ABC) = P(A)P(B) P(C): This condition is necessary, for pairwise independence does not ensure that, for example, P((AB)C) = P(AB)P(C). It is easy to verify that if A;B and C are independent events, then A[B and C are independent, and A \ B and C are independent. More generally, a family A of events are (mutually) independent if, for every nite collection A 1 ;:::;A n of events in A, REFERENCES n\ P( A i ) = ny P(A i ): (2.15) Billingsley P. (1979) Probability and Measure, Wiley, New York. Feller, W. (1968) An Introduction to Probability Theory and Its Applications (3rd ed.), Vol. 1, Wiley, New York. Halmos, P.R. (1974)Measure Theory, Springer, New York. Kolmogorov, A.N. and Fomin S.V. (1970) Introductory Real Analysis, Dover, New York. Loµeve, M. (1977)Probability Theory (4th ed.), Vol. 1, Springer, New York. Royden H.L. (1968)Real Analysis (2nd ed.), MacMillan, New York.
Introduction to Probability
3 Introduction to Probability Given a fair coin, what can we expect to be the frequency of tails in a sequence of 10 coin tosses? Tossing a coin is an example of a chance experiment, namely a process which
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationBasic Probability Concepts
page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes
More informationLecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett
Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.
More informationLecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University
Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationIAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION
IAM 530 ELEMENTS OF PROBABILITY AND STATISTICS INTRODUCTION 1 WHAT IS STATISTICS? Statistics is a science of collecting data, organizing and describing it and drawing conclusions from it. That is, statistics
More informationMath/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability
Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock
More informationLECTURE NOTES IN MEASURE THEORY. Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12)
1 LECTURE NOTES IN MEASURE THEORY Christer Borell Matematik Chalmers och Göteborgs universitet 412 96 Göteborg (Version: January 12) 2 PREFACE These are lecture notes on integration theory for a eight-week
More informationk, then n = p2α 1 1 pα k
Powers of Integers An integer n is a perfect square if n = m for some integer m. Taking into account the prime factorization, if m = p α 1 1 pα k k, then n = pα 1 1 p α k k. That is, n is a perfect square
More informationDiscrete Mathematics
Discrete Mathematics Chih-Wei Yi Dept. of Computer Science National Chiao Tung University March 16, 2009 2.1 Sets 2.1 Sets 2.1 Sets Basic Notations for Sets For sets, we ll use variables S, T, U,. We can
More informationSo let us begin our quest to find the holy grail of real analysis.
1 Section 5.2 The Complete Ordered Field: Purpose of Section We present an axiomatic description of the real numbers as a complete ordered field. The axioms which describe the arithmetic of the real numbers
More informationChapter 4. Probability and Probability Distributions
Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the
More informationProbability and Statistics Vocabulary List (Definitions for Middle School Teachers)
Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence
More informationUndergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics
Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights
More information5. Probability Calculus
5. Probability Calculus So far we have concentrated on descriptive statistics (deskriptiivinen eli kuvaileva tilastotiede), that is methods for organizing and summarizing data. As was already indicated
More informationStatistics in Geophysics: Introduction and Probability Theory
Statistics in Geophysics: Introduction and Steffen Unkel Department of Statistics Ludwig-Maximilians-University Munich, Germany Winter Term 2013/14 1/32 What is Statistics? Introduction Statistics is the
More information6.3 Conditional Probability and Independence
222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted
More informationExtension of measure
1 Extension of measure Sayan Mukherjee Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard.
More informationProbability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..
Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,
More informationCombinatorial Proofs
Combinatorial Proofs Two Counting Principles Some proofs concerning finite sets involve counting the number of elements of the sets, so we will look at the basics of counting. Addition Principle: If A
More informationChapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum a-posteriori Hypotheses
Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum a-posteriori Hypotheses ML:IV-1 Statistical Learning STEIN 2005-2015 Area Overview Mathematics Statistics...... Stochastics
More informationQuestion: What is the probability that a five-card poker hand contains a flush, that is, five cards of the same suit?
ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationSample Induction Proofs
Math 3 Worksheet: Induction Proofs III, Sample Proofs A.J. Hildebrand Sample Induction Proofs Below are model solutions to some of the practice problems on the induction worksheets. The solutions given
More informationMA651 Topology. Lecture 6. Separation Axioms.
MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples
More informationDefinition and Calculus of Probability
In experiments with multivariate outcome variable, knowledge of the value of one variable may help predict another. For now, the word prediction will mean update the probabilities of events regarding the
More informationMathematics for Computer Science/Software Engineering. Notes for the course MSM1F3 Dr. R. A. Wilson
Mathematics for Computer Science/Software Engineering Notes for the course MSM1F3 Dr. R. A. Wilson October 1996 Chapter 1 Logic Lecture no. 1. We introduce the concept of a proposition, which is a statement
More information3. Mathematical Induction
3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)
More informationMathematics for Econometrics, Fourth Edition
Mathematics for Econometrics, Fourth Edition Phoebus J. Dhrymes 1 July 2012 1 c Phoebus J. Dhrymes, 2012. Preliminary material; not to be cited or disseminated without the author s permission. 2 Contents
More informationA Little Set Theory (Never Hurt Anybody)
A Little Set Theory (Never Hurt Anybody) Matthew Saltzman Department of Mathematical Sciences Clemson University Draft: August 21, 2013 1 Introduction The fundamental ideas of set theory and the algebra
More informationGod created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886)
Chapter 2 Numbers God created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886) God created the integers and the rest is the work
More information36 CHAPTER 1. LIMITS AND CONTINUITY. Figure 1.17: At which points is f not continuous?
36 CHAPTER 1. LIMITS AND CONTINUITY 1.3 Continuity Before Calculus became clearly de ned, continuity meant that one could draw the graph of a function without having to lift the pen and pencil. While this
More informationCHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.
CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,
More informationI. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
More informationCollinear Points in Permutations
Collinear Points in Permutations Joshua N. Cooper Courant Institute of Mathematics New York University, New York, NY József Solymosi Department of Mathematics University of British Columbia, Vancouver,
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
More informationPeople have thought about, and defined, probability in different ways. important to note the consequences of the definition:
PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationTHE BANACH CONTRACTION PRINCIPLE. Contents
THE BANACH CONTRACTION PRINCIPLE ALEX PONIECKI Abstract. This paper will study contractions of metric spaces. To do this, we will mainly use tools from topology. We will give some examples of contractions,
More informationCartesian Products and Relations
Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special
More informationGambling Systems and Multiplication-Invariant Measures
Gambling Systems and Multiplication-Invariant Measures by Jeffrey S. Rosenthal* and Peter O. Schwartz** (May 28, 997.. Introduction. This short paper describes a surprising connection between two previously
More informationRepresentation of functions as power series
Representation of functions as power series Dr. Philippe B. Laval Kennesaw State University November 9, 008 Abstract This document is a summary of the theory and techniques used to represent functions
More informationThe Binomial Distribution
The Binomial Distribution James H. Steiger November 10, 00 1 Topics for this Module 1. The Binomial Process. The Binomial Random Variable. The Binomial Distribution (a) Computing the Binomial pdf (b) Computing
More informationINDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS
INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem
More informationMetric Spaces Joseph Muscat 2003 (Last revised May 2009)
1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of
More informationBasic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011
Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011 A. Miller 1. Introduction. The definitions of metric space and topological space were developed in the early 1900 s, largely
More informationBasics of Counting. The product rule. Product rule example. 22C:19, Chapter 6 Hantao Zhang. Sample question. Total is 18 * 325 = 5850
Basics of Counting 22C:19, Chapter 6 Hantao Zhang 1 The product rule Also called the multiplication rule If there are n 1 ways to do task 1, and n 2 ways to do task 2 Then there are n 1 n 2 ways to do
More information4. Continuous Random Variables, the Pareto and Normal Distributions
4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random
More informationLecture 17 : Equivalence and Order Relations DRAFT
CS/Math 240: Introduction to Discrete Mathematics 3/31/2011 Lecture 17 : Equivalence and Order Relations Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last lecture we introduced the notion
More informationChapter 4 Lecture Notes
Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,
More informationSTAT 319 Probability and Statistics For Engineers PROBABILITY. Engineering College, Hail University, Saudi Arabia
STAT 319 robability and Statistics For Engineers LECTURE 03 ROAILITY Engineering College, Hail University, Saudi Arabia Overview robability is the study of random events. The probability, or chance, that
More informationFull and Complete Binary Trees
Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full
More informationa 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
More informationPUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.
PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include
More informationChapter 13 & 14 - Probability PART
Chapter 13 & 14 - Probability PART IV : PROBABILITY Dr. Joseph Brennan Math 148, BU Dr. Joseph Brennan (Math 148, BU) Chapter 13 & 14 - Probability 1 / 91 Why Should We Learn Probability Theory? Dr. Joseph
More informationA Tutorial on Probability Theory
Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,
More informationFUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES
FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied
More informationSums of Independent Random Variables
Chapter 7 Sums of Independent Random Variables 7.1 Sums of Discrete Random Variables In this chapter we turn to the important question of determining the distribution of a sum of independent random variables
More informationSection 1.3 P 1 = 1 2. = 1 4 2 8. P n = 1 P 3 = Continuing in this fashion, it should seem reasonable that, for any n = 1, 2, 3,..., = 1 2 4.
Difference Equations to Differential Equations Section. The Sum of a Sequence This section considers the problem of adding together the terms of a sequence. Of course, this is a problem only if more than
More informationECE302 Spring 2006 HW1 Solutions January 16, 2006 1
ECE302 Spring 2006 HW1 Solutions January 16, 2006 1 Solutions to HW1 Note: These solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics
More informationTOPPER Sample Paper - I. Class : XI MATHEMATICS. Questions. Time Allowed : 3 Hrs Maximum Marks: 100
TOPPER Sample Paper - I Class : XI MATHEMATICS Questions Time Allowed : 3 Hrs Maximum Marks: 100 1. All questions are compulsory.. The question paper consist of 9 questions divided into three sections
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
More informationYou flip a fair coin four times, what is the probability that you obtain three heads.
Handout 4: Binomial Distribution Reading Assignment: Chapter 5 In the previous handout, we looked at continuous random variables and calculating probabilities and percentiles for those type of variables.
More informationExact Nonparametric Tests for Comparing Means - A Personal Summary
Exact Nonparametric Tests for Comparing Means - A Personal Summary Karl H. Schlag European University Institute 1 December 14, 2006 1 Economics Department, European University Institute. Via della Piazzuola
More informationSOLUTIONS TO EXERCISES FOR. MATHEMATICS 205A Part 3. Spaces with special properties
SOLUTIONS TO EXERCISES FOR MATHEMATICS 205A Part 3 Fall 2008 III. Spaces with special properties III.1 : Compact spaces I Problems from Munkres, 26, pp. 170 172 3. Show that a finite union of compact subspaces
More informationNormal distribution. ) 2 /2σ. 2π σ
Normal distribution The normal distribution is the most widely known and used of all distributions. Because the normal distribution approximates many natural phenomena so well, it has developed into a
More informationReal Roots of Univariate Polynomials with Real Coefficients
Real Roots of Univariate Polynomials with Real Coefficients mostly written by Christina Hewitt March 22, 2012 1 Introduction Polynomial equations are used throughout mathematics. When solving polynomials
More informationThe sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].
Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real
More informationLEARNING OBJECTIVES FOR THIS CHAPTER
CHAPTER 2 American mathematician Paul Halmos (1916 2006), who in 1942 published the first modern linear algebra book. The title of Halmos s book was the same as the title of this chapter. Finite-Dimensional
More informationINTRODUCTORY SET THEORY
M.Sc. program in mathematics INTRODUCTORY SET THEORY Katalin Károlyi Department of Applied Analysis, Eötvös Loránd University H-1088 Budapest, Múzeum krt. 6-8. CONTENTS 1. SETS Set, equal sets, subset,
More informationHow To Solve A Minimum Set Covering Problem (Mcp)
Measuring Rationality with the Minimum Cost of Revealed Preference Violations Mark Dean and Daniel Martin Online Appendices - Not for Publication 1 1 Algorithm for Solving the MASP In this online appendix
More information1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
More informationSet theory as a foundation for mathematics
V I I I : Set theory as a foundation for mathematics This material is basically supplementary, and it was not covered in the course. In the first section we discuss the basic axioms of set theory and the
More informationNotes on Probability. Peter J. Cameron
Notes on Probability Peter J. Cameron ii Preface Here are the course lecture notes for the course MAS108, Probability I, at Queen Mary, University of London, taken by most Mathematics students and some
More informationAn example of a computable
An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept
More informationFormal Languages and Automata Theory - Regular Expressions and Finite Automata -
Formal Languages and Automata Theory - Regular Expressions and Finite Automata - Samarjit Chakraborty Computer Engineering and Networks Laboratory Swiss Federal Institute of Technology (ETH) Zürich March
More informationArkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan
Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan 3 Binary Operations We are used to addition and multiplication of real numbers. These operations combine two real numbers
More informationContinued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
More informationSection 6-5 Sample Spaces and Probability
492 6 SEQUENCES, SERIES, AND PROBABILITY 52. How many committees of 4 people are possible from a group of 9 people if (A) There are no restrictions? (B) Both Juan and Mary must be on the committee? (C)
More informationMath 55: Discrete Mathematics
Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What
More informationMEASURE AND INTEGRATION. Dietmar A. Salamon ETH Zürich
MEASURE AND INTEGRATION Dietmar A. Salamon ETH Zürich 12 May 2016 ii Preface This book is based on notes for the lecture course Measure and Integration held at ETH Zürich in the spring semester 2014. Prerequisites
More informationBaltic Way 1995. Västerås (Sweden), November 12, 1995. Problems and solutions
Baltic Way 995 Västerås (Sweden), November, 995 Problems and solutions. Find all triples (x, y, z) of positive integers satisfying the system of equations { x = (y + z) x 6 = y 6 + z 6 + 3(y + z ). Solution.
More informationCHAPTER 5. Number Theory. 1. Integers and Division. Discussion
CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a
More informationPractice with Proofs
Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using
More informationThe Ideal Class Group
Chapter 5 The Ideal Class Group We will use Minkowski theory, which belongs to the general area of geometry of numbers, to gain insight into the ideal class group of a number field. We have already mentioned
More informationMathematical Methods of Engineering Analysis
Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................
More informationLemma 5.2. Let S be a set. (1) Let f and g be two permutations of S. Then the composition of f and g is a permutation of S.
Definition 51 Let S be a set bijection f : S S 5 Permutation groups A permutation of S is simply a Lemma 52 Let S be a set (1) Let f and g be two permutations of S Then the composition of f and g is a
More informationLecture 1: Systems of Linear Equations
MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables
More informationRandom variables, probability distributions, binomial random variable
Week 4 lecture notes. WEEK 4 page 1 Random variables, probability distributions, binomial random variable Eample 1 : Consider the eperiment of flipping a fair coin three times. The number of tails that
More informationProbability. Sample space: all the possible outcomes of a probability experiment, i.e., the population of outcomes
Probability Basic Concepts: Probability experiment: process that leads to welldefined results, called outcomes Outcome: result of a single trial of a probability experiment (a datum) Sample space: all
More informationChapter 4: Probability and Counting Rules
Chapter 4: Probability and Counting Rules Learning Objectives Upon successful completion of Chapter 4, you will be able to: Determine sample spaces and find the probability of an event using classical
More informationc 2008 Je rey A. Miron We have described the constraints that a consumer faces, i.e., discussed the budget constraint.
Lecture 2b: Utility c 2008 Je rey A. Miron Outline: 1. Introduction 2. Utility: A De nition 3. Monotonic Transformations 4. Cardinal Utility 5. Constructing a Utility Function 6. Examples of Utility Functions
More informationA Simpli ed Axiomatic Approach to Ambiguity Aversion
A Simpli ed Axiomatic Approach to Ambiguity Aversion William S. Neilson Department of Economics University of Tennessee Knoxville, TN 37996-0550 wneilson@utk.edu March 2009 Abstract This paper takes the
More informationChapter 3. Cartesian Products and Relations. 3.1 Cartesian Products
Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing
More informationNotes on metric spaces
Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.
More informationCS 598CSC: Combinatorial Optimization Lecture date: 2/4/2010
CS 598CSC: Combinatorial Optimization Lecture date: /4/010 Instructor: Chandra Chekuri Scribe: David Morrison Gomory-Hu Trees (The work in this section closely follows [3]) Let G = (V, E) be an undirected
More informationHow To Find Out How To Calculate A Premeasure On A Set Of Two-Dimensional Algebra
54 CHAPTER 5 Product Measures Given two measure spaces, we may construct a natural measure on their Cartesian product; the prototype is the construction of Lebesgue measure on R 2 as the product of Lebesgue
More information