# P (A) = lim P (A) = N(A)/N,

Save this PDF as:

Size: px
Start display at page:

Download "P (A) = lim P (A) = N(A)/N,"

## Transcription

1 1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we can produce a series of independent trials under identical conditions. This accumulated experience reveals a remarkable regularity of behaviour in the following sense. In each observation, depending on chance, a particular event A either occurs or does not occur. Let n be the total number of observations in this series and let n(a) denote the number of times A occurs. Then the ratio n(a)/n is called the relative frequency of the event A (in this given series of independent and identical trials). It has been empirically observed that the relative frequency becomes stable in the long run. This stability is the basis of probability theory. That is, there exists some constant P (A), called the probability of the event A, such that for large n, P (A) n(a)/n. In fact, the ratio approaches this limit such that P (A) = lim n n(a)/n. Although the preceding definition is certainly intuitively pleasing and should always be kept in mind, it possesses a serious drawback: How do we know that n(a)/n will converge to some constant limiting value that will be the same for each possible sequence of repetitions of the experiment? Proponents of this relative frequency definition of probability usually answer this objection by stating that the convergence of n(a)/n to a constant limiting value is an assumption, or an axiom, of the system. However, to assume that n(a)/n will necessarily converge to some constant value seems to be a very complex assumption, even though this convergence is indeed backed up by many of our empirical experiences. Historically, probability theory began with the study of games of chance. The starting point is an experiment with a finite number of mutually exclusive outcomes which are equiprobable or equally likely because of the nature and setup of the experiment. Let A denote some event associated with certain possible outcomes of the experiment. Then the probability P (A) of the event A is defined as the fraction of the outcomes in which A occurs. That is, P (A) = N(A)/N, where N is the total number of outcomes of the experiment and N(A) is the number of outcomes leading to the occurrence of event A. For example, in tossing a fair or unbiased die, let A be the event of getting an even number of spots. Then N = 6 and N(A) = 3. Hence P (A) = 1/2. This classical definition of probability is essentially circular since the idea of equally likely is the same as that of with equal probability, which has not been defined. The modern treatment of probability theory is purely axiomatic. This means that the probabilities of our events can be prefectly arbitrary, except that they must satisfy a set of simple axioms listed below. The classical theory will correspond to the special case of so-called equiprobable spaces. 1.2 Sample Space and Events. The set Ω of all mutually exclusive outcomes of some given random experiment is called the sample space. An elementary outcome ω, which is an element of Ω, is called a sample point. An event A is assoicated with the sample point ω if, given ωεω, we can always decide whether ω leads 1

2 2 to the occurrence of A. That is, event A occurs if and only if ωεa. Hence event A consists of a set of outcomes or, in other words, simply a subset of the underlying sample space Ω. The event {ω} consisting of a single sample point ωεω is called an elementary event. The empty set φ and Ω itself are events; φ is sometimes called the impossible event, and Ω the certain or sure event. Two events A and B are idential if A occurs iff B occurs. That is, they lead to the same subset of Ω. Two events A and B are mutually exclusive or incompatible if A and B cannot occur simultaneously. That is, A B = φ. If the sample space Ω is finite or countably infinite, then every subset of Ω is an event. On the other hand, if Ω is uncountable, we shall be concerned with the σ-algebra E of subsets of Ω, which is closed under the operations of set union, set intersection, set difference and set complement. 1.3 Axioms of Probability. Let Ω be a sample space, let E be the class of events, and let P be a real-valued function defined on E. Then P is called a probability function, and P (A) is called the probability of the event A if the following axioms hold: Axiom 1. For every event A, 0 P (A) 1. Axiom 2. P (Ω) = 1. Axiom 3. If A 1, A 2,... is a sequence of mutually exclusive events, then P (A 1 A 2... ) = P (A 1 ) + P (A 2 ) +. The assumption of the existence of a set function P, defined on the events of a sample space and satisfying Axioms 1, 2 and 3 constitutes the modern mathematical approach to probability theory. The axioms are natual and in accordance with our intuitive concept of probability as related to chance and randomness. Furthermore, using these axioms, we shall be able to prove that the relative frequency of a specific event A will equal P (A) with probability 1 under a series of independent and identical trials of an experiment. This is the well known result of the strong law of large numbers in probability theory. A few easy and useful results concerning the probabilities of events follow. Proposition 1. P (φ) = 0. Proposition 2. P (A) = 1 P (A), where A is the complement of A. Proposition 3. If A B, then P (A) P (B). Proposition 4. P (A B) = P (A) + P (B) P (A B). 1.4 Additive Laws. For a sequence of n mutually exclusive events A 1, A 2,..., A n, since A i A j = φ for i j, we have the additive law for mutually exclusive events n P ( A k ) = P (A k ). For a set of arbitrary events, there is the more general result stating that the probability of the union of n events equals the sum of the probabilities of these events taken one at a time, minus the sum of the probabilities of the events taken two at a time, plus the sum of the probabilities of these events taken three at a time, and so on.

3 3 Theorem. Given n events A 1, A 2,..., A n in Ω, let P 1 = P 2 = P 3 = n P (A i ) i=1 1 i<j n 1 i<j<k n.. P (A i A j ) P n = P (A 1 A 2 A n ) P (A i A j A k ) Then P ( A k ) = P 1 P 2 + P 3 + ( 1) n+1 P n. The proof is by induction on the number of events. For n = 2, we readily have P (A 1 A 2 ) = P (A 1 ) + P (A 2 ) P (A 1 A 2 ). Suppose the statement holds for n 1 events A 2, A 3,..., A n, i.e. Hence P ( P ( A k ) = k=2 A 1 A k ) = k=2 2 i n 2 i n P (A i ) P (A 1 A i ) 2 i<j n 2 i<j n Noting that n A k = A 1 ( n k=2 A k), we have P (A i A j ) + P (A 1 A i A j ) + P ( A k ) = P (A 1 ) + P ( A k ) P ( A 1 A k ) k=2 = P (A 1 ) + 2 i n 2 j n = P 1 P 2 + P 3 P (A i ) P (A 1 A j ) + k=2 2 i<j n 2 j<k n P (A i A j ) + 2 i<j<k n 2 i<j<k n 2 i<j<k n P (A 1 A j A k ) P (A i A j A k ) P (A 1 A i A j A k ) P (A i A j A k ) Example. Suppose n students have n identical raincoats all put together. Each student selects a raincoat at random. 1. What is the probability that at least one raincoat ends up with its original owner? 2. What is the probability that exactly k students select their own raincoats? Let A be the event that at least one raincoat ends up with its original owner. Let A k be the event that the k th student selects his own (the k th ) raincoat. Then A = n A k.

4 4 Every outcome of this experiment consisting of randomly selecting the raincoats is described by a permutation (i 1, i 2,..., i n ), where i k is the index of the raincoat selected by the k th student. For m n, the event A k1 A k2 A km occurs whenever i k1 = k 1, i k2 = k 2,..., i km = k m and the other n m indices of i k take the remaining n m values in any order. (In words, students k 1, k 2,..., k m select their own raincoats.) Therefore P (A k1 A k2 A km ) = N(A k 1 A k2 A km ) N = (n m)! n! for any set of m indices k 1, k 2,..., k m. There are exactly C n m distinct events of the type A k1 A k2 A km. It follows that P m = P (A k1 A k2 1 k 1 <k 2 < <k m n A km ) = Cm n (n m)! = 1 n! m!. Hence P (A) = P ( A k ) = P 1 P ( 1) n+1 P n = 1 1 2! + 1 3! + 1 ( 1)n+1 n!. (This is seen to be the partial sum of the power series expansion of 1 e 1.) Thus, for large n, P (A) 1 e The probability that none of the students selects his own raincoat equals P (A) = 1 P (A) = 1 2! 1 3! + + ( 1)n 1 n!, which is the well-known derangement number of order n. For large n, this is approximately equal to e (An incorrect intuition might leads us to think that P (A) would go to 1 as n goes to infinity.) To obtain the probability of event B such that exactly k students select their own raincoats, we first note that the number of ways in which a particular set of k students selecting their own is equal to the number of ways in which none of the other n k students selects his own raincoat. This is equal to (n k)! [derangement number of order n k] =(n k)! [ 1 2! 1 3! + + ( 1)n k 1 (n k)! ] As there are C n k possible selections of a group of k students, it follows that there are C n k (n k)! [ 1 2! 1 3! + + ( 1)n k 1 (n k)! ] ways in which exactly k of the students select their own raincoats. Thus P (B) = Ck n (n k)! [ 1 2! 1 3! ( 1)n k (n k)! ]/n! n k i=0 = ( 1)i /i!, k! which for n large is approximately e 1 /k!. The values e 1 /k!, k = 0, 1,... are of importance as they represent the values associated with the Possion distribution with parameter equal to unity. This will be elaborated upon when the Poisson approximation is discussed.

5 5 2.1 Continuity Property of Probability Function. Theorem. If A 1, A 2,... is an increasing sequence of events such that A 1 A 2, then denoting lim n A n = k A k, we have P ( k A k ) = P ( lim n A n) = lim n P (A n). Let us define events B 1, B 2,... as follows. B 1 A 1, B 2 A 2 A 1 = A 2 B 1, B 3 A 3 A 2 = A 3 (B 1 B 2 ),..., B n = A n n 1 B k,.... Thus (a) B 1, B 2,... are mutually exclusive, (b) k B k = k A k, and (c) A n = n B k A 3 A 2 A 1 B 1 B 2 B 3 Hence P ( k A k ) = P ( k B k ) = k P (B k ) = lim n = lim P ( n B k ) = lim P (A n). n n n P (B k ) Theorem. If A 1, A 2,... is a decreasing sequence of events such that A 1 A 2, then denoting lim n A n = k A k, we have P ( k A k ) = P ( lim n A n) = lim n P (A n). We have A 1 A 2. Hence P ( k A k ) = P (Ω k A k ) = 1 P ( k A k ) = 1 lim n P (A n ) = lim n [1 P (A n )] = lim n P (A n ). For mutually exclusive events A 1, A 2,..., we know that P ( k A k ) = k P (A k). For arbitrary events, there is the following. Theorem. P ( k A k ) k P (A k) for arbitrary events A 1, A 2,.... Again we define B n = A n n 1 B k as before. Then we have k A k = k B k and B k A k for all k. Hence P ( k A k ) = P ( k B k ) = k P (B k) k P (A k).

6 6 Remark. It is not only for increasing or decreasing sequence of events that we can define a limit. In general, for any sequence of events {A n, n 1}, define (a) lim sup n A n = n=1 i=n A i. [Interpretation: This consists of all points that are contained in an infinite number of the events A n, n 1.] (b) lim inf n A n = n=1 i=n A i. [Interpretation: This consists of all points that are contained in all but a finite number of the events A n, n 1.] Definition. We say that lim n A n exists if lim sup A n = lim inf A n ( lim A n). n n n Theorem. If lim n A n exists, then P (lim n A n ) = lim n P (A n ). (See S.M. Ross, A First Course in Probability, pp ) 2.2 Recurrence of Events. Theorem. (First Borel-Cantelli Lemma) Given a sequence of events A 1, A 2,... with probabilities P (A k ) = p k, k = 1, 2,.... Suppose k p k < converges. Then, with probability 1 only finitely many of the events A 1, A 2,... occur. Let B be the event that infinitely many of the events A 1, A 1,... occur. Let B n = k n A k, i.e. at least one of the events A n, A n+1,... occurs. Then B occurs iff B n occurs for every n = 1, 2,... (That is ωεb iff ωεb 1 B 2 ). Hence B = n B n = n ( k n A k ). Now as B 1 B 2 is a decreasing sequence, we have P (B) = P ( n B n ) = lim n P (B n). But P (B n ) = P ( k n A k ) k n P (A k) = k n p k approaches zero as n approaches. Hence P (B) = lim n P (B n ) = 0, or P (B) = 1 P (B) = 1 as asserted. Theorem. (Second Borel-Cantelli Lemma) Given a sequence of independent events A 1, A 2,... with probabilities P (A k ) = p k, k = 1, 2,.... Suppose k p k = diverges. Then, with probability 1 infinitely many of the events A 1, A 2,... occur. Let B n = k n A k and B = n B n = n ( k n A k ), so that B occurs iff each B n occur (i.e. infintiely many of A 1, A 2,..., occur). Taking complements yields B n = k n A k, B = n B n = n ( k n A k ).

7 7 Now B n = k n A k n+m k=n A k, m = 0, 1, 2,.... Therefore n+m P (B n ) P ( A k ) = P (A n )P (A n+1 ) P (A n+m ) k=n = (1 p n )(1 p n+1 ) (1 p n+m ) n+m e pn e pn+1 e pn+m = exp( p k ), where we used the fact that 1 x e x for x 0. But n+m k=n p k approaches as m approaches, hence P (B n ) = lim m P (B n) lim n+m exp( m k=n k=n Finally P (B) n P (B n) = 0 implies that P (B) = 1 P (B) = 1. p k ) = 0, n = 1, 2,. 3.1 Conditional Probability as a Probability Function. In terms of the dependency of two events A and B, we have: (a) A and B are mutually exclusive, i.e.ab = φ and P (AB) = 0. Hence A never occurs if B does. (b) A has complete dependence on B, i.e. A B and P (A) P (B). Hence A always occurs if B does. (c) A and B are independent, i.e. P (AB) = P (A)P (B). Here the outcomes of A and B are not influenced by that of the other. More generally, we define the conditional probability of A given B (with P (B) > 0) by P (A B) = P (AB)/P (B). Thus in terms of conditional probabilities: (a) P (A B) = 0 if A and B are mutually exclusive. (b) P (A B) = 1 if A is implied by B. (c) P (A B) = P (A) if A and B are independent. Conditional probabilities satisfy all the properties of ordinary probabilities. As a set function, P (A B) satisfies the three axioms while restricting the set of possible outcomes (sample space) to those that are in B. Theorem. Suppose B Ω is not an impossible event, i.e. P (B) > 0. (a) 0 P (A B) 1. (b) P (Ω B) = 1. (c) If A 1, A 2,... are mutually exclusive events, then P ( k A k B) = k P (A k B). (a) We have 0 P (AB)/P (B) P (B)/P (B) = 1. (b) P (ΩB)/P (B) = P (B)/P (B) = 1. (c) We have A 1 B, A 2 B,... being mutually exclusive, hence P ( k A k B)/P (B) = k P (A kb)/p (B) = k P (A k B).

8 8 The upshot is that all properties about ordinary probability function are true for conditional probability function. For example, it easily follows that P (A 1 A 2 B) = P (A 1 B) + P (A 2 B) P (A 1 A 2 B); and P (A 1 B) P (A 2 B) if A 1 A Total Probability Formula and Conditioning Argument. We say that B 1, B 2,... form a full set of events if they are mutually exclusive and collectively exhaustive, i.e. one and only one of B 1, B 2,... always occur and k B k = Ω. A powerful computation tool for the probability of an event is by way of the conditioning argument: P (A) = P (A B k )P (B k ). k The argument is simple. As A = AΩ = k AB k, the total probability formula yields P (A) = P ( k AB k ) = k P (AB k ), since B 1, B 2,... form a full set. In fact, many well-knwon formulas in probability theory are direct consequences of this. Proposition. (Baye s Formula) Suppose B 1, B 2,... form a full set of events, then P (B k A) = P (B ka) P (A) = P (B k)p (A B k ) k P (B k)p (A B k ). Example. (The Gambler s Ruin) Consider the game of calling head or tail on a toss of a fair coin. A correct (incorrect) call wins (loses) a dollar. Suppose the gambler s initial capital is k dollars and the game continues until he either wins an amount of m dollars, stipulated in advance, or else loses all his capital and is ruined. We would like to calculate the probability p(k) that the player will be ruined, starting with 0 < k < m dollars. Let B 1 be the event that the player wins the first call, and B 2 be that he loses the first call. Hence B 1 and B 2 form a full set for one play of the game. Now P (B 1 ) = P (B 2 ) = 1/2. Also P (Ruin B 1 ) = p(k + 1) and P (Ruin B 2 ) = p(k 1). Hence by conditioning on the first call, we have P (Ruin) = P (Ruin B 1 )P (B 1 ) + P (Ruin B 2 )P (B 2 ). That is, p(k) = p(k + 1)/2 + p(k 1)/2, 1 k m 1. Solution to this set of difference equations has the form of p(k) = c 1 + c 2 k. Using the boundary conditions that p(0) = 1 and p(m) = 0, we have c 1 = 1 and c 2 = 1/m. Hence p(k) = 1 k/m, 0 k m, is the required probability. [For general winning probability 0 < p < 1, p(k) = 1 (p/q)m k 1 (p/q) m in Probability, pp ]. See S.M. Ross, A First Course

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Basic Probability Concepts

page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### Elements of probability theory

2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted

### Math/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Chapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum a-posteriori Hypotheses

Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum a-posteriori Hypotheses ML:IV-1 Statistical Learning STEIN 2005-2015 Area Overview Mathematics Statistics...... Stochastics

### JANUARY TERM 2012 FUN and GAMES WITH DISCRETE MATHEMATICS Module #9 (Geometric Series and Probability)

JANUARY TERM 2012 FUN and GAMES WITH DISCRETE MATHEMATICS Module #9 (Geometric Series and Probability) Author: Daniel Cooney Reviewer: Rachel Zax Last modified: January 4, 2012 Reading from Meyer, Mathematics

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Basics of Probability

Basics of Probability August 27 and September 1, 2009 1 Introduction A phenomena is called random if the exact outcome is uncertain. The mathematical study of randomness is called the theory of probability.

### Examples of infinite sample spaces. Math 425 Introduction to Probability Lecture 12. Example of coin tosses. Axiom 3 Strong form

Infinite Discrete Sample Spaces s of infinite sample spaces Math 425 Introduction to Probability Lecture 2 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan February 4,

### Chap2: The Real Number System (See Royden pp40)

Chap2: The Real Number System (See Royden pp40) 1 Open and Closed Sets of Real Numbers The simplest sets of real numbers are the intervals. We define the open interval (a, b) to be the set (a, b) = {x

### Lecture 4: Probability Spaces

EE5110: Probability Foundations for Electrical Engineers July-November 2015 Lecture 4: Probability Spaces Lecturer: Dr. Krishna Jagannathan Scribe: Jainam Doshi, Arjun Nadh and Ajay M 4.1 Introduction

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### Probability - Part I. Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty.

Probability - Part I Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty. Definition : The sample space (denoted S) of a random experiment

### A natural introduction to probability theory. Ronald Meester

A natural introduction to probability theory Ronald Meester ii Contents Preface v 1 Experiments 1 1.1 Definitions and examples........................ 1 1.2 Counting and combinatorics......................

### Combinatorics: The Fine Art of Counting

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

### This chapter describes set theory, a mathematical theory that underlies all of modern mathematics.

Appendix A Set Theory This chapter describes set theory, a mathematical theory that underlies all of modern mathematics. A.1 Basic Definitions Definition A.1.1. A set is an unordered collection of elements.

### Climbing an Infinite Ladder

Section 5.1 Climbing an Infinite Ladder Suppose we have an infinite ladder and the following capabilities: 1. We can reach the first rung of the ladder. 2. If we can reach a particular rung of the ladder,

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### Toss a coin twice. Let Y denote the number of heads.

! Let S be a discrete sample space with the set of elementary events denoted by E = {e i, i = 1, 2, 3 }. A random variable is a function Y(e i ) that assigns a real value to each elementary event, e i.

### Math212a1010 Lebesgue measure.

Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.

### Probability OPRE 6301

Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.

### Lecture 13: Martingales

Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of

### PROBABILITY NOTIONS. Summary. 1. Random experiment

PROBABILITY NOTIONS Summary 1. Random experiment... 1 2. Sample space... 2 3. Event... 2 4. Probability calculation... 3 4.1. Fundamental sample space... 3 4.2. Calculation of probability... 3 4.3. Non

### Lecture 2 : Basics of Probability Theory

Lecture 2 : Basics of Probability Theory When an experiment is performed, the realization of the experiment is an outcome in the sample space. If the experiment is performed a number of times, different

### 10.2 Series and Convergence

10.2 Series and Convergence Write sums using sigma notation Find the partial sums of series and determine convergence or divergence of infinite series Find the N th partial sums of geometric series and

### Lecture 2: Probability

Lecture 2: Probability Assist. Prof. Dr. Emel YAVUZ DUMAN MCB1007 Introduction to Probability and Statistics İstanbul Kültür University Outline 1 Introduction 2 Sample Spaces 3 Event 4 The Probability

### Chapter 6 Finite sets and infinite sets. Copyright 2013, 2005, 2001 Pearson Education, Inc. Section 3.1, Slide 1

Chapter 6 Finite sets and infinite sets Copyright 013, 005, 001 Pearson Education, Inc. Section 3.1, Slide 1 Section 6. PROPERTIES OF THE NATURE NUMBERS 013 Pearson Education, Inc.1 Slide Recall that denotes

### Our goal first will be to define a product measure on A 1 A 2.

1. Tensor product of measures and Fubini theorem. Let (A j, Ω j, µ j ), j = 1, 2, be two measure spaces. Recall that the new σ -algebra A 1 A 2 with the unit element is the σ -algebra generated by the

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### 6.042/18.062J Mathematics for Computer Science. Expected Value I

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

### Chapter 3: The basic concepts of probability

Chapter 3: The basic concepts of probability Experiment: a measurement process that produces quantifiable results (e.g. throwing two dice, dealing cards, at poker, measuring heights of people, recording

### Mathematical Induction

Mathematical Induction Victor Adamchik Fall of 2005 Lecture 2 (out of three) Plan 1. Strong Induction 2. Faulty Inductions 3. Induction and the Least Element Principal Strong Induction Fibonacci Numbers

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### Jan 17 Homework Solutions Math 151, Winter 2012. Chapter 2 Problems (pages 50-54)

Jan 17 Homework Solutions Math 11, Winter 01 Chapter Problems (pages 0- Problem In an experiment, a die is rolled continually until a 6 appears, at which point the experiment stops. What is the sample

### Finite Sets. Theorem 5.1. Two non-empty finite sets have the same cardinality if and only if they are equivalent.

MATH 337 Cardinality Dr. Neal, WKU We now shall prove that the rational numbers are a countable set while R is uncountable. This result shows that there are two different magnitudes of infinity. But we

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

### NOTES ON MEASURE THEORY. M. Papadimitrakis Department of Mathematics University of Crete. Autumn of 2004

NOTES ON MEASURE THEORY M. Papadimitrakis Department of Mathematics University of Crete Autumn of 2004 2 Contents 1 σ-algebras 7 1.1 σ-algebras............................... 7 1.2 Generated σ-algebras.........................

### Probability Review. ICPSR Applied Bayesian Modeling

Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

### Problem Set. Problem Set #2. Math 5322, Fall December 3, 2001 ANSWERS

Problem Set Problem Set #2 Math 5322, Fall 2001 December 3, 2001 ANSWERS i Problem 1. [Problem 18, page 32] Let A P(X) be an algebra, A σ the collection of countable unions of sets in A, and A σδ the collection

### 1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let

Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Chapter 4 Lecture Notes

Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,

### 3. Recurrence Recursive Definitions. To construct a recursively defined function:

3. RECURRENCE 10 3. Recurrence 3.1. Recursive Definitions. To construct a recursively defined function: 1. Initial Condition(s) (or basis): Prescribe initial value(s) of the function.. Recursion: Use a

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### An approach to Calculus of Probabilities through real situations

MaMaEuSch Management Mathematics for European Schools http://www.mathematik.unikl.de/ mamaeusch An approach to Calculus of Probabilities through real situations Paula Lagares Barreiro Federico Perea Rojas-Marcos

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Math 317 HW #5 Solutions

Math 317 HW #5 Solutions 1. Exercise 2.4.2. (a) Prove that the sequence defined by x 1 = 3 and converges. x n+1 = 1 4 x n Proof. I intend to use the Monotone Convergence Theorem, so my goal is to show

### A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes.

Chapter 7 Probability 7.1 Experiments, Sample Spaces, and Events A (random) experiment is an activity with observable results. The sample space S of an experiment is the set of all outcomes. Each outcome

### Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..

Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,

### The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics.

Elementary probability theory The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics. 1.1 Experiments, outcomes and sample

### CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE

CARDINALITY, COUNTABLE AND UNCOUNTABLE SETS PART ONE With the notion of bijection at hand, it is easy to formalize the idea that two finite sets have the same number of elements: we just need to verify

### Basic concepts in probability. Sue Gordon

Mathematics Learning Centre Basic concepts in probability Sue Gordon c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Set Notation You may omit this section if you are

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Chapter 4. Probability and Probability Distributions

Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

### IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Tuesday, September 11 Normal Approximations and the Central Limit Theorem

IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Tuesday, September 11 Normal Approximations and the Central Limit Theorem Time on my hands: Coin tosses. Problem Formulation: Suppose that I have

### MATHEMATICS 117/E-217, SPRING 2012 PROBABILITY AND RANDOM PROCESSES WITH ECONOMIC APPLICATIONS Module #5 (Borel Field and Measurable Functions )

MATHEMATICS 117/E-217, SPRING 2012 PROBABILITY AND RANDOM PROCESSES WITH ECONOMIC APPLICATIONS Module #5 (Borel Field and Measurable Functions ) Last modified: May 1, 2012 Reading Dineen, pages 57-67 Proof

### Set theory, and set operations

1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### How to build a probability-free casino

How to build a probability-free casino Adam Chalcraft CCR La Jolla dachalc@ccrwest.org Chris Freiling Cal State San Bernardino cfreilin@csusb.edu Randall Dougherty CCR La Jolla rdough@ccrwest.org Jason

### Chapter 15. Definitions: experiment: is the act of making an observation or taking a measurement.

MATH 11008: Probability Chapter 15 Definitions: experiment: is the act of making an observation or taking a measurement. outcome: one of the possible things that can occur as a result of an experiment.

### Chapter 1. Sigma-Algebras. 1.1 Definition

Chapter 1 Sigma-Algebras 1.1 Definition Consider a set X. A σ algebra F of subsets of X is a collection F of subsets of X satisfying the following conditions: (a) F (b) if B F then its complement B c is

### Measure Theory. 1 Measurable Spaces

1 Measurable Spaces Measure Theory A measurable space is a set S, together with a nonempty collection, S, of subsets of S, satisfying the following two conditions: 1. For any A, B in the collection S,

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Gambling Systems and Multiplication-Invariant Measures

Gambling Systems and Multiplication-Invariant Measures by Jeffrey S. Rosenthal* and Peter O. Schwartz** (May 28, 997.. Introduction. This short paper describes a surprising connection between two previously

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

### Notes on counting finite sets

Notes on counting finite sets Murray Eisenberg February 26, 2009 Contents 0 Introduction 2 1 What is a finite set? 2 2 Counting unions and cartesian products 4 2.1 Sum rules......................................

### 3 σ-algebras. Solutions to Problems

3 σ-algebras. Solutions to Problems 3.1 3.12 Problem 3.1 (i It is clearly enough to show that A, B A = A B A, because the case of N sets follows from this by induction, the induction step being A 1...

### Sets and functions. {x R : x > 0}.

Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.

### 2 Elementary probability

2 Elementary probability This first chapter devoted to probability theory contains the basic definitions and concepts in this field, without the formalism of measure theory. However, the range of problems

### 6.3 Conditional Probability and Independence

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

### Sigma Field Notes for Economics 770 : Econometric Theory

1 Sigma Field Notes for Economics 770 : Econometric Theory Jonathan B. Hill Dept. of Economics, University of North Carolina 0. DEFINITIONS AND CONVENTIONS The following is a small list of repeatedly used

### F. ABTAHI and M. ZARRIN. (Communicated by J. Goldstein)

Journal of Algerian Mathematical Society Vol. 1, pp. 1 6 1 CONCERNING THE l p -CONJECTURE FOR DISCRETE SEMIGROUPS F. ABTAHI and M. ZARRIN (Communicated by J. Goldstein) Abstract. For 2 < p

### Assigning Probabilities

What is a Probability? Probabilities are numbers between 0 and 1 that indicate the likelihood of an event. Generally, the statement that the probability of hitting a target- that is being fired at- is

### FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES

FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied

### Math 317 HW #7 Solutions

Math 17 HW #7 Solutions 1. Exercise..5. Decide which of the following sets are compact. For those that are not compact, show how Definition..1 breaks down. In other words, give an example of a sequence

### Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University

Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments

### 13 Infinite Sets. 13.1 Injections, Surjections, and Bijections. mcs-ftl 2010/9/8 0:40 page 379 #385

mcs-ftl 2010/9/8 0:40 page 379 #385 13 Infinite Sets So you might be wondering how much is there to say about an infinite set other than, well, it has an infinite number of elements. Of course, an infinite

### Mathematical Induction

Mathematical Induction MAT30 Discrete Mathematics Fall 016 MAT30 (Discrete Math) Mathematical Induction Fall 016 1 / 19 Outline 1 Mathematical Induction Strong Mathematical Induction MAT30 (Discrete Math)

### Definition of Random Variable A random variable is a function from a sample space S into the real numbers.

.4 Random Variable Motivation example In an opinion poll, we might decide to ask 50 people whether they agree or disagree with a certain issue. If we record a for agree and 0 for disagree, the sample space

### Section 1.3 P 1 = 1 2. = 1 4 2 8. P n = 1 P 3 = Continuing in this fashion, it should seem reasonable that, for any n = 1, 2, 3,..., = 1 2 4.

Difference Equations to Differential Equations Section. The Sum of a Sequence This section considers the problem of adding together the terms of a sequence. Of course, this is a problem only if more than

### Sequences of Functions

Sequences of Functions Uniform convergence 9. Assume that f n f uniformly on S and that each f n is bounded on S. Prove that {f n } is uniformly bounded on S. Proof: Since f n f uniformly on S, then given

### Discrete probability and the laws of chance

Chapter 8 Discrete probability and the laws of chance 8.1 Introduction In this chapter we lay the groundwork for calculations and rules governing simple discrete probabilities. These steps will be essential

### A Curious Case of a Continued Fraction

A Curious Case of a Continued Fraction Abigail Faron Advisor: Dr. David Garth May 30, 204 Introduction The primary purpose of this paper is to combine the studies of continued fractions, the Fibonacci

### Limits and convergence.

Chapter 2 Limits and convergence. 2.1 Limit points of a set of real numbers 2.1.1 Limit points of a set. DEFINITION: A point x R is a limit point of a set E R if for all ε > 0 the set (x ε,x + ε) E is

### This asserts two sets are equal iff they have the same elements, that is, a set is determined by its elements.

3. Axioms of Set theory Before presenting the axioms of set theory, we first make a few basic comments about the relevant first order logic. We will give a somewhat more detailed discussion later, but

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### REAL ANALYSIS I HOMEWORK 2

REAL ANALYSIS I HOMEWORK 2 CİHAN BAHRAN The questions are from Stein and Shakarchi s text, Chapter 1. 1. Prove that the Cantor set C constructed in the text is totally disconnected and perfect. In other

### M2L1. Random Events and Probability Concept

M2L1 Random Events and Probability Concept 1. Introduction In this lecture, discussion on various basic properties of random variables and definitions of different terms used in probability theory and

### Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics

Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights

### An example of a computable

An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept

### 3. Examples of discrete probability spaces.. Here α ( j)

3. EXAMPLES OF DISCRETE PROBABILITY SPACES 13 3. Examples of discrete probability spaces Example 17. Toss n coins. We saw this before, but assumed that the coins are fair. Now we do not. The sample space

### Double Sequences and Double Series

Double Sequences and Double Series Eissa D. Habil Islamic University of Gaza P.O. Box 108, Gaza, Palestine E-mail: habil@iugaza.edu Abstract This research considers two traditional important questions,

### Lecture 3. Mathematical Induction

Lecture 3 Mathematical Induction Induction is a fundamental reasoning process in which general conclusion is based on particular cases It contrasts with deduction, the reasoning process in which conclusion

### CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.

CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,

### Introduction to Probability. Experiments. Sample Space. Event. Basic Requirements for Assigning Probabilities. Experiments

Introduction to Probability Experiments These are processes that generate welldefined outcomes Experiments Counting Rules Combinations Permutations Assigning Probabilities Experiment Experimental Outcomes