# Set theory, and set operations

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we discuss the probability of two types of events: (1 infinitely repeated operations such as infinite sequences of coin tosses (2 infinitely fine operations such as drawing uniformly from the unit interval. In both these cases we have to worry about infinite events so issues of countability, limits, and measure occur. Measure theoretic probability is needed to understand when and how our intuition about discrete events carries over to both these cases. It also gives us a common framework for both of the above events. Why need for care? One example is order of integration which for Bayesian statistics is very important since we condition and marginalize extensively If we first integrate y and then x: 1 5. If we first integrate x and then y: R = {0 x 2, 0 y 1} f(x, y = xy(x2 y 2 (x 2 + y 2, for(x, y (0, 0 3? = f(x, ydxdy R Another example is the Dirichlet process. The following is a simple way of constructing a Dirichlet process: 1. Draw X 1,..., X N iid Poi(λ 2. Normalize Z i = Xi Pi Xi. What is the distribution underlying this random process or algorithm? Is there a sense of convergence of the outputs of this algorithm? Important ideas Law of large numbers: Given X 1,..., X n of iid random variables we state that what does convergence ( mean? 1 n n X i µ, Central limit theorem: 1 n n i=1 Z n = X i µ No(0, 1, σ what does convergence ( mean? What is convergence in distribution? Law of the iterated logarithm: what is lim sup? Notation lim sup n i=1 Ω: set of possible outcomes of some experiment. ( n i=1 X i nµ 2nσ2 log log n = 1,

2 2 ω Ω: one of the outcomes of the experiment. A Ω A is a subset of Ω; A is true if ω A. 2 Ω : All subsets of Ω called th power set P(Ω Operations Complement; A c = not A = {w : w / A} Union over arbitrary index set Intersection over arbitrary index set Set difference; symmetric difference Relations Containment: A B : ω A ω B Disjoint: A B = Equality: A = B : ω A iff ω B De Morgan s Law: A = {w : w A for at least one }, A B = A or B (or perhaps both. A = {w : w A for all }, A B = AB = both A and B. A \ B = A B c A B = (A B c (A c B. ( c A = (A c ( c A = (A c Montone sequence of sets: A n A means that A 1 A 2... and lim A n = n=1 A n. A n A means that A 1 A 2... and lim A n = A n A means that A 1 A 2... and lim A n = n=1 A n. n=1 A n. A n A means that A 1 A 2... and lim A n = n=1 A n. Limits of sets Infinitely often (io: lim sup A n m 1 n m A n. Also called infinitely often as A n occurs infinitely often as n. The event lim sup A n occurs if and only if infinitely many of the A n occur. Almost all (aa: lim inf A n m 1 n m A n. Also called almost all since A n occurs with at most finitely many exceptions. {lim inf A n } {lim sup A n } A n = {n, n + 1,...} A n = {1,..., n} A 2n = {2, 4, 6,...} lim sup A n = lim inf A n = lim sup A n = lim inf A n = N lim sup A n = N, lim inf A n = Convergence: A n A lim inf A n = lim sup A n = A. Indicator functions Indicator functions allow for Boolean algebra to be turned into ordinary algebra.

3 3 I A (x = { 1 if x A 0 otherwise. The operations of unions and intersections correspond to pointwise maxima ( or minima ( I Ai (x = i I Ai (x I Ai (x = i I Ai (x. I A c(x = 1 I A (x I A\B (x = max(0, I A (x I B (x I A B (x = I A (x I B (x. ( n ( n n A i B i (A i B i I Ai i=1 i=1 i=1 i i I Bi max I Ai I Bi. i Revisit lim sup: lim sup of a set is really much like lim sup of a function I lim supn A n lim sup = lim sup I An. n lim lim inf lim ( sup ( m n x m inf x m m n lim inf x n lim sup x n. x n x lim inf x n = lim sup x n = x. Some sets are bigger than others The cardinality of a set Ω is the number of elements in the set. Theorem (Cantor For any set Ω and power set P(Ω, Ω < P(Ω. Example Ω = N an infinite but countable set P(Ω uncountable R uncountable Q the rationals are countable. Fields and algebras Definition (algebra Let F be a collection of subsets of Ω. F is called a field (algebra if Ω F and F is closed under complementation and finite union: 1. (i Ω F 2. (ii A F A c F 3. (iii A 1,..., A n F n A j F. We could replace closure under finite unions with closure under finite intersections (due to de Morgan s law n A 1,..., A n F A j F. Definition (σ-algebra Let F be a collection of subsets of Ω. F is called a field (algebra if Ω F and F is closed under complementation and countable unions:

4 4 1. (i Ω F 2. (ii A F A c F 3. (iii A 1,..., A n F n A j F 4. (iv A 1,..., A n,... F A j F An algebra is a σ-algebra but not vice versa. Example Ω = R, A n = (a, b]. F is the collection of all finite disjoint unions of (a, b]. F is an algebra. F is not a σ-algebra. (0, n 1 ] = (0, 1. n=1 Definition (Borel set Let Ω = R and B 0 be the field of right-semiclosed intervals. Then σ(b 0 is the Borel σ-algebra of R. This idea can be extended to metric spaces such as R d or the space of continuous functions. Probability spaces The idea of a probability space is to assign a measure or nonnegative number to subsets of of Ω. Definition (Probability space A probability space is a triple (Ω, F, P where P is a function (probability measure P : F [0, 1] such that 1. (i P(A 0, A F 2. ii P(Ω = 1 3. iii if A j F are disjoint P = A j P(A j. Properties of probability spaces 1. (i monotonicity: if A B then P(A < P(B 2. (ii σ-subadditive: for A n, n 1 3. (iii continuity: if A n A and A n F then P(A n P(A if A n A and A n F then P(A n P(A ( P A n P(A n n=1 4. (iv inclusion-exclusion: for A 1,..., A n (related to Euler characteristic in topology n P n = P(A j P(A i A j + P(A i A j A k ( 1 n+1 P(A 1 A n A j 1 i<j n n=1 1 i<j<k n Example Two events that are disjoint but the sum of the probability of the union is equal to the probability of the sum. A = (0,.5] and B = [.5, 1 with P(A =.5 and P(B =.5. P(A B = 1 = P(A + P(B and A B = {.5}.

5 5 Lemma (Fatou For A n F for n 1 1. (i 2. (ii If A n A then P(A n P(A. P(lim inf A n lim inf P(A n lim sup P(A n P(lim sup A n For a probability measure µ and a sequence of functions f n (think f n = I An (i is sometimes written with functional notation lim inf f n dµ lim inf f n dµ. A typical use of Fatous lemma is the following. Suppose we have f n f and sup n 1 f n K <. Then f n f, and by Fatou s lemma f K. (What does the above have to do with any problem of inference? Example Discrete probability measure on the real numbers whose support is the real line. The support of a discrete probability mesure is not necessarily just the set of points at which it places mass, two points really really close may not be individually distinguished. Let r 1, r 2,.. be an enumeration of the rational numbers (how would one enumerate them? and let P place mass 1/2 n at r n. P is discrete but has support on the line since for any x and any ε > 0 P({x ε, x + ε} > 0, since the interval contains at leasnt one rational number. Does this example have any uses in non-parametric statistical models? Definition (Distribution function A function F : R [0, 1] is a probability distribution function (df with F (x = P((, x], x R, if F 1. is continuous 2. is monotone and non-decreasing 3. has limits at ± F ( := lim F (x = 1 x F ( := lim F (x = 0. x Example Distributions that are neither discrete nor continuous. 1. Define a distribution F that is χ-squared with two degrees of freedom. 1. Observe K from a Poisson distribution with mean λ. 2. Sum K observations from F and report this value, if K = 0 report Define a distribution F that is standard normal. 1. Observe X from a Bernoulli distribution with paramater p. 2. If X = 0 report 0 otherwise report a draw from F. Are these distributions interesting priors? Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard. It is easier to define the measure on a much smaller collection of the power set and state consistency conditions that allow us to extend the measure to the the power set. This is what Dynkin s π λ theorem was designed to do.

6 6 Definition (π-system Given a set Ω a π system is a collection of subsets P that are closed under finite intersections. 1 P is non-empty; 2 A B P whenever A, B P. Definition (λ-system Given a set Ω a λ system is a collection of subsets L that contains Ω and is closed under complementation and disjoint countable unions. 1 Ω L ; 2 A L A c L ; 3 A n L, n 1 with A i A j = i j n=1 A n L. Definition (σ-algebra Let F be a collection of subsets of Ω. F is called a field (algebra if Ω F and F is closed under complementation and countable unions, 1 Ω F; 2 A F A c F; 3 A 1,..., A n F n A j F; 4 A 1,..., A n,... F A j F. A σ-algebra is a π system. A σ-algebra is a λ system but a λ system need not be a σ algebra, a λ system is a weaker system. Example Ω = {a, b, c, d} and L = {Ω,, {a, b}, {b, c}, {c, d}, {a, d}, {b, d}, {a, c}} L is closed under disjoint unions but not unions. However: Lemma A class that is both a π system and a λ system is a σ-algebra. Next lecture we will start constructing probability measures on infinite sets and we will need to push the idea of extension of measure. In this the following theorem is central. Theorem (Dynkin If P is a π system and L is a λ system, then P L implies σ(p L.

### Extension of measure

1 Extension of measure Sayan Mukherjee Dynkin s π λ theorem We will soon need to define probability measures on infinite and possible uncountable sets, like the power set of the naturals. This is hard.

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### CHAPTER 5. Product Measures

54 CHAPTER 5 Product Measures Given two measure spaces, we may construct a natural measure on their Cartesian product; the prototype is the construction of Lebesgue measure on R 2 as the product of Lebesgue

### Measure Theory. Jesus Fernandez-Villaverde University of Pennsylvania

Measure Theory Jesus Fernandez-Villaverde University of Pennsylvania 1 Why Bother with Measure Theory? Kolmogorov (1933). Foundation of modern probability. Deals easily with: 1. Continuous versus discrete

### Chapter 1 - σ-algebras

Page 1 of 17 PROBABILITY 3 - Lecture Notes Chapter 1 - σ-algebras Let Ω be a set of outcomes. We denote by P(Ω) its power set, i.e. the collection of all the subsets of Ω. If the cardinality Ω of Ω is

### INTRODUCTORY SET THEORY

M.Sc. program in mathematics INTRODUCTORY SET THEORY Katalin Károlyi Department of Applied Analysis, Eötvös Loránd University H-1088 Budapest, Múzeum krt. 6-8. CONTENTS 1. SETS Set, equal sets, subset,

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Basic Measure Theory

Basic Measure Theory Heinrich v. Weizsäcker Revised Translation 2004/2008 Fachbereich Mathematik Technische Universität Kaiserslautern 2 Measure Theory (October 20, 2008) Preface The aim of this text is

### Mathematics for Econometrics, Fourth Edition

Mathematics for Econometrics, Fourth Edition Phoebus J. Dhrymes 1 July 2012 1 c Phoebus J. Dhrymes, 2012. Preliminary material; not to be cited or disseminated without the author s permission. 2 Contents

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### MEASURE AND INTEGRATION. Dietmar A. Salamon ETH Zürich

MEASURE AND INTEGRATION Dietmar A. Salamon ETH Zürich 12 May 2016 ii Preface This book is based on notes for the lecture course Measure and Integration held at ETH Zürich in the spring semester 2014. Prerequisites

### The Basics of Financial Mathematics. Spring Richard F. Bass Department of Mathematics University of Connecticut

The Basics of Financial Mathematics Spring 23 Richard F. Bass Department of Mathematics University of Connecticut These notes are c 23 by Richard Bass. They may be used for personal use or class use, but

### Jointly Distributed Random Variables

Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### REAL ANALYSIS LECTURE NOTES: 1.4 OUTER MEASURE

REAL ANALYSIS LECTURE NOTES: 1.4 OUTER MEASURE CHRISTOPHER HEIL 1.4.1 Introduction We will expand on Section 1.4 of Folland s text, which covers abstract outer measures also called exterior measures).

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Elements of probability theory

2 Elements of probability theory Probability theory provides mathematical models for random phenomena, that is, phenomena which under repeated observations yield di erent outcomes that cannot be predicted

### DISINTEGRATION OF MEASURES

DISINTEGRTION OF MESURES BEN HES Definition 1. Let (, M, λ), (, N, µ) be sigma-finite measure spaces and let T : be a measurable map. (T, µ)-disintegration is a collection {λ y } y of measures on M such

### Stochastic Processes and Advanced Mathematical Finance. Laws of Large Numbers

Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebraska-Lincoln Lincoln, NE 68588-0130 http://www.math.unl.edu Voice: 402-472-3731 Fax: 402-472-8466 Stochastic Processes and Advanced

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Mikołaj Krupski. Regularity properties of measures on compact spaces. Instytut Matematyczny PAN. Praca semestralna nr 1 (semestr zimowy 2010/11)

Mikołaj Krupski Instytut Matematyczny PAN Regularity properties of measures on compact spaces Praca semestralna nr 1 (semestr zimowy 2010/11) Opiekun pracy: Grzegorz Plebanek Regularity properties of measures

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### The Lebesgue Measure and Integral

The Lebesgue Measure and Integral Mike Klaas April 12, 2003 Introduction The Riemann integral, defined as the limit of upper and lower sums, is the first example of an integral, and still holds the honour

### Lecture Notes on Measure Theory and Functional Analysis

Lecture Notes on Measure Theory and Functional Analysis P. Cannarsa & T. D Aprile Dipartimento di Matematica Università di Roma Tor Vergata cannarsa@mat.uniroma2.it daprile@mat.uniroma2.it aa 2006/07 Contents

### Chapter 1. Metric Spaces. Metric Spaces. Examples. Normed linear spaces

Chapter 1. Metric Spaces Metric Spaces MA222 David Preiss d.preiss@warwick.ac.uk Warwick University, Spring 2008/2009 Definitions. A metric on a set M is a function d : M M R such that for all x, y, z

### Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability

Chapter 1 Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (1999. Other treatments of probability theory include Gallant (1997, Casella & Berger

### Mathematical Background

Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,

### Lecture 7: Continuous Random Variables

Lecture 7: Continuous Random Variables 21 September 2005 1 Our First Continuous Random Variable The back of the lecture hall is roughly 10 meters across. Suppose it were exactly 10 meters, and consider

### Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..

Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,

### Math212a1010 Lebesgue measure.

Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.

### Our goal first will be to define a product measure on A 1 A 2.

1. Tensor product of measures and Fubini theorem. Let (A j, Ω j, µ j ), j = 1, 2, be two measure spaces. Recall that the new σ -algebra A 1 A 2 with the unit element is the σ -algebra generated by the

### Sections 2.1, 2.2 and 2.4

SETS Sections 2.1, 2.2 and 2.4 Chapter Summary Sets The Language of Sets Set Operations Set Identities Introduction Sets are one of the basic building blocks for the types of objects considered in discrete

### Introduction to Probability

Introduction to Probability EE 179, Lecture 15, Handout #24 Probability theory gives a mathematical characterization for experiments with random outcomes. coin toss life of lightbulb binary data sequence

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### Notes on metric spaces

Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.

### The Chinese Restaurant Process

COS 597C: Bayesian nonparametrics Lecturer: David Blei Lecture # 1 Scribes: Peter Frazier, Indraneel Mukherjee September 21, 2007 In this first lecture, we begin by introducing the Chinese Restaurant Process.

### Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011

Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011 A. Miller 1. Introduction. The definitions of metric space and topological space were developed in the early 1900 s, largely

### Examination 110 Probability and Statistics Examination

Examination 0 Probability and Statistics Examination Sample Examination Questions The Probability and Statistics Examination consists of 5 multiple-choice test questions. The test is a three-hour examination

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### MA651 Topology. Lecture 6. Separation Axioms.

MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples

### PROBABILITY THEORY - PART 1 MEASURE THEORETICAL FRAMEWORK CONTENTS

PROBABILITY THEORY - PART 1 MEASURE THEORETICAL FRAMEWORK MANJUNATH KRISHNAPUR CONTENTS 1. Discrete probability space 2 2. Uncountable probability spaces? 3 3. Sigma algebras and the axioms of probability

### 6.4 The Infinitely Many Alleles Model

6.4. THE INFINITELY MANY ALLELES MODEL 93 NE µ [F (X N 1 ) F (µ)] = + 1 i

### Basic Measure Theory. 1.1 Classes of Sets

1 Basic Measure Theory In this chapter, we introduce the classes of sets that allow for a systematic treatment of events and random observations in the framework of probability theory. Furthermore, we

### Math/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

### {f 1 (U), U F} is an open cover of A. Since A is compact there is a finite subcover of A, {f 1 (U 1 ),...,f 1 (U n )}, {U 1,...

44 CHAPTER 4. CONTINUOUS FUNCTIONS In Calculus we often use arithmetic operations to generate new continuous functions from old ones. In a general metric space we don t have arithmetic, but much of it

### Basic Probability Concepts

page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes

### What is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference

0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### PART I. THE REAL NUMBERS

PART I. THE REAL NUMBERS This material assumes that you are already familiar with the real number system and the representation of the real numbers as points on the real line. I.1. THE NATURAL NUMBERS

### CONDITIONAL EXPECTATION AND MARTINGALES

CONDITIONAL EXPECTATION AND MARTINGALES 1. INTRODUCTION Martingales play a role in stochastic processes roughly similar to that played by conserved quantities in dynamical systems. Unlike a conserved quantity

### 9 More on differentiation

Tel Aviv University, 2013 Measure and category 75 9 More on differentiation 9a Finite Taylor expansion............... 75 9b Continuous and nowhere differentiable..... 78 9c Differentiable and nowhere monotone......

### SOLUTIONS TO EXERCISES FOR. MATHEMATICS 205A Part 3. Spaces with special properties

SOLUTIONS TO EXERCISES FOR MATHEMATICS 205A Part 3 Fall 2008 III. Spaces with special properties III.1 : Compact spaces I Problems from Munkres, 26, pp. 170 172 3. Show that a finite union of compact subspaces

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### Recursion Theory in Set Theory

Contemporary Mathematics Recursion Theory in Set Theory Theodore A. Slaman 1. Introduction Our goal is to convince the reader that recursion theoretic knowledge and experience can be successfully applied

### Finite and discrete probability distributions

8 Finite and discrete probability distributions To understand the algorithmic aspects of number theory and algebra, and applications such as cryptography, a firm grasp of the basics of probability theory

### Automata and Rational Numbers

Automata and Rational Numbers Jeffrey Shallit School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1 Canada shallit@cs.uwaterloo.ca http://www.cs.uwaterloo.ca/~shallit 1/40 Representations

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### x a x 2 (1 + x 2 ) n.

Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number

### Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.

Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()

### The fundamental group of the Hawaiian earring is not free (International Journal of Algebra and Computation Vol. 2, No. 1 (1992), 33 37) Bart de Smit

The fundamental group of the Hawaiian earring is not free Bart de Smit The fundamental group of the Hawaiian earring is not free (International Journal of Algebra and Computation Vol. 2, No. 1 (1992),

### God created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886)

Chapter 2 Numbers God created the integers and the rest is the work of man. (Leopold Kronecker, in an after-dinner speech at a conference, Berlin, 1886) God created the integers and the rest is the work

### Introduction to probability theory in the Discrete Mathematics course

Introduction to probability theory in the Discrete Mathematics course Jiří Matoušek (KAM MFF UK) Version: Oct/18/2013 Introduction This detailed syllabus contains definitions, statements of the main results

### Lecture 13: Martingales

Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of

### IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Tuesday, September 11 Normal Approximations and the Central Limit Theorem

IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Tuesday, September 11 Normal Approximations and the Central Limit Theorem Time on my hands: Coin tosses. Problem Formulation: Suppose that I have

### Metric Spaces. Chapter 1

Chapter 1 Metric Spaces Many of the arguments you have seen in several variable calculus are almost identical to the corresponding arguments in one variable calculus, especially arguments concerning convergence

### and s n (x) f(x) for all x and s.t. s n is measurable if f is. REAL ANALYSIS Measures. A (positive) measure on a measurable space

RAL ANALYSIS A survey of MA 641-643, UAB 1999-2000 M. Griesemer Throughout these notes m denotes Lebesgue measure. 1. Abstract Integration σ-algebras. A σ-algebra in X is a non-empty collection of subsets

### Basics of Probability

Basics of Probability 1 Sample spaces, events and probabilities Begin with a set Ω the sample space e.g., 6 possible rolls of a die. ω Ω is a sample point/possible world/atomic event A probability space

### The Ergodic Theorem and randomness

The Ergodic Theorem and randomness Peter Gács Department of Computer Science Boston University March 19, 2008 Peter Gács (Boston University) Ergodic theorem March 19, 2008 1 / 27 Introduction Introduction

### M2S1 Lecture Notes. G. A. Young http://www2.imperial.ac.uk/ ayoung

M2S1 Lecture Notes G. A. Young http://www2.imperial.ac.uk/ ayoung September 2011 ii Contents 1 DEFINITIONS, TERMINOLOGY, NOTATION 1 1.1 EVENTS AND THE SAMPLE SPACE......................... 1 1.1.1 OPERATIONS

### THE CENTRAL LIMIT THEOREM TORONTO

THE CENTRAL LIMIT THEOREM DANIEL RÜDT UNIVERSITY OF TORONTO MARCH, 2010 Contents 1 Introduction 1 2 Mathematical Background 3 3 The Central Limit Theorem 4 4 Examples 4 4.1 Roulette......................................

### Absolute continuity of measures and preservation of Randomness

Absolute continuity of measures and preservation of Randomness Mathieu Hoyrup 1 and Cristóbal Rojas 2 1 LORIA - 615, rue du jardin botanique, BP 239 54506 Vandœuvre-lès-Nancy, FRANCE, mathieu.hoyrup@loria.fr.

### CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.

CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,

### 13 Infinite Sets. 13.1 Injections, Surjections, and Bijections. mcs-ftl 2010/9/8 0:40 page 379 #385

mcs-ftl 2010/9/8 0:40 page 379 #385 13 Infinite Sets So you might be wondering how much is there to say about an infinite set other than, well, it has an infinite number of elements. Of course, an infinite

### RANDOM INTERVAL HOMEOMORPHISMS. MICHA L MISIUREWICZ Indiana University Purdue University Indianapolis

RANDOM INTERVAL HOMEOMORPHISMS MICHA L MISIUREWICZ Indiana University Purdue University Indianapolis This is a joint work with Lluís Alsedà Motivation: A talk by Yulij Ilyashenko. Two interval maps, applied

### SOLUTIONS TO ASSIGNMENT 1 MATH 576

SOLUTIONS TO ASSIGNMENT 1 MATH 576 SOLUTIONS BY OLIVIER MARTIN 13 #5. Let T be the topology generated by A on X. We want to show T = J B J where B is the set of all topologies J on X with A J. This amounts

### Lecture 8: Random Walk vs. Brownian Motion, Binomial Model vs. Log-Normal Distribution

Lecture 8: Random Walk vs. Brownian Motion, Binomial Model vs. Log-ormal Distribution October 4, 200 Limiting Distribution of the Scaled Random Walk Recall that we defined a scaled simple random walk last

### CmSc 175 Discrete Mathematics Lesson 10: SETS A B, A B

CmSc 175 Discrete Mathematics Lesson 10: SETS Sets: finite, infinite, : empty set, U : universal set Describing a set: Enumeration = {a, b, c} Predicates = {x P(x)} Recursive definition, e.g. sequences

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### , for x = 0, 1, 2, 3,... (4.1) (1 + 1/n) n = 2.71828... b x /x! = e b, x=0

Chapter 4 The Poisson Distribution 4.1 The Fish Distribution? The Poisson distribution is named after Simeon-Denis Poisson (1781 1840). In addition, poisson is French for fish. In this chapter we will

### Normal distribution Approximating binomial distribution by normal 2.10 Central Limit Theorem

1.1.2 Normal distribution 1.1.3 Approimating binomial distribution by normal 2.1 Central Limit Theorem Prof. Tesler Math 283 October 22, 214 Prof. Tesler 1.1.2-3, 2.1 Normal distribution Math 283 / October

### 4. Joint Distributions

Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose

### Stationary random graphs on Z with prescribed iid degrees and finite mean connections

Stationary random graphs on Z with prescribed iid degrees and finite mean connections Maria Deijfen Johan Jonasson February 2006 Abstract Let F be a probability distribution with support on the non-negative

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Metric Spaces Joseph Muscat 2003 (Last revised May 2009)

1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of

### We give a basic overview of the mathematical background required for this course.

1 Background We give a basic overview of the mathematical background required for this course. 1.1 Set Theory We introduce some concepts from naive set theory (as opposed to axiomatic set theory). The

### Combinatorics: The Fine Art of Counting

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### 1.1 Logical Form and Logical Equivalence 1

Contents Chapter I The Logic of Compound Statements 1.1 Logical Form and Logical Equivalence 1 Identifying logical form; Statements; Logical connectives: not, and, and or; Translation to and from symbolic

### An example of a computable

An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept

### Sufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Sufficient Statistics and Exponential Family 1 Statistics and Sufficient Statistics Suppose we have a random sample X 1,, X n taken from a distribution

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### Omega Automata: Minimization and Learning 1

Omega Automata: Minimization and Learning 1 Oded Maler CNRS - VERIMAG Grenoble, France 2007 1 Joint work with A. Pnueli, late 80s Summary Machine learning in general and of formal languages in particular

### The Henstock-Kurzweil-Stieltjes type integral for real functions on a fractal subset of the real line

The Henstock-Kurzweil-Stieltjes type integral for real functions on a fractal subset of the real line D. Bongiorno, G. Corrao Dipartimento di Ingegneria lettrica, lettronica e delle Telecomunicazioni,

### Chapters 5. Multivariate Probability Distributions

Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are