# Introduction to probability theory in the Discrete Mathematics course

Save this PDF as:

Size: px
Start display at page:

Download "Introduction to probability theory in the Discrete Mathematics course"

## Transcription

1 Introduction to probability theory in the Discrete Mathematics course Jiří Matoušek (KAM MFF UK) Version: Oct/18/2013 Introduction This detailed syllabus contains definitions, statements of the main results and concise comments. It should not substitute a textbook, and you will find no proofs and no solved exercises here. A part of the material is covered in more detail in the book J. Matoušek, J. Nešetřil: Invitation to Discrete Mathematics, 2nd edition, Oxford University Press, 2009 (Chap. 10). There are many textbooks of probability theory; an introduction for beginners is, e.g., H. Tijms: Understanding Probability. Cambridge Univ. Press., 2004, and a more advanced and comprehensive book is G. R. Grimmett, D. R. Stirzaker: Probability and Random Processes, Oxford University Press, A reasonable-looking textbook at can even be freely downloaded at present. 1. Chance, or randomness, is often symbolized by a die (plural: dice). The probability of getting a six in a single roll is The probability of two sixes in two rolls is 36. Tossing a fair coin is still a simpler experiment: both heads and tails have probability What is meant by probability? For a real probability this is a philosophical question, with various answers, none of them completely satisfactory. (Dice can be rolled repeatedly, empirical probability can be defined as a limit for an enormous number of rolls but does it make any sense to talk about the probability of unique events, such as the gorillas, or the humans, surviving the year 2015?) Where does randomness come from? (Maybe from quantum theory? Or from chaos theory what we in our ignorance perceive as random might perhaps be really predetermined?) 3. The mathematical theory of probability does not consider these philosophical conundrums. It construct a mathematical model of probability. This model works amazingly well, ask insurance companies (one that are not going bankrupt, that is). But for reallife problems it must be applied thoughtfully (ask insurance companies that are going bankrupt). It is well known to psychologists that people generally cannot estimate probabilities reliably, especially very small or very large ones (see, for example, Thinking fast and slow by Daniel Kahneman). For example, in an experiment, only 45 % students were able to finish their thesis by the time for which they were 99 % sure to finish it. 4. Probability in computer science: randomized algorithms (simpler and faster than deterministic ones; often the only feasible approach for massive amounts of data), statistical tests and evaluation of experiments, mathematical proofs. 1

2 2 Probability spaces, mainly discrete ones 5. We perform a random experiment. Let Ω denote the set of all possible outcomes. Examples: Tossing a coin: Ω = {H, T } (heads, tails). Rolling a die: Ω = {1, 2, 3, 4, 5, 6}. Three consecutive coin tosses: Ω = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }. The first raindrop falling on a square garden table: Ω = all points of a square. Elements of Ω are called elementary events. 6. An event is a subset A Ω. Examples: coin toss we can consider 4 events: A 1 = {H} (heads tossed), A 2 = {T } (tails tossed), A 3 = {H, T } (heads or tails tossed, this is a certain event, it always occurs), and A 4 = (nothing tossed, this is the impossible event, it never occurs). rolling a die, Ω = {1, 2, 3, 4, 5, 6}: an even number rolled A 1 = {2, 4, 6}, six rolled A 2 = {6}, nothing rolled A 3 =. first raindrop: falls in the left half of the table, falls at most 10 cm from the border, falls on a flower printed on the tablecloth, etc. 7. To every event A we assign a real number P[A] [0, 1], called the probability of A. Examples: For tossing a fair coin we have P[{H}] = P[{T }] = 1 2, P[{H, T }] = 1, P[ ] = 0. For a fair die we have P[{1}] = P[{2}] = = P[{6}] = 1 6, and, for example, P[{1, 3, 5}] = 1 2. We can also consider a loaded die (of a cheater) with P[{6}] = 1 5, P[{1}] = P[{2}] = = P[{5}] = Thus, not all one-element events need to have the same probability. 8. A probability space (also sometimes called a sample space) is formally a triple (Ω, F, P), where Ω tells us, which elementary events are considered; F 2 Ω specifies, which subsets of Ω are admitted as events; and P: F [0, 1] is a function that assigns to every event its probability. 9. Every probability space has to satisfy certain axioms. Here we will not present them in full generality, since we do not yet have the necessary mathematical tools ready (specifically, contemporaty probability is based in measure theory, which is usually covered only later in the curriculum, if at all). We introduce only discrete probability spaces precisely, ones with the set Ω finite or countable, and such that all subsets of Ω are events.

3 3 We are thus leaving aside, e.g., geometric probability (what is the probability of the first raindrop falling at most 10 cm from the border, etc.), as well as probabilities concerning arbitrarily long sequences. (For example: We repeatedly toss a coin. Is it more likely that we first encounter the sequence HHT, or HTT? This is a real question, you can make bets about it! And it has a surprising answer: HHT wins twice as often.) 10. A discrete probability space is a triple (Ω, F, P), where Ω is a finite or countable set, F = 2 Ω (that is, every subset is an event), the probability of every event A Ω satisfies P[A] = ω A P[{ω}], and P[Ω] = 1. This means that a discrete probability space is fully determined by the probabilities of all one-element events. The probabilities of these singletons can be chosen as arbitrary nonnegative numbers whose sum over Ω equals 1 (for infinite Ω this is a sum of an infinite series). 11. Here we will work mainly with finite discrete probability spaces, where Ω is a finite set. (In this case we will usually omit the word discrete.) 12. A basic example of a finite probability space is a classical probability space, where P[A] = ; thus, all singleton events have the same probability. A Ω 13. Remark: If we wanted every singleton to have the same probability for Ω infinite, these probabilities would have to be all 0. Indeed, in geometric probability (a random point in a square, say), every single point has zero probability. At the same time, there are events, such as the whole square, with nonzero probability, which looks paradoxical how can infinitely many points with zero probability combine into something with nonzero probability? This is where measure theory is needed, to define geometric probability in a consistent way. 14. Here are some specific examples of classical probability spaces, particularly important for discrete mathematics. A random sequence of n tosses by a fair coin: Ω = {H, T } n, every possible sequence of outcomes has probability 1/2 n. A random permutation of the set {1, 2,..., n}: Ω = S n (the set of all permutations), each permutation has probability 1/n!. 15. Let us stress that for a given mathematical or practical problem, a probability space can often be chosen in several different ways; some of them may turn out to be more convenient than others. 16. What are probability spaces good for? They provide a safe foundation and often they help in clarifying various tricky questions. Example: Bertrand s box paradox; here we present it in a version with cards. There are three cards in a hat: one with both sides red, another with both sides black, and a third one with one side red and one black. You

4 4 blindly draw one of the cards and put it on the table. Then you look and see that the top side is red. What is the probability that the bottom side is red too? An intuitive answer is 1 2, but the truth is 2 3. (Also see a modern version the Monty Hall problem). Conditional probability, independent events 17. The conditional probability of an event A assuming that an event B has occurred is defined as P[A B] P[A B] = P[B] (it is defined only for P[B] > 0). 18. Often it is useful to compute the probability of A by case distinction : if B 1,..., B n are disjoint events whose union is all of Ω, then P[A] = P[A B 1 ] P[B 1 ] + P[A B 2 ] P[B 2 ] + + P[A B n ] P[B n ]. This simple statement is sometimes called the theorem about complete probability. 19. Example (somewhat artificial but valuable). Let us say that 0.1% of the population is HIV-infected. Let us imagine that you had a HIV test, which is known to give a positive result for an actually infected person in 95 % of cases, while for persons not infected it gives a (false) positive result in 5 %. You have no particular reason to expect to be infected (i.e., you can be regarded as a random person in this respect), but your test comes out positively. What is the probability, that you are really infected? Here the probability space is the population (and you are a randomly selected person), the event H = HIV positive, the event T = tests positively, assumptions: P[H] = 0.001, P[T H] = 0.95, P[T not H] = 0.05; we want to know: P[H T ]. The result is less than 2 %, so you need not worry too much. By formulating the reasoning in this example in a general way, we obtain the Bayes theorem: P[A B i ] P[B i ] P[B i A] = nj=1 P[A B j ] P[B j ], where B 1,..., B n are events as in the previous item (disjoint and covering Ω). 20. Two events A and B are called independent if P[A B] = P[A] P[B]. Equivalently, A and B are independent if P[B] = 0 or P[A B] = P[A]. (Intuition: if we learn whether event B has occurred, we have gained no new information about the probability of B occurring.) More generally: events A 1, A 2,..., A n are independent if we have, for every index set I {1, 2,..., n}, the equality P[ i I A i] = i I P[A i]. 21. Example: we consider a random sequence of n zeros and ones (or n fair coin tosses), A 1 = {the first 5 entries are all 1s}, A 2 = {the sixth entry is 0}, A 3 = {there are an odd number of 1s among the last 5 entries}. The events A 1 and A 2 are obviously independent. The events A 1, A 2, A 3 are also independent, but this has to be checked carefully according to the definition. Another example: a random permutation π of {1, 2,..., 32 (a well-mixed pack of cards); the events A = {π(1) = 1} and B = {π(2) = 2} are not independent!

5 5 22. There is a widespread error in intuitive interpretations of independent events ( I have not rolled a six for such a long time so now it must come! in reality, the dice have no memory and the probability does not depend on what happened before). 23. Example: Ω = {blue-eyed blond woman, brown-eyed brown-haired woman, blue-eyed brown-haired man, brown-eyed blond man}, all four have probability 1 4 (classical probability space); events A 1 = blue eyes, A 2 = blond hair, A 3 = woman (or, if you prefer numbers, Ω = {000, 011, 110, 101}). Every two of these events are independent, but all three are not. 24. Product of (discrete) probability spaces: (Ω 1, 2 Ω 1, P 1 ) (Ω 2, 2 Ω 2, P 2 ) = (Ω, 2 Ω, P), where Ω = Ω 1 Ω 2 and P[A] = (ω 1,ω 2 ) A P 1[{ω 1 }]P 2 [{ω 2 }]. 25. Let us consider a product probability space with Ω = Ω 1 Ω 2, and let A 1 be an event that depends only on the first component; that is, A 1 = A 1 Ω 2 for some A 1 Ω 1. Similarly, let A 2 be an event that depends only on the second component. Then A 1 and A 2 are independent. 26. Example: a sequence of n tosses of a fair coin can be constructed as a product probability space, the product of n spaces, one for each coin toss. A more interesting example: Ω = {0, 1}, P[{1}] = p (an experiment with success probability p), the product of n copies of this space models a sequence of n independent repetitions of this experiment. For ω {0, 1} n we have P[{ω}] = p j (1 p) n j, where j is the number of 1s in ω. Random variables, expectation 27. A (real) random variable on a discrete probability space (Ω, 2 Ω, P) is an arbitrary function X: Ω R. Example: We toss a fair coin n times, so Ω = {H, T } n, i.e., all n-term sequences of H s and T s. Examples of random variables on this space: the number of times heads was tossed; the number of heads minus the number of tails, the sine of the number of heads. (Remark: One can also consider complex random variables, or random variables whose values are points in the plane, etc. But here we talk only about real random variables.) Be very careful to distinguish between an event (only two possible outcomes, occurred/not occurred formally an event is a set), and a random variable (the result is a number, and formally a random variable is a function). In a sense, an event can be regarded as a special case of a random variable, as will be mentioned later. 28. Let X: Ω R be a random variable on a discrete probability space (Ω, 2 Ω, P). The expectation of X is defined as E[X] := ω Ω P[{ω}] X(ω). (For Ω infinite, the infinite series in the definition of E[X] need not converge, and then the expectation of X does not exist.) Intuition: E[X] is the average value of X if we make a very large number of experiments.

6 6 29. E[X] can also be computed as v V v P[X = v], where V is the set of all values attained by X (this is just rearranging the sum in the definition). Here P[X = v] is a commonly used shorthand for the probability of the event {ω Ω : X(ω) = v}. 30. Another notion, which in practice is sometimes confused with the expectation: the median of a real random variable X is a number m such that P[X < m] 1 2 and P[X > m] 1 2. For example, half of people have salary smaller than the median and half larger. But, as a rule, a substantial majority has salary smaller than average (and everyone wants at least the average salary which many politicians are willing to promise before elections). 31. Linearity of expectation: E[αX] = αe[x], E[X + Y ] = E[X]+E[Y ] for any two, totally arbitrary, random variables X and Y, and α R (assuming that all the expectations exist). The proof is immediate from the definition. But, for example, E[XY ] E[X] E[Y ] in general. 32. The indicator of an event A is the random variable I A : Ω {0, 1} given by I A (ω) = { 1 for ω A 0 for ω A. We have E[I A ] = P[A]. The indicator I A is the counterpart of A in the realm of random variables; in this sense, random variables are more general than events. 33. Examples (calculating the expectation): X = number of heads in a sequence of n tosses of a fair coin. The expectation E[X] can be calculated according to the definition. A much easier way, the method of indicators: A i is the event the ith comes out heads ; X = I A1 + I A2 + + I An, E[I Ai ] = 1 2, E[X] = n 2. Y = the number of left maxima of a random permutation π of the set {1, 2,..., n}, i.e., the number of i such that π(j) < π(i) for all j < i. Event A i := i is a left maximum. We calculate E[Y ] = n ln n. 34. Example: Every graph G = (V, E) has a bipartite subgraph with at least 1 2 E edges. Proof: Partition the vertex set into two parts at random, let X be the number of edges going across, and calculate E[X] = 1 2 E. 35. Random variables X, Y (on the same probability space) are called independent if for every a, b R, the events X a and Y b are independent. Similarly for more than 2 random variables. 36. For independent random variables we have E[XY ] = E[X] E[Y ].

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES Contents 1. Probabilistic experiments 2. Sample space 3. Discrete probability

### Probability II (MATH 2647)

Probability II (MATH 2647) Lecturer Dr. O. Hryniv email Ostap.Hryniv@durham.ac.uk office CM309 http://maths.dur.ac.uk/stats/courses/probmc2h/probability2h.html or via DUO This term we shall consider: Review

### Lecture 11: Probability models

Lecture 11: Probability models Probability is the mathematical toolbox to describe phenomena or experiments where randomness occur. To have a probability model we need the following ingredients A sample

### Expected Value 10/11/2005

Expected Value 10/11/2005 Definition Let X be a numerically-valued discrete random variable with sample space Ω and distribution function m(x). The expected value E(X) is defined by E(X) = x Ω xm(x), provided

### Combinatorics: The Fine Art of Counting

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

### Lecture 2: Probability

Lecture 2: Probability Assist. Prof. Dr. Emel YAVUZ DUMAN MCB1007 Introduction to Probability and Statistics İstanbul Kültür University Outline 1 Introduction 2 Sample Spaces 3 Event 4 The Probability

### 3. Examples of discrete probability spaces.. Here α ( j)

3. EXAMPLES OF DISCRETE PROBABILITY SPACES 13 3. Examples of discrete probability spaces Example 17. Toss n coins. We saw this before, but assumed that the coins are fair. Now we do not. The sample space

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 13. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 3 Random Variables: Distribution and Expectation Random Variables Question: The homeworks of 20 students are collected

### Math/Stats 425 Introduction to Probability. 1. Uncertainty and the axioms of probability

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

### Lecture 4: Probability Spaces

EE5110: Probability Foundations for Electrical Engineers July-November 2015 Lecture 4: Probability Spaces Lecturer: Dr. Krishna Jagannathan Scribe: Jainam Doshi, Arjun Nadh and Ajay M 4.1 Introduction

### Math 421: Probability and Statistics I Note Set 2

Math 421: Probability and Statistics I Note Set 2 Marcus Pendergrass September 13, 2013 4 Discrete Probability Discrete probability is concerned with situations in which you can essentially list all the

### Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22

CS 70 Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice, roulette

### Events. Independence. Coin Tossing. Random Phenomena

Random Phenomena Events A random phenomenon is a situation in which we know what outcomes could happen, but we don t know which particular outcome did or will happen For any random phenomenon, each attempt,

### Topic 6: Conditional Probability and Independence

Topic 6: September 15-20, 2011 One of the most important concepts in the theory of probability is based on the question: How do we modify the probability of an event in light of the fact that something

### Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab

Lecture 2 Probability BMIR Lecture Series in Probability and Statistics Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab 2.1 1 Sample Spaces and Events Random

### Toss a coin twice. Let Y denote the number of heads.

! Let S be a discrete sample space with the set of elementary events denoted by E = {e i, i = 1, 2, 3 }. A random variable is a function Y(e i ) that assigns a real value to each elementary event, e i.

### Examples of infinite sample spaces. Math 425 Introduction to Probability Lecture 12. Example of coin tosses. Axiom 3 Strong form

Infinite Discrete Sample Spaces s of infinite sample spaces Math 425 Introduction to Probability Lecture 2 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan February 4,

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### Basic concepts in probability. Sue Gordon

Mathematics Learning Centre Basic concepts in probability Sue Gordon c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Set Notation You may omit this section if you are

### I. WHAT IS PROBABILITY?

C HAPTER 3 PROBABILITY Random Experiments I. WHAT IS PROBABILITY? The weatherman on 0 o clock news program states that there is a 20% chance that it will snow tomorrow, a 65% chance that it will rain and

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice,

### PROBABILITY NOTIONS. Summary. 1. Random experiment

PROBABILITY NOTIONS Summary 1. Random experiment... 1 2. Sample space... 2 3. Event... 2 4. Probability calculation... 3 4.1. Fundamental sample space... 3 4.2. Calculation of probability... 3 4.3. Non

### lecture notes February 10, Probability Theory

8.30 lecture notes February 0, 205 Probability Theory Lecturer: Michel Goemans These notes cover the basic definitions of discrete probability theory, and then present some results including Bayes rule,

### Basics of Probability

Basics of Probability August 27 and September 1, 2009 1 Introduction A phenomena is called random if the exact outcome is uncertain. The mathematical study of randomness is called the theory of probability.

### 33 Probability: Some Basic Terms

33 Probability: Some Basic Terms In this and the coming sections we discuss the fundamental concepts of probability at a level at which no previous exposure to the topic is assumed. Probability has been

### PROBABILITY. Chapter Overview Conditional Probability

PROBABILITY Chapter. Overview.. Conditional Probability If E and F are two events associated with the same sample space of a random experiment, then the conditional probability of the event E under the

### Probability distributions

Probability distributions (Notes are heavily adapted from Harnett, Ch. 3; Hayes, sections 2.14-2.19; see also Hayes, Appendix B.) I. Random variables (in general) A. So far we have focused on single events,

### 6.042/18.062J Mathematics for Computer Science. Expected Value I

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

### MAT 1000. Mathematics in Today's World

MAT 1000 Mathematics in Today's World We talked about Cryptography Last Time We will talk about probability. Today There are four rules that govern probabilities. One good way to analyze simple probabilities

### Finite Markov Chains and Algorithmic Applications. Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet

Finite Markov Chains and Algorithmic Applications Olle Häggström Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE

### Question: What is the probability that a five-card poker hand contains a flush, that is, five cards of the same suit?

ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the

### Probability. A random sample is selected in such a way that every different sample of size n has an equal chance of selection.

1 3.1 Sample Spaces and Tree Diagrams Probability This section introduces terminology and some techniques which will eventually lead us to the basic concept of the probability of an event. The Rare Event

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### MATH 56A SPRING 2008 STOCHASTIC PROCESSES 31

MATH 56A SPRING 2008 STOCHASTIC PROCESSES 3.3. Invariant probability distribution. Definition.4. A probability distribution is a function π : S [0, ] from the set of states S to the closed unit interval

### Set theory, and set operations

1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we

### 3 Multiple Discrete Random Variables

3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f

### 7.5 Conditional Probability; Independent Events

7.5 Conditional Probability; Independent Events Conditional Probability Example 1. Suppose there are two boxes, A and B containing some red and blue stones. The following table gives the number of stones

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### Probabilities. Probability of a event. From Random Variables to Events. From Random Variables to Events. Probability Theory I

Victor Adamchi Danny Sleator Great Theoretical Ideas In Computer Science Probability Theory I CS 5-25 Spring 200 Lecture Feb. 6, 200 Carnegie Mellon University We will consider chance experiments with

### Chapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum a-posteriori Hypotheses

Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum a-posteriori Hypotheses ML:IV-1 Statistical Learning STEIN 2005-2015 Area Overview Mathematics Statistics...... Stochastics

### Probability Theory. Florian Herzog. A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T..

Probability Theory A random variable is neither random nor variable. Gian-Carlo Rota, M.I.T.. Florian Herzog 2013 Probability space Probability space A probability space W is a unique triple W = {Ω, F,

### Lecture 1 Probability review

Lecture : Probability review of 9 Course: M32K Intro to Stochastic Processes Term: Fall 20 Instructor: Gordan Zitkovic Lecture Probability review RANDOM VARIABLES A large chunk of probability is about

### Introduction to Random Variables (RVs)

ECE270: Handout 5 Introduction to Random Variables (RVs) Outline: 1. informal definition of a RV, 2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV, 3. a general rule to find probability

### MATHEMATICS 117/E-217, SPRING 2012 PROBABILITY AND RANDOM PROCESSES WITH ECONOMIC APPLICATIONS Module #5 (Borel Field and Measurable Functions )

MATHEMATICS 117/E-217, SPRING 2012 PROBABILITY AND RANDOM PROCESSES WITH ECONOMIC APPLICATIONS Module #5 (Borel Field and Measurable Functions ) Last modified: May 1, 2012 Reading Dineen, pages 57-67 Proof

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### 1. The sample space S is the set of all possible outcomes. 2. An event is a set of one or more outcomes for an experiment. It is a sub set of S.

1 Probability Theory 1.1 Experiment, Outcomes, Sample Space Example 1 n psychologist examined the response of people standing in line at a copying machines. Student volunteers approached the person first

### 6.3 Conditional Probability and Independence

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

### Chapter 4 Lecture Notes

Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,

### Basic Probability Concepts

page 1 Chapter 1 Basic Probability Concepts 1.1 Sample and Event Spaces 1.1.1 Sample Space A probabilistic (or statistical) experiment has the following characteristics: (a) the set of all possible outcomes

### Notes 3 Autumn Independence. Two events A and B are said to be independent if

MAS 108 Probability I Notes 3 Autumn 2005 Independence Two events A and B are said to be independent if P(A B) = P(A) P(B). This is the definition of independence of events. If you are asked in an exam

### Worked examples Basic Concepts of Probability Theory

Worked examples Basic Concepts of Probability Theory Example 1 A regular tetrahedron is a body that has four faces and, if is tossed, the probability that it lands on any face is 1/4. Suppose that one

### Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University

Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments

### MATH 10: Elementary Statistics and Probability Chapter 4: Discrete Random Variables

MATH 10: Elementary Statistics and Probability Chapter 4: Discrete Random Variables Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of slides, you

### General theory of stochastic processes

CHAPTER 1 General theory of stochastic processes 1.1. Definition of stochastic process First let us recall the definition of a random variable. A random variable is a random number appearing as a result

### 3. Conditional probability & independence

3. Conditional probability & independence Conditional Probabilities Question: How should we modify P(E) if we learn that event F has occurred? Derivation: Suppose we repeat the experiment n times. Let

### NOTES ON ELEMENTARY PROBABILITY

NOTES ON ELEMENTARY PROBABILITY KARL PETERSEN 1. Probability spaces Probability theory is an attempt to work mathematically with the relative uncertainties of random events. In order to get started, we

### Example. Two fair dice are tossed and the two outcomes recorded. As is familiar, we have

Lectures 9-10 jacques@ucsd.edu 5.1 Random Variables Let (Ω, F, P ) be a probability space. The Borel sets in R are the sets in the smallest σ- field on R that contains all countable unions and complements

### Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.

Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()

### Example. For example, if we roll a die

3 Probability A random experiment has an unknown outcome, but a well defined set of possible outcomes S. The set S is called the sample set. An element of the sample set S is called a sample point (elementary

### A Simple Example. Sample Space and Event. Tree Diagram. Tree Diagram. Probability. Probability - 1. Probability and Counting Rules

Probability and Counting Rules researcher claims that 10% of a large population have disease H. random sample of 100 people is taken from this population and examined. If 20 people in this random sample

### Students in their first advanced mathematics classes are often surprised

CHAPTER 8 Proofs Involving Sets Students in their first advanced mathematics classes are often surprised by the extensive role that sets play and by the fact that most of the proofs they encounter are

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### Using Laws of Probability. Sloan Fellows/Management of Technology Summer 2003

Using Laws of Probability Sloan Fellows/Management of Technology Summer 2003 Uncertain events Outline The laws of probability Random variables (discrete and continuous) Probability distribution Histogram

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Chapter 3: The basic concepts of probability

Chapter 3: The basic concepts of probability Experiment: a measurement process that produces quantifiable results (e.g. throwing two dice, dealing cards, at poker, measuring heights of people, recording

### Chapter 4 Probability

The Big Picture of Statistics Chapter 4 Probability Section 4-2: Fundamentals Section 4-3: Addition Rule Sections 4-4, 4-5: Multiplication Rule Section 4-7: Counting (next time) 2 What is probability?

7 Probability Copyright Cengage Learning. All rights reserved. 7.1 Sample Spaces and Events Copyright Cengage Learning. All rights reserved. Sample Spaces 3 Sample Spaces At the beginning of a football

### CMPSCI611: Primality Testing Lecture 14

CMPSCI611: Primality Testing Lecture 14 One of the most important domains for randomized algorithms is number theory, particularly with its applications to cryptography. Today we ll look at one of the

### Probabilistic Strategies: Solutions

Probability Victor Xu Probabilistic Strategies: Solutions Western PA ARML Practice April 3, 2016 1 Problems 1. You roll two 6-sided dice. What s the probability of rolling at least one 6? There is a 1

### Introduction. Purpose of the Course. The purpose of this course is to study mathematically the behaviour of stochastic systems.

1 Introduction Purpose of the Course The purpose of this course is to study mathematically the behaviour of stochastic systems. Examples 1. A CPU with jobs arriving to it in a random fashion. 2. A communications

### I. WHAT IS PROBABILITY?

C HAPTER 3 PROAILITY Random Experiments I. WHAT IS PROAILITY? The weatherman on 10 o clock news program states that there is a 20% chance that it will snow tomorrow, a 65% chance that it will rain and

### Random variables, probability distributions, binomial random variable

Week 4 lecture notes. WEEK 4 page 1 Random variables, probability distributions, binomial random variable Eample 1 : Consider the eperiment of flipping a fair coin three times. The number of tails that

### Lecture 8: Continuous random variables, expectation and variance

Lecture 8: Continuous random variables, expectation and variance Lejla Batina Institute for Computing and Information Sciences Digital Security Version: spring 2012 Lejla Batina Version: spring 2012 Wiskunde

### Outcomes, events, and probability

2 Outcomes, events, and probability The world around us is full of phenomena we perceive as random or unpredictable We aim to model these phenomena as outcomes of some experiment, where you should think

### Mathematical foundations of Econometrics

Mathematical foundations of Econometrics G.Gioldasis, UniFe & prof. A.Musolesi, UniFe April 7, 2016.Gioldasis, UniFe & prof. A.Musolesi, UniFe Mathematical foundations of Econometrics April 7, 2016 1 /

### Probability & Probability Distributions

Probability & Probability Distributions Carolyn J. Anderson EdPsych 580 Fall 2005 Probability & Probability Distributions p. 1/61 Probability & Probability Distributions Elementary Probability Theory Definitions

### Probability Review. ICPSR Applied Bayesian Modeling

Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

### Introduction to Discrete Probability Theory and Bayesian Networks

Introduction to Discrete Probability Theory and Bayesian Networks Dr Michael Ashcroft September 15, 2011 This document remains the property of Inatas. Reproduction in whole or in part without the written

### Stat 110. Unit 3: Random Variables Chapter 3 in the Text

Stat 110 Unit 3: Random Variables Chapter 3 in the Text 1 Unit 3 Outline Random Variables (RVs) and Distributions Discrete RVs and Probability Mass Functions (PMFs) Bernoulli and Binomial Distributions

### Chapter 1. Probability Theory: Introduction. Definitions Algebra. RS Chapter 1 Random Variables

Chapter 1 Probability Theory: Introduction Definitions Algebra Definitions: Semiring A collection of sets F is called a semiring if it satisfies: Φ F. If A, B F, then A B F. If A, B F, then there exists

### Math 30530: Introduction to Probability, Spring 2012

Name: Math 30530: Introduction to Probability, Spring 01 Midterm Exam I Monday, February 0, 01 This exam contains problems on 7 pages (including the front cover). Calculators may be used. Show all your

### Definition and Calculus of Probability

In experiments with multivariate outcome variable, knowledge of the value of one variable may help predict another. For now, the word prediction will mean update the probabilities of events regarding the

### Probability and Counting

Probability and Counting Basic Counting Principles Permutations and Combinations Sample Spaces, Events, Probability Union, Intersection, Complements; Odds Conditional Probability, Independence Bayes Formula

### 4.1 4.2 Probability Distribution for Discrete Random Variables

4.1 4.2 Probability Distribution for Discrete Random Variables Key concepts: discrete random variable, probability distribution, expected value, variance, and standard deviation of a discrete random variable.

### Math/Stat 360-1: Probability and Statistics, Washington State University

Math/Stat 360-1: Probability and Statistics, Washington State University Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 3 Haijun Li Math/Stat 360-1: Probability and

### The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics.

Elementary probability theory The concept of probability is fundamental in statistical analysis. Theory of probability underpins most of the methods used in statistics. 1.1 Experiments, outcomes and sample

### Bayesian Tutorial (Sheet Updated 20 March)

Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that

### Discrete Structures for Computer Science: Counting, Recursion, and Probability

Discrete Structures for Computer Science: Counting, Recursion, and Probability Michiel Smid School of Computer Science Carleton University Ottawa, Ontario Canada michiel@scs.carleton.ca December 15, 2016

### M2L1. Random Events and Probability Concept

M2L1 Random Events and Probability Concept 1. Introduction In this lecture, discussion on various basic properties of random variables and definitions of different terms used in probability theory and

### Discrete Random Variables; Expectation Spring 2014 Jeremy Orloff and Jonathan Bloom

Discrete Random Variables; Expectation 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom This image is in the public domain. http://www.mathsisfun.com/data/quincunx.html http://www.youtube.com/watch?v=9xubhhm4vbm

### Finite and discrete probability distributions

8 Finite and discrete probability distributions To understand the algorithmic aspects of number theory and algebra, and applications such as cryptography, a firm grasp of the basics of probability theory

### Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Ch. 13.3: More about Probability

Ch. 13.3: More about Probability Complementary Probabilities Given any event, E, of some sample space, U, of a random experiment, we can always talk about the complement, E, of that event: this is the