# Basics of Probability

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Basics of Probability 1

2 Sample spaces, events and probabilities Begin with a set Ω the sample space e.g., 6 possible rolls of a die. ω Ω is a sample point/possible world/atomic event A probability space or probability model is a sample space with an assignment P(ω) for every ω Ω s.t. 0 P(ω) 1 Σ ω P(ω) = 1 e.g., P(1) =P(2) = P(3) = P(4) =P(5) = P(6) = 1/6. An event A is any subset of Ω P(A) = Σ {ω A} P(ω) E.g., P(die roll < 4) = P(1) + P(2) + P(3) = 1/6 + 1/6 + 1/6 = 1/2 2

3 Random variables A random variable is a function from sample points to some range, e.g., the reals or Booleans e.g., Odd(1) = true. P induces a probability distribution for any r.v. X: P(X =x i ) = Σ {ω:x(ω) =xi }P(ω) e.g., P(Odd = true) = P(1) + P(3) + P(5) = 1/6 + 1/6 + 1/6 = 1/2 3

4 Propositions Think of a proposition as the event (set of sample points) where the proposition is true Given Boolean random variables A and B: event a = set of sample points where A(ω) =true event a = set of sample points where A(ω) = false event a b = points where A(ω) = true and B(ω) = true Often in applications, the sample points are defined by the values of a set of random variables, i.e., the sample space is the Cartesian product of the ranges of the variables With Boolean variables, sample point = propositional logic model e.g., A = true, B =false, or a b. Proposition = disjunction of atomic events in which it is true e.g., (a b) ( a b) (a b) (a b) P(a b) = P( a b) + P(a b) + P(a b) 4

5 Why use probability? The definitions imply that certain logically related events must have related probabilities E.g., P(a b) = P(a) + P(b) P(a b) True A A B > B de Finetti (1931): an agent who bets according to probabilities that violate these axioms can be forced to bet so as to lose money regardless of outcome. 5

6 Syntax for propositions Propositional or Boolean random variables e.g., Cavity (do I have a cavity?) Cavity =true is a proposition, also written cavity Discrete random variables (finite or infinite) e.g., Weather is one of sunny,rain,cloudy,snow W eather = rain is a proposition Values must be exhaustive and mutually exclusive Continuous random variables (bounded or unbounded) e.g., T emp = 21.6; also allow, e.g., T emp < Arbitrary Boolean combinations of basic propositions 6

7 Prior probability Prior or unconditional probabilities of propositions e.g., P(Cavity =true) = 0.1 and P(Weather = sunny) = 0.72 correspond to belief prior to arrival of any (new) evidence Probability distribution gives values for all possible assignments: P(W eather) = 0.72, 0.1, 0.08, 0.1 (normalized, i.e., sums to 1) Joint probability distribution for a set of r.v.s gives the probability of every atomic event on those r.v.s (i.e., every sample point) P(Weather, Cavity) = a 4 2 matrix of values: W eather = sunny rain cloudy snow Cavity = true Cavity = f alse Every question about a domain can be answered by the joint distribution because every event is a sum of sample points 7

8 Probability for continuous variables Express distribution as a parameterized function of value: P(X = x) = U[18, 26](x) = uniform density between 18 and Here P is a density; integrates to 1. P(X = 20.5) = really means 18 dx 26 lim P(20.5 X dx)/dx = dx 0 8

9 P(x) = 1 2πσ e (x µ)2 /2σ 2 Gaussian density 0 9

10 Conditional probability Conditional or posterior probabilities e.g., P(cavity toothache) = 0.8 i.e., given that toothache is all I know NOT if toothache then 80% chance of cavity (Notation for conditional distributions: P(Cavity Toothache) = 2-element vector of 2-element vectors) If we know more, e.g., cavity is also given, then we have P(cavity toothache, cavity) = 1 Note: the less specific belief remains valid after more evidence arrives, but is not always useful New evidence may be irrelevant, allowing simplification, e.g., P(cavity toothache, 49ersW in) = P(cavity toothache) = 0.8 This kind of inference, sanctioned by domain knowledge, is crucial 10

11 Definition of conditional probability: P(a b) = P(a b) P(b) Conditional probability if P(b) 0 Product rule gives an alternative formulation: P(a b) = P(a b)p(b) = P(b a)p(a) A general version holds for whole distributions, e.g., P(W eather, Cavity) = P(W eather Cavity)P(Cavity) (View as a 4 2 set of equations, not matrix mult.) Chain rule is derived by successive application of product rule: P(X 1,...,X n ) = P(X 1,...,X n 1 ) P(X n X 1,...,X n 1 ) = P(X 1,...,X n 2 ) P(X n 1 X 1,...,X n 2 ) P(X n X 1,...,X n 1 ) =... = Π n i = 1P(X i X 1,...,X i 1 ) 11

12 L L Inference by enumeration Start with the joint distribution: toothache cavity Lcavity Ltoothache For any proposition φ, sum the atomic events where it is true: P(φ) = Σ ω:ω =φ P(ω) 12

13 L L Inference by enumeration Start with the joint distribution: toothache cavity Lcavity Ltoothache For any proposition φ, sum the atomic events where it is true: P(φ) = Σ ω:ω =φ P(ω) P(toothache) = =

14 L L Inference by enumeration Start with the joint distribution: toothache cavity Lcavity Ltoothache For any proposition φ, sum the atomic events where it is true: P(φ) = Σ ω:ω =φ P(ω) P(cavity toothache) = =

15 L L Inference by enumeration Start with the joint distribution: toothache cavity Lcavity Ltoothache Can also compute conditional probabilities: P( cavity toothache) = = P( cavity toothache) P(toothache) =

16 L L Normalization cavity Lcavity toothache Ltoothache Denominator can be viewed as a normalization constant α P(Cavity toothache) = α P(Cavity, toothache) = α [P(Cavity, toothache, ) + P(Cavity, toothache, )] = α [ 0.108, , ] = α 0.12, 0.08 = 0.6, 0.4 General idea: compute distribution on query variable by fixing evidence variables and summing over hidden variables 16

17 Inference by enumeration, contd. Let X be all the variables. Typically, we want the posterior joint distribution of the query variables Y given specific values e for the evidence variables E Let the hidden variables be H = X Y E Then the required summation of joint entries is done by summing out the hidden variables: P(Y E =e) = αp(y,e=e) = ασ h P(Y,E=e,H=h) The terms in the summation are joint entries because Y, E, and H together exhaust the set of random variables Obvious problems: 1) Worst-case time complexity O(d n ) where d is the largest arity 2) Space complexity O(d n ) to store the joint distribution 3) How to find the numbers for O(d n ) entries??? 17

18 Independence A and B are independent iff P(A B) = P(A) or P(B A) = P(B) or P(A, B) = P(A)P(B) Toothache Cavity Weather Catch decomposes into P(T oothache, Catch, Cavity, W eather) = P(T oothache, Catch, Cavity)P(W eather) Cavity Toothache Catch Weather 32 entries reduced to 12; for n independent biased coins, 2 n n Absolute independence powerful but rare Dentistry is a large field with hundreds of variables, none of which are independent. What to do? 18

19 Conditional independence P(Toothache,Cavity,Catch) has = 7 independent entries If I have a cavity, the probability that the probe es in it doesn t depend on whether I have a toothache: (1) P( toothache, cavity) = P( cavity) The same independence holds if I haven t got a cavity: (2) P( toothache, cavity) = P( cavity) Catch is conditionally independent of Toothache given Cavity: P(Catch T oothache, Cavity) = P(Catch Cavity) Equivalent statements: P(T oothache Catch, Cavity) = P(T oothache Cavity) P(T oothache, Catch Cavity) = P(T oothache Cavity)P(Catch Cavity) 19

20 Conditional independence contd. Write out full joint distribution using chain rule: P(T oothache, Catch, Cavity) = P(T oothache Catch, Cavity)P(Catch, Cavity) = P(T oothache Catch, Cavity)P(Catch Cavity)P(Cavity) = P(T oothache Cavity)P(Catch Cavity)P(Cavity) I.e., = 5 independent numbers (equations 1 and 2 remove 2) In most cases, the use of conditional independence reduces the size of the representation of the joint distribution from exponential in n to linear in n. Conditional independence is our most basic and robust form of knowledge about uncertain environments. 20

21 Bayes theorem Product rule P(a b) = P(a b)p(b) = P(b a)p(a) Bayes theorem: P(a b) = P(b a)p(a) P(b) or in distribution form P(Y X) = P(X Y )P(Y ) P(X) = αp(x Y )P(Y ) Useful for assessing diagnostic probability from causal probability: P(Cause Effect) = P(Effect Cause)P(Cause) P(Effect) E.g., let M be meningitis, S be stiff neck: P(m s) = P(s m)p(m) P(s) = = Note: posterior probability of meningitis still very small! 21

22 Bayes theorem and conditional independence P(Cavity toothache ) = α P(toothache Cavity)P(Cavity) = α P(toothache Cavity)P( Cavity)P(Cavity) This is an example of a naive Bayes model: P(Cause,Effect 1,...,Effect n ) = P(Cause)Π i P(Effect i Cause) Cavity Cause Toothache Catch Effect 1 Effect n Total number of parameters is linear in n 22

### Uncertainty. Chapter 13. Chapter 13 1

Uncertainty Chapter 13 Chapter 13 1 Attribution Modified from Stuart Russell s slides (Berkeley) Chapter 13 2 Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes Rule

### Artificial Intelligence

Artificial Intelligence ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #13 Uncertainty Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty

### Outline. Uncertainty. Uncertainty. Making decisions under uncertainty. Probability. Methods for handling uncertainty

Outline Uncertainty Chapter 13 Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty Let action A t = leave for airport t minutes before flight Will A t get me

### / Informatica. Intelligent Systems. Agents dealing with uncertainty. Example: driving to the airport. Why logic fails

Intelligent Systems Prof. dr. Paul De Bra Technische Universiteit Eindhoven debra@win.tue.nl Agents dealing with uncertainty Uncertainty Probability Syntax and Semantics Inference Independence and Bayes'

### 13.3 Inference Using Full Joint Distribution

191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent

### 22c:145 Artificial Intelligence

22c:145 Artificial Intelligence Fall 2005 Uncertainty Cesare Tinelli The University of Iowa Copyright 2001-05 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material and may not be used

### Marginal Independence and Conditional Independence

Marginal Independence and Conditional Independence Computer Science cpsc322, Lecture 26 (Textbook Chpt 6.1-2) March, 19, 2010 Recap with Example Marginalization Conditional Probability Chain Rule Lecture

### Logic, Probability and Learning

Logic, Probability and Learning Luc De Raedt luc.deraedt@cs.kuleuven.be Overview Logic Learning Probabilistic Learning Probabilistic Logic Learning Closely following : Russell and Norvig, AI: a modern

### Probability, Conditional Independence

Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms

### CS 771 Artificial Intelligence. Reasoning under uncertainty

CS 771 Artificial Intelligence Reasoning under uncertainty Today Representing uncertainty is useful in knowledge bases Probability provides a coherent framework for uncertainty Review basic concepts in

### CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization

Full Joint Probability Distributions CS 331: Artificial Intelligence Fundamentals of Probability II Toothache Cavity Catch Toothache, Cavity, Catch false false false 0.576 false false true 0.144 false

### Artificial Intelligence. Conditional probability. Inference by enumeration. Independence. Lesson 11 (From Russell & Norvig)

Artificial Intelligence Conditional probability Conditional or posterior probabilities e.g., cavity toothache) = 0.8 i.e., given that toothache is all I know tation for conditional distributions: Cavity

### Outline. Probability. Random Variables. Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule.

Probability 1 Probability Random Variables Outline Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule Inference Independence 2 Inference in Ghostbusters A ghost

### Introduction to Probability (1)

Introduction to Probability (1) John Kelleher & Brian Mac Namee 1 Introduction Summarizing Uncertainty 2 Where Do Probabilities Come From? 3 Basic Probability Notation Basics 4 The Axioms of Probabilities

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### BIN504 - Lecture XI. Bayesian Inference & Markov Chains

BIN504 - Lecture XI Bayesian Inference & References: Lee, Chp. 11 & Sec. 10.11 Ewens-Grant, Chps. 4, 7, 11 c 2012 Aybar C. Acar Including slides by Tolga Can Rev. 1.2 (Build 20130528191100) BIN 504 - Bayes

### Markov Chains, part I

Markov Chains, part I December 8, 2010 1 Introduction A Markov Chain is a sequence of random variables X 0, X 1,, where each X i S, such that P(X i+1 = s i+1 X i = s i, X i 1 = s i 1,, X 0 = s 0 ) = P(X

### Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1

Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields

### Statistics in Geophysics: Introduction and Probability Theory

Statistics in Geophysics: Introduction and Steffen Unkel Department of Statistics Ludwig-Maximilians-University Munich, Germany Winter Term 2013/14 1/32 What is Statistics? Introduction Statistics is the

### CS 188: Artificial Intelligence. Probability recap

CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability

### Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab

Lecture 2 Probability BMIR Lecture Series in Probability and Statistics Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab 2.1 1 Sample Spaces and Events Random

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES Contents 1. Probabilistic experiments 2. Sample space 3. Discrete probability

### Set theory, and set operations

1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### Probability II (MATH 2647)

Probability II (MATH 2647) Lecturer Dr. O. Hryniv email Ostap.Hryniv@durham.ac.uk office CM309 http://maths.dur.ac.uk/stats/courses/probmc2h/probability2h.html or via DUO This term we shall consider: Review

### Introduction to Discrete Probability Theory and Bayesian Networks

Introduction to Discrete Probability Theory and Bayesian Networks Dr Michael Ashcroft September 15, 2011 This document remains the property of Inatas. Reproduction in whole or in part without the written

### Math/Stat 360-1: Probability and Statistics, Washington State University

Math/Stat 360-1: Probability and Statistics, Washington State University Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 3 Haijun Li Math/Stat 360-1: Probability and

### Probability and statistics; Rehearsal for pattern recognition

Probability and statistics; Rehearsal for pattern recognition Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception

### Topic 2: Scalar random variables. Definition of random variables

Topic 2: Scalar random variables Discrete and continuous random variables Probability distribution and densities (cdf, pmf, pdf) Important random variables Expectation, mean, variance, moments Markov and

### Probability OPRE 6301

Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.

### Mathematical foundations of Econometrics

Mathematical foundations of Econometrics G.Gioldasis, UniFe & prof. A.Musolesi, UniFe April 7, 2016.Gioldasis, UniFe & prof. A.Musolesi, UniFe Mathematical foundations of Econometrics April 7, 2016 1 /

### The basics of probability theory. Distribution of variables, some important distributions

The basics of probability theory. Distribution of variables, some important distributions 1 Random experiment The outcome is not determined uniquely by the considered conditions. For example, tossing a

### Probability - Part I. Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty.

Probability - Part I Definition : A random experiment is an experiment or a process for which the outcome cannot be predicted with certainty. Definition : The sample space (denoted S) of a random experiment

### General theory of stochastic processes

CHAPTER 1 General theory of stochastic processes 1.1. Definition of stochastic process First let us recall the definition of a random variable. A random variable is a random number appearing as a result

### PROBABILITY. Chapter Overview Conditional Probability

PROBABILITY Chapter. Overview.. Conditional Probability If E and F are two events associated with the same sample space of a random experiment, then the conditional probability of the event E under the

### Chapter 1. Probability Theory: Introduction. Definitions Algebra. RS Chapter 1 Random Variables

Chapter 1 Probability Theory: Introduction Definitions Algebra Definitions: Semiring A collection of sets F is called a semiring if it satisfies: Φ F. If A, B F, then A B F. If A, B F, then there exists

### An Introduction to First-Order Logic

Outline An Introduction to K. 1 1 Lane Department of Computer Science and Electrical Engineering West Virginia University Syntax, Semantics and Outline Outline 1 The Syntax of 2 of Expressions Model Different

### MATH 56A SPRING 2008 STOCHASTIC PROCESSES 31

MATH 56A SPRING 2008 STOCHASTIC PROCESSES 3.3. Invariant probability distribution. Definition.4. A probability distribution is a function π : S [0, ] from the set of states S to the closed unit interval

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### Contents. TTM4155: Teletraffic Theory (Teletrafikkteori) Probability Theory Basics. Yuming Jiang. Basic Concepts Random Variables

TTM4155: Teletraffic Theory (Teletrafikkteori) Probability Theory Basics Yuming Jiang 1 Some figures taken from the web. Contents Basic Concepts Random Variables Discrete Random Variables Continuous Random

### Proof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems.

Math 232 - Discrete Math 2.1 Direct Proofs and Counterexamples Notes Axiom: Proposition that is assumed to be true. Proof: A logical argument establishing the truth of the theorem given the truth of the

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)

Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision

### Statistiek (WISB361)

Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17

### Gaussian Classifiers CS498

Gaussian Classifiers CS498 Today s lecture The Gaussian Gaussian classifiers A slightly more sophisticated classifier Nearest Neighbors We can classify with nearest neighbors x m 1 m 2 Decision boundary

### Bayesian Classification

CS 650: Computer Vision Bryan S. Morse BYU Computer Science Statistical Basis Training: Class-Conditional Probabilities Suppose that we measure features for a large training set taken from class ω i. Each

### Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### Renewal Theory. (iv) For s < t, N(t) N(s) equals the number of events in (s, t].

Renewal Theory Def. A stochastic process {N(t), t 0} is said to be a counting process if N(t) represents the total number of events that have occurred up to time t. X 1, X 2,... times between the events

### Math 312 Lecture Notes Markov Chains

Math 312 Lecture Notes Markov Chains Warren Weckesser Department of Mathematics Colgate University Updated, 30 April 2005 Markov Chains A (finite) Markov chain is a process with a finite number of states

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### NOTES ON ELEMENTARY PROBABILITY

NOTES ON ELEMENTARY PROBABILITY KARL PETERSEN 1. Probability spaces Probability theory is an attempt to work mathematically with the relative uncertainties of random events. In order to get started, we

### Artificial Intelligence II

Artificial Intelligence II Some supplementary notes on probability Sean B Holden, 2010-14 1 Introduction These notes provide a reminder of some simple manipulations that turn up a great deal when dealing

### A Tutorial on Probability Theory

Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,

### Markov Chains. Chapter Introduction and Definitions 110SOR201(2002)

page 5 SOR() Chapter Markov Chains. Introduction and Definitions Consider a sequence of consecutive times ( or trials or stages): n =,,,... Suppose that at each time a probabilistic experiment is performed,

### Mathematical Background

Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,

### Math/Stat 370: Engineering Statistics, Washington State University

Math/Stat 370: Engineering Statistics, Washington State University Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 2 Haijun Li Math/Stat 370: Engineering Statistics,

### Math/CSE 1019: Discrete Mathematics for Computer Science Fall Suprakash Datta

Math/CSE 1019: Discrete Mathematics for Computer Science Fall 2011 Suprakash Datta datta@cse.yorku.ca Office: CSEB 3043 Phone: 416-736-2100 ext 77875 Course page: http://www.cse.yorku.ca/course/1019 1

### Probability & Probability Distributions

Probability & Probability Distributions Carolyn J. Anderson EdPsych 580 Fall 2005 Probability & Probability Distributions p. 1/61 Probability & Probability Distributions Elementary Probability Theory Definitions

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Probability for Estimation (review)

Probability for Estimation (review) In general, we want to develop an estimator for systems of the form: x = f x, u + η(t); y = h x + ω(t); ggggg y, ffff x We will primarily focus on discrete time linear

### If rules "a" and "a => b ^ c" exist in the rule base, show the steps necessary to prove "c" is true

Fill in the following truth table: a b a ^ b a v b a => b a b false false false true true false true true If probability of event a (P(a))=.2, P(b)=.3, P(c)=.5, P(a ^ b)=.1 (a AND b), P(a ^ c)=.15,

### EE 302 Division 1. Homework 5 Solutions.

EE 32 Division. Homework 5 Solutions. Problem. A fair four-sided die (with faces labeled,, 2, 3) is thrown once to determine how many times a fair coin is to be flipped: if N is the number that results

### 5. Conditional Expected Value

Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 5. Conditional Expected Value As usual, our starting point is a random experiment with probability measure P on a sample space Ω. Suppose that X is

### Random Variables and Their Expected Values

Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution Function Discrete and Continuous Random Variables The Probability Mass Function The (Cumulative) Distribution

### m (t) = e nt m Y ( t) = e nt (pe t + q) n = (pe t e t + qe t ) n = (qe t + p) n

1. For a discrete random variable Y, prove that E[aY + b] = ae[y] + b and V(aY + b) = a 2 V(Y). Solution: E[aY + b] = E[aY] + E[b] = ae[y] + b where each step follows from a theorem on expected value from

### Finite Markov Chains and Algorithmic Applications. Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet

Finite Markov Chains and Algorithmic Applications Olle Häggström Matematisk statistik, Chalmers tekniska högskola och Göteborgs universitet PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE

### Chapter ML:IV. IV. Statistical Learning. Probability Basics Bayes Classification Maximum a-posteriori Hypotheses

Chapter ML:IV IV. Statistical Learning Probability Basics Bayes Classification Maximum a-posteriori Hypotheses ML:IV-1 Statistical Learning STEIN 2005-2015 Area Overview Mathematics Statistics...... Stochastics

### The Calculus of Probability

The Calculus of Probability Let A and B be events in a sample space S. Partition rule: P(A) = P(A B) + P(A B ) Example: Roll a pair of fair dice P(Total of 10) = P(Total of 10 and double) + P(Total of

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Math 150 Sample Exam #2

Problem 1. (16 points) TRUE or FALSE. a. 3 die are rolled, there are 1 possible outcomes. b. If two events are complementary, then they are mutually exclusive events. c. If A and B are two independent

### Probabilities for machine learning

Probabilities for machine learning Roland Memisevic February 2, 2006 Why probabilities? One of the hardest problems when building complex intelligent systems is brittleness. How can we keep tiny irregularities

### 15-381: Artificial Intelligence. Probabilistic Reasoning and Inference

5-38: Artificial Intelligence robabilistic Reasoning and Inference Advantages of probabilistic reasoning Appropriate for complex, uncertain, environments - Will it rain tomorrow? Applies naturally to many

### Introduction to probability theory in the Discrete Mathematics course

Introduction to probability theory in the Discrete Mathematics course Jiří Matoušek (KAM MFF UK) Version: Oct/18/2013 Introduction This detailed syllabus contains definitions, statements of the main results

### Expected Value 10/11/2005

Expected Value 10/11/2005 Definition Let X be a numerically-valued discrete random variable with sample space Ω and distribution function m(x). The expected value E(X) is defined by E(X) = x Ω xm(x), provided

### CHAPTER III - MARKOV CHAINS

CHAPTER III - MARKOV CHAINS JOSEPH G. CONLON 1. General Theory of Markov Chains We have already discussed the standard random walk on the integers Z. A Markov Chain can be viewed as a generalization of

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### Probability Review. ICPSR Applied Bayesian Modeling

Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

### Random Variables, Expectation, Distributions

Random Variables, Expectation, Distributions CS 5960/6960: Nonparametric Methods Tom Fletcher January 21, 2009 Review Random Variables Definition A random variable is a function defined on a probability

### LECTURE 4. Conditional Probability and Bayes Theorem

LECTURE 4 Conditional Probability and Bayes Theorem 1 The conditional sample space Physical motivation for the formal mathematical theory 1. Roll a fair die once so the sample space S is given by S {1,

### Elements of Probability Distribution Theory

Chapter 1 Elements of Probability Distribution Theory 1.1 Introductory Definitions Statistics gives us methods to make inference about a population based on a random sample representing this population.

### Non-Markovian Logic-Probabilistic Modeling and Inference

Non-Markovian Logic-Probabilistic Modeling and Inference Eduardo Menezes de Morais Glauber De Bona Marcelo Finger Department of Computer Science University of São Paulo São Paulo, Brazil WL4AI 2015 What

### Basic Probability Theory (I)

Basic Probability Theory (I) Intro to Bayesian Data Analysis & Cognitive Modeling Adrian Brasoveanu [partly based on slides by Sharon Goldwater & Frank Keller and John K. Kruschke] Fall 2012 UCSC Linguistics

### The Foundations: Logic and Proofs. Chapter 1, Part III: Proofs

The Foundations: Logic and Proofs Chapter 1, Part III: Proofs Rules of Inference Section 1.6 Section Summary Valid Arguments Inference Rules for Propositional Logic Using Rules of Inference to Build Arguments

### Bayesian Tutorial (Sheet Updated 20 March)

Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that

### Continuous random variables

Continuous random variables So far we have been concentrating on discrete random variables, whose distributions are not continuous. Now we deal with the so-called continuous random variables. A random

10-601 Machine Learning http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html Course data All up-to-date info is on the course web page: http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html

### f(x) is a singleton set for all x A. If f is a function and f(x) = {y}, we normally write

Math 525 Chapter 1 Stuff If A and B are sets, then A B = {(x,y) x A, y B} denotes the product set. If S A B, then S is called a relation from A to B or a relation between A and B. If B = A, S A A is called

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Example. Two fair dice are tossed and the two outcomes recorded. As is familiar, we have

Lectures 9-10 jacques@ucsd.edu 5.1 Random Variables Let (Ω, F, P ) be a probability space. The Borel sets in R are the sets in the smallest σ- field on R that contains all countable unions and complements

### Welcome to Stochastic Processes 1. Welcome to Aalborg University No. 1 of 31

Welcome to Stochastic Processes 1 Welcome to Aalborg University No. 1 of 31 Welcome to Aalborg University No. 2 of 31 Course Plan Part 1: Probability concepts, random variables and random processes Lecturer:

### Lecture 9: Bayesian Learning

Lecture 9: Bayesian Learning Cognitive Systems II - Machine Learning SS 2005 Part II: Special Aspects of Concept Learning Bayes Theorem, MAL / ML hypotheses, Brute-force MAP LEARNING, MDL principle, Bayes

### Continuous Random Variables

Continuous Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Continuous Random Variables 2 11 Introduction 2 12 Probability Density Functions 3 13 Transformations 5 2 Mean, Variance and Quantiles

### 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is:

CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx

### Math 421: Probability and Statistics I Note Set 2

Math 421: Probability and Statistics I Note Set 2 Marcus Pendergrass September 13, 2013 4 Discrete Probability Discrete probability is concerned with situations in which you can essentially list all the