CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization

Size: px
Start display at page:

Download "CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization"

Transcription

1 Full Joint Probability Distributions CS 331: Artificial Intelligence Fundamentals of Probability II Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true Catch means the dentist s probe catches in my teeth Thanks to Andrew Moore for some course material 1 This cell means Toothache = true, Cavity = true, Catch = true = Full Joint Probability Distributions Joint Probability Distribution Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true From the full joint probability distribution, we can calculate any probability involving the three random variables in this world eg. Toothache = true OR Cavity = true = Toothache=true true, Cavity=false, Catch=false + Toothache=true, Cavity=false, Catch=true + Toothache=false, Cavity=true, Catch=false + Toothache=false, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false + Toothache=true, Cavity = true, Catch=true + = = 0.28 The probabilities in the last column sum to 1 3 Marginalization We can even calculate marginal probabilities (the probability distribution over a subset of the variables eg: Toothache=true true, Cavity=true true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false = = 0.12 Marginalization Or even: Cavity=true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false Toothache=false, Cavity=true, Catch=true + Toothache=false, Cavity=true, Catch=false = =

2 Marginalization The general marginalization rule for any sets of variables Y and Z: Y Y, z or z Y Y z z z z is over all possible combinations of values of Z (remember Z is a set Marginalization For continuous variables, marginalization involves taking the integral: Y Y, z dz 7 8 Normalization Cavity true Toothache true Cavity true, Toothache true Toothache true Cavity false Toothache true Cavity false, Toothache true Toothache true Note that t 1/Toothache=true h t remains constant in the two equations. Normalization In fact, 1/toothache can be viewed as a normalization constant for Cavity toothache, ensuring it adds up to 1 We will refer to normalization constants with the symbol Cavity toothache Cavity, toothache 10 Inference Suppose you get a query such as Cavity = true Toothache = true Toothache is called the evidence variable because we observe it. More generally, it s a set of variables. Cavity is called the query variable (we ll assume it s a single variable for now There are also unobserved (aka hidden variables like Catch 11 Inference We will write the query as X e This is a probability distribution hence the boldface X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables 12 2

3 Inference We will write the query as X e P ( X e X, e X, e, y Summation is over all possible combinations of values of the unobserved variables Y X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables y Inference P ( X e X, e X, e, y Computing X e involves going through all possible entries of the full joint probability distribution and adding up probabilities with X=x i, E=e, and Y=y Suppose you have a domain with n Boolean variables. What is the space and time complexity of computing X e? y 14 How do you avoid the exponential space and time complexity of inference? Use independence (aka factoring Suppose the full joint distribution now consists of four variables: Toothache = {true, false} Catch ={true {true, false} Cavity = {true, false} Weather = {sunny, rain, cloudy, snow} There are now 32 entries in the full joint distribution table Does the weather influence one s dental problems? Is Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy? In other words, is Weather independent of Toothache, Catch and Cavity? We say that variables X and Y are independent if any of the following hold: (note that they are all equivalent P ( X Y P ( X or Y X Y or X, Y X Y

4 Why is independence useful? Assume that Weather is independent of toothache, catch, cavity ie. Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Why is independence useful? Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity Now we can calculate: l Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity * toothache, catch, cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity 19 This table has 4 values This table has 8 values You now need to store 12 values to calculate Weather, Toothache, Catch, Cavity If Weather was not independent of Toothache, Catch, and Cavity then you would have needed 32 values 20 Another example: Suppose you have n coin flips and you want to calculate the joint distribution C 1,, C n If the coin flips are not independent, you need 2 n values in the table If the coin flips are independent, then n 1,..., Cn C i i1 C Each C i table has 2 entries and there are n of them for a total of 2n values is powerful! It required extra domain knowledge. A different kind of knowledge than numerical probabilities. It needed an understanding of causation Bayes Rule The product rule can be written in two ways: A, B = A BB A, B = B Bayes Rule More generally, the following is known as Bayes Rule: B A B B Note that these are distributions Sometimes, you can treat B as a normalization constant α You can combine the equations above to get: A B B B A B B

5 More General Forms of Bayes Rule Bayes Rule Example If A takes 2 values: A B If A takes k values: Av B i Av Av n A k1 i Av Av k i k 25 Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m (0.5( m s s 0.05 Bayes Rule Example Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m P m s s Note: Even though s m = 0.5, m s = (0.5( ( When is Bayes Rule Useful? Sometimes it s easier to get X Y than Y X. Information is typically available in the form effect cause rather than cause effect For example, symptom disease is easy to measure empirically but obtaining disease symptom is harder 28 How is Bayes Rule Used In machine learning, we use Bayes rule in the following way: Bayes Rule With More Than One Piece of Evidence Suppose you now have 2 evidence variables Toothache = true and Catch = catch (note that Cavity is uninstantiated below Likelihood of the data Prior probability D h h h D D Posterior probability h = hypothesis D = data 29 Cavity Toothache = true, Catch = true = α Toothache h h = true, Catch = true Cavity Cavity i In order to calculate Toothache = true, Catch = true Cavity, you need a table of 4 probability values. With N evidence variables, you need 2 N probability values. 30 5

6 Conditional Are Toothache and Catch independent? No if probe catches in the tooth, it likely has a cavity which causes the toothache. But given the presence or absence of the cavity, they are independent d (since they are directly caused dby the cavity but don t have a direct effect on each other Conditional independence: Toothache = true, Catch = catch Cavity = Toothache = true Cavity * Catch = true Cavity 31 Conditional General form: A, B C A C B C Or equivalently: A B, C A C B A, C B C and How to think about conditional independence: In A B, C = A C: if knowing C tells me everything about A, I don t gain anything by knowing B 32 Conditional 7 independent values in table (have to sum to 1 Toothache, Catch, Cavity = Toothache, (, Catch Cavity Cavity ( = Toothache Cavity Catch Cavity Cavity What You Should Know How to do inference in joint probability distributions How to use Bayes Rule Why independence and conditional independence is useful 2 independent values in table 2 independent values in table 1 independent value in table Conditional independence permits probabilistic systems to scale up!

13.3 Inference Using Full Joint Distribution

13.3 Inference Using Full Joint Distribution 191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent

More information

Outline. Probability. Random Variables. Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule.

Outline. Probability. Random Variables. Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule. Probability 1 Probability Random Variables Outline Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule Inference Independence 2 Inference in Ghostbusters A ghost

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #13 Uncertainty Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty

More information

/ Informatica. Intelligent Systems. Agents dealing with uncertainty. Example: driving to the airport. Why logic fails

/ Informatica. Intelligent Systems. Agents dealing with uncertainty. Example: driving to the airport. Why logic fails Intelligent Systems Prof. dr. Paul De Bra Technische Universiteit Eindhoven debra@win.tue.nl Agents dealing with uncertainty Uncertainty Probability Syntax and Semantics Inference Independence and Bayes'

More information

Outline. Uncertainty. Uncertainty. Making decisions under uncertainty. Probability. Methods for handling uncertainty

Outline. Uncertainty. Uncertainty. Making decisions under uncertainty. Probability. Methods for handling uncertainty Outline Uncertainty Chapter 13 Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty Let action A t = leave for airport t minutes before flight Will A t get me

More information

Logic, Probability and Learning

Logic, Probability and Learning Logic, Probability and Learning Luc De Raedt luc.deraedt@cs.kuleuven.be Overview Logic Learning Probabilistic Learning Probabilistic Logic Learning Closely following : Russell and Norvig, AI: a modern

More information

Basics of Probability

Basics of Probability Basics of Probability 1 Sample spaces, events and probabilities Begin with a set Ω the sample space e.g., 6 possible rolls of a die. ω Ω is a sample point/possible world/atomic event A probability space

More information

22c:145 Artificial Intelligence

22c:145 Artificial Intelligence 22c:145 Artificial Intelligence Fall 2005 Uncertainty Cesare Tinelli The University of Iowa Copyright 2001-05 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material and may not be used

More information

Artificial Intelligence. Conditional probability. Inference by enumeration. Independence. Lesson 11 (From Russell & Norvig)

Artificial Intelligence. Conditional probability. Inference by enumeration. Independence. Lesson 11 (From Russell & Norvig) Artificial Intelligence Conditional probability Conditional or posterior probabilities e.g., cavity toothache) = 0.8 i.e., given that toothache is all I know tation for conditional distributions: Cavity

More information

Uncertainty. Chapter 13. Chapter 13 1

Uncertainty. Chapter 13. Chapter 13 1 Uncertainty Chapter 13 Chapter 13 1 Attribution Modified from Stuart Russell s slides (Berkeley) Chapter 13 2 Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes Rule

More information

CS 188: Artificial Intelligence. Probability recap

CS 188: Artificial Intelligence. Probability recap CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability

More information

15-381: Artificial Intelligence. Probabilistic Reasoning and Inference

15-381: Artificial Intelligence. Probabilistic Reasoning and Inference 5-38: Artificial Intelligence robabilistic Reasoning and Inference Advantages of probabilistic reasoning Appropriate for complex, uncertain, environments - Will it rain tomorrow? Applies naturally to many

More information

Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)

Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision

More information

Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) = Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

More information

Lecture 2: Introduction to belief (Bayesian) networks

Lecture 2: Introduction to belief (Bayesian) networks Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (I-maps) January 7, 2008 1 COMP-526 Lecture 2 Recall from last time: Conditional

More information

CS 771 Artificial Intelligence. Reasoning under uncertainty

CS 771 Artificial Intelligence. Reasoning under uncertainty CS 771 Artificial Intelligence Reasoning under uncertainty Today Representing uncertainty is useful in knowledge bases Probability provides a coherent framework for uncertainty Review basic concepts in

More information

Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify

More information

Probability Review. ICPSR Applied Bayesian Modeling

Probability Review. ICPSR Applied Bayesian Modeling Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

More information

Marginal Independence and Conditional Independence

Marginal Independence and Conditional Independence Marginal Independence and Conditional Independence Computer Science cpsc322, Lecture 26 (Textbook Chpt 6.1-2) March, 19, 2010 Recap with Example Marginalization Conditional Probability Chain Rule Lecture

More information

Reasoning Component Architecture

Reasoning Component Architecture Architecture of a Spam Filter Application By Avi Pfeffer A spam filter consists of two components. In this article, based on my book Practical Probabilistic Programming, first describe the architecture

More information

Section 6.1 Joint Distribution Functions

Section 6.1 Joint Distribution Functions Section 6.1 Joint Distribution Functions We often care about more than one random variable at a time. DEFINITION: For any two random variables X and Y the joint cumulative probability distribution function

More information

Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11 CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

More information

Introduction to Game Theory IIIii. Payoffs: Probability and Expected Utility

Introduction to Game Theory IIIii. Payoffs: Probability and Expected Utility Introduction to Game Theory IIIii Payoffs: Probability and Expected Utility Lecture Summary 1. Introduction 2. Probability Theory 3. Expected Values and Expected Utility. 1. Introduction We continue further

More information

Chapter 4. Probability and Probability Distributions

Chapter 4. Probability and Probability Distributions Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

More information

General Naïve Bayes. CS 188: Artificial Intelligence Fall Example: Spam Filtering. Example: OCR. Generalization and Overfitting

General Naïve Bayes. CS 188: Artificial Intelligence Fall Example: Spam Filtering. Example: OCR. Generalization and Overfitting CS 88: Artificial Intelligence Fall 7 Lecture : Perceptrons //7 General Naïve Bayes A general naive Bayes model: C x E n parameters C parameters n x E x C parameters E E C E n Dan Klein UC Berkeley We

More information

The Basics of Graphical Models

The Basics of Graphical Models The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

More information

Querying Joint Probability Distributions

Querying Joint Probability Distributions Querying Joint Probability Distributions Sargur Srihari srihari@cedar.buffalo.edu 1 Queries of Interest Probabilistic Graphical Models (BNs and MNs) represent joint probability distributions over multiple

More information

Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing

Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and

More information

1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error.

1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error. 1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error. 8.5 Goodness of Fit Test Suppose we want to make an inference about

More information

Statistical Natural Language Processing

Statistical Natural Language Processing Statistical Natural Language Processing Prasad Tadepalli CS430 lecture Natural Language Processing Some subproblems are partially solved Spelling correction, grammar checking Information retrieval with

More information

Machine Learning

Machine Learning Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 8, 2011 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

More information

4.4 Equations of the Form ax + b = cx + d

4.4 Equations of the Form ax + b = cx + d 4.4 Equations of the Form ax + b = cx + d We continue our study of equations in which the variable appears on both sides of the equation. Suppose we are given the equation: 3x + 4 = 5x! 6 Our first step

More information

1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016

1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016 and P (B X). In order to do find those conditional probabilities, we ll use Bayes formula. We can easily compute the reverse probabilities A short introduction to Bayesian statistics, part I Math 18, Mathematical

More information

Graphical Models. CS 343: Artificial Intelligence Bayesian Networks. Raymond J. Mooney. Conditional Probability Tables.

Graphical Models. CS 343: Artificial Intelligence Bayesian Networks. Raymond J. Mooney. Conditional Probability Tables. Graphical Models CS 343: Artificial Intelligence Bayesian Networks Raymond J. Mooney University of Texas at Austin 1 If no assumption of independence is made, then an exponential number of parameters must

More information

MAT 1000. Mathematics in Today's World

MAT 1000. Mathematics in Today's World MAT 1000 Mathematics in Today's World We talked about Cryptography Last Time We will talk about probability. Today There are four rules that govern probabilities. One good way to analyze simple probabilities

More information

5.7 Literal Equations

5.7 Literal Equations 5.7 Literal Equations Now that we have learned to solve a variety of different equations (linear equations in chapter 2, polynomial equations in chapter 4, and rational equations in the last section) we

More information

The Foundations: Logic and Proofs. Chapter 1, Part III: Proofs

The Foundations: Logic and Proofs. Chapter 1, Part III: Proofs The Foundations: Logic and Proofs Chapter 1, Part III: Proofs Rules of Inference Section 1.6 Section Summary Valid Arguments Inference Rules for Propositional Logic Using Rules of Inference to Build Arguments

More information

Bayesian Networks. Mausam (Slides by UW-AI faculty)

Bayesian Networks. Mausam (Slides by UW-AI faculty) Bayesian Networks Mausam (Slides by UW-AI faculty) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation & inference BNs provide

More information

Climbing an Infinite Ladder

Climbing an Infinite Ladder Section 5.1 Climbing an Infinite Ladder Suppose we have an infinite ladder and the following capabilities: 1. We can reach the first rung of the ladder. 2. If we can reach a particular rung of the ladder,

More information

1. Chi-Squared Tests

1. Chi-Squared Tests 1. Chi-Squared Tests We'll now look at how to test statistical hypotheses concerning nominal data, and specifically when nominal data are summarized as tables of frequencies. The tests we will considered

More information

7 Probability. Copyright Cengage Learning. All rights reserved.

7 Probability. Copyright Cengage Learning. All rights reserved. 7 Probability Copyright Cengage Learning. All rights reserved. 7.1 Sample Spaces and Events Copyright Cengage Learning. All rights reserved. Sample Spaces 3 Sample Spaces At the beginning of a football

More information

A crash course in probability and Naïve Bayes classification

A crash course in probability and Naïve Bayes classification Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

More information

Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1

Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1 Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields

More information

CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam)

CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam) CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam) Problem Points Possible 1. Perceptrons and SVMs 20 2. Probabilistic Models 20 3. Markov Models

More information

Probability, Conditional Independence

Probability, Conditional Independence Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms

More information

FOR INTERNAL SCRUTINY (date of this version: 13/3/2009) UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS

FOR INTERNAL SCRUTINY (date of this version: 13/3/2009) UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS COGNITIVE MODELLING (LEVEL 10) COGNITIVE MODELLING (LEVEL 11) Friday 1 April 2005 00:00 to 00:00 Year 4 Courses Convener:

More information

Bayes Rule. How is this rule derived? Using Bayes rule for probabilistic inference: Evidence Cause) Evidence)

Bayes Rule. How is this rule derived? Using Bayes rule for probabilistic inference: Evidence Cause) Evidence) Bayes Rule P ( A B = B A A B ( Rev. Thomas Bayes (1702-1761 How is this rule derived? Using Bayes rule for probabilistic inference: P ( Cause Evidence = Evidence Cause Cause Evidence Cause Evidence: diagnostic

More information

Bayesian probability theory

Bayesian probability theory Bayesian probability theory Bruno A. Olshausen arch 1, 2004 Abstract Bayesian probability theory provides a mathematical framework for peforming inference, or reasoning, using probability. The foundations

More information

Statistical Inference

Statistical Inference Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this

More information

Chi Square for Contingency Tables

Chi Square for Contingency Tables 2 x 2 Case Chi Square for Contingency Tables A test for p 1 = p 2 We have learned a confidence interval for p 1 p 2, the difference in the population proportions. We want a hypothesis testing procedure

More information

CS 341 Homework 9 Languages That Are and Are Not Regular

CS 341 Homework 9 Languages That Are and Are Not Regular CS 341 Homework 9 Languages That Are and Are Not Regular 1. Show that the following are not regular. (a) L = {ww R : w {a, b}*} (b) L = {ww : w {a, b}*} (c) L = {ww' : w {a, b}*}, where w' stands for w

More information

Bayesian Networks Chapter 14. Mausam (Slides by UW-AI faculty & David Page)

Bayesian Networks Chapter 14. Mausam (Slides by UW-AI faculty & David Page) Bayesian Networks Chapter 14 Mausam (Slides by UW-AI faculty & David Page) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation

More information

Artificial Intelligence II

Artificial Intelligence II Artificial Intelligence II Some supplementary notes on probability Sean B Holden, 2010-14 1 Introduction These notes provide a reminder of some simple manipulations that turn up a great deal when dealing

More information

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to

More information

Bayesian networks. Data Mining. Abraham Otero. Data Mining. Agenda. Bayes' theorem Naive Bayes Bayesian networks Bayesian networks (example)

Bayesian networks. Data Mining. Abraham Otero. Data Mining. Agenda. Bayes' theorem Naive Bayes Bayesian networks Bayesian networks (example) Bayesian networks 1/40 Agenda Introduction Bayes' theorem Bayesian networks Quick reference 2/40 1 Introduction Probabilistic methods: They are backed by a strong mathematical formalism. They can capture

More information

The rule for computing conditional property can be interpreted different. In Question 2, P B

The rule for computing conditional property can be interpreted different. In Question 2, P B Question 4: What is the product rule for probability? The rule for computing conditional property can be interpreted different. In Question 2, P A and B we defined the conditional probability PA B. If

More information

Probability for AI students who have seen some probability before but didn t understand any of it

Probability for AI students who have seen some probability before but didn t understand any of it Probability for AI students who have seen some probability before but didn t understand any of it I am inflicting these proofs on you for two reasons: 1. These kind of manipulations will need to be second

More information

reductio ad absurdum null hypothesis, alternate hypothesis

reductio ad absurdum null hypothesis, alternate hypothesis Chapter 10 s Using a Single Sample 10.1: Hypotheses & Test Procedures Basics: In statistics, a hypothesis is a statement about a population characteristic. s are based on an reductio ad absurdum form of

More information

Coin Flip Questions. Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT.

Coin Flip Questions. Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT. Coin Flip Questions Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT. 1 How many ways can you get exactly 1 head? 2 How many ways can you get exactly 2 heads?

More information

Module 7: Hypothesis Testing I Statistics (OA3102)

Module 7: Hypothesis Testing I Statistics (OA3102) Module 7: Hypothesis Testing I Statistics (OA3102) Professor Ron Fricker Naval Postgraduate School Monterey, California Reading assignment: WM&S chapter 10.1-10.5 Revision: 2-12 1 Goals for this Module

More information

E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

More information

People have thought about, and defined, probability in different ways. important to note the consequences of the definition:

People have thought about, and defined, probability in different ways. important to note the consequences of the definition: PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models

More information

+ Section 6.2 and 6.3

+ Section 6.2 and 6.3 Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities

More information

Lecture 9: Bayesian hypothesis testing

Lecture 9: Bayesian hypothesis testing Lecture 9: Bayesian hypothesis testing 5 November 27 In this lecture we ll learn about Bayesian hypothesis testing. 1 Introduction to Bayesian hypothesis testing Before we go into the details of Bayesian

More information

Economics 304 Fall 2014

Economics 304 Fall 2014 Economics 304 Fall 014 Country-Analysis Project Part 4: Economic Growth Analysis Introduction In this part of the project, you will analyze the economic growth performance of your country over the available

More information

Probabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm

Probabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm Probabilistic Graphical Models 10-708 Homework 1: Due January 29, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 1-3. You must complete all four problems to

More information

LECTURE 4. Conditional Probability and Bayes Theorem

LECTURE 4. Conditional Probability and Bayes Theorem LECTURE 4 Conditional Probability and Bayes Theorem 1 The conditional sample space Physical motivation for the formal mathematical theory 1. Roll a fair die once so the sample space S is given by S {1,

More information

Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence

More information

Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23

Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23 Probability Theory Uncertainty is key concept in machine learning. Probability provides consistent framework for the quantification and manipulation of uncertainty. Probability of an event is the fraction

More information

Knowledge-based systems and the need for learning

Knowledge-based systems and the need for learning Knowledge-based systems and the need for learning The implementation of a knowledge-based system can be quite difficult. Furthermore, the process of reasoning with that knowledge can be quite slow. This

More information

Introductory Probability. MATH 107: Finite Mathematics University of Louisville. March 5, 2014

Introductory Probability. MATH 107: Finite Mathematics University of Louisville. March 5, 2014 Introductory Probability MATH 07: Finite Mathematics University of Louisville March 5, 204 What is probability? Counting and probability 2 / 3 Probability in our daily lives We see chances, odds, and probabilities

More information

The Rate Laws. aa + bb cd + dd. The rate law for the above equation is. rate α [A] x [B] y

The Rate Laws. aa + bb cd + dd. The rate law for the above equation is. rate α [A] x [B] y The Rate Laws We are interested to know how the rate of reaction depends on the concentrations of reactants. When chemical reaction takes place, the concentrations of reactants and products change continuously

More information

STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012)

STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012) STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012) TA: Zhen (Alan) Zhang zhangz19@stt.msu.edu Office hour: (C500 WH) 1:45 2:45PM Tuesday (office tel.: 432-3342) Help-room: (A102 WH) 11:20AM-12:30PM,

More information

Hypothesis Testing & Data Analysis. Statistics. Descriptive Statistics. What is the difference between descriptive and inferential statistics?

Hypothesis Testing & Data Analysis. Statistics. Descriptive Statistics. What is the difference between descriptive and inferential statistics? 2 Hypothesis Testing & Data Analysis 5 What is the difference between descriptive and inferential statistics? Statistics 8 Tools to help us understand our data. Makes a complicated mess simple to understand.

More information

Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

More information

Proof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems.

Proof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems. Math 232 - Discrete Math 2.1 Direct Proofs and Counterexamples Notes Axiom: Proposition that is assumed to be true. Proof: A logical argument establishing the truth of the theorem given the truth of the

More information

Bayesian Networks. Read R&N Ch. 14.1-14.2. Next lecture: Read R&N 18.1-18.4

Bayesian Networks. Read R&N Ch. 14.1-14.2. Next lecture: Read R&N 18.1-18.4 Bayesian Networks Read R&N Ch. 14.1-14.2 Next lecture: Read R&N 18.1-18.4 You will be expected to know Basic concepts and vocabulary of Bayesian networks. Nodes represent random variables. Directed arcs

More information

Lecture 3: Summations and Analyzing Programs with Loops

Lecture 3: Summations and Analyzing Programs with Loops high school algebra If c is a constant (does not depend on the summation index i) then ca i = c a i and (a i + b i )= a i + b i There are some particularly important summations, which you should probably

More information

Chapter 7 Part 2. Hypothesis testing Power

Chapter 7 Part 2. Hypothesis testing Power Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship

More information

Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment,

Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment, Uncertainty Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment, E.g., loss of sensory information such as vision Incorrectness in

More information

Summary of Probability

Summary of Probability Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

More information

CS 441 Discrete Mathematics for CS Lecture 2. Propositional logic. CS 441 Discrete mathematics for CS. Course administration

CS 441 Discrete Mathematics for CS Lecture 2. Propositional logic. CS 441 Discrete mathematics for CS. Course administration CS 441 Discrete Mathematics for CS Lecture 2 Propositional logic Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Course administration Homework 1 First homework assignment is out today will be posted

More information

Markov Chains. Ben Langmead. Department of Computer Science

Markov Chains. Ben Langmead. Department of Computer Science Markov Chains Ben Langmead Department of Computer Science You are free to use these slides. If you do, please sign the guestbook (www.langmead-lab.org/teaching-materials), or email me (ben.langmead@gmail.com)

More information

Math 58. Rumbos Fall 2008 1. Solutions to Assignment #3

Math 58. Rumbos Fall 2008 1. Solutions to Assignment #3 Math 58. Rumbos Fall 2008 1 Solutions to Assignment #3 1. Use randomization to test the null hypothesis that there is no difference between calcium supplementation and a placebo for the experimental data

More information

Chapter 5 - Probability

Chapter 5 - Probability Chapter 5 - Probability 5.1 Basic Ideas An experiment is a process that, when performed, results in exactly one of many observations. These observations are called the outcomes of the experiment. The set

More information

Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to convert between odds and probability. 2. Be able to update prior odds to posterior odds

More information

Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

More information

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,

More information

EXAM. Exam #3. Math 1430, Spring 2002. April 21, 2001 ANSWERS

EXAM. Exam #3. Math 1430, Spring 2002. April 21, 2001 ANSWERS EXAM Exam #3 Math 1430, Spring 2002 April 21, 2001 ANSWERS i 60 pts. Problem 1. A city has two newspapers, the Gazette and the Journal. In a survey of 1, 200 residents, 500 read the Journal, 700 read the

More information

Philosophical argument

Philosophical argument Michael Lacewing Philosophical argument At the heart of philosophy is philosophical argument. Arguments are different from assertions. Assertions are simply stated; arguments always involve giving reasons.

More information

Joint Distribution and Correlation

Joint Distribution and Correlation Joint Distribution and Correlation Michael Ash Lecture 3 Reminder: Start working on the Problem Set Mean and Variance of Linear Functions of an R.V. Linear Function of an R.V. Y = a + bx What are the properties

More information

Midterm Examination 1 with Solutions - Math 574, Frank Thorne Thursday, February 9, 2012

Midterm Examination 1 with Solutions - Math 574, Frank Thorne Thursday, February 9, 2012 Midterm Examination 1 with Solutions - Math 574, Frank Thorne (thorne@math.sc.edu) Thursday, February 9, 2012 1. (3 points each) For each sentence below, say whether it is logically equivalent to the sentence

More information

Machine Learning. CS 188: Artificial Intelligence Naïve Bayes. Example: Digit Recognition. Other Classification Tasks

Machine Learning. CS 188: Artificial Intelligence Naïve Bayes. Example: Digit Recognition. Other Classification Tasks CS 188: Artificial Intelligence Naïve Bayes Machine Learning Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters

More information

The hardest logic puzzle ever becomes even tougher

The hardest logic puzzle ever becomes even tougher The hardest logic puzzle ever becomes even tougher Abstract The hardest logic puzzle ever presented by George Boolos became a target for philosophers and logicians who tried to modify it and make it even

More information

New Zealand Mathematical Olympiad Committee. Induction

New Zealand Mathematical Olympiad Committee. Induction New Zealand Mathematical Olympiad Committee Induction Chris Tuffley 1 Proof by dominoes Mathematical induction is a useful tool for proving statements like P is true for all natural numbers n, for example

More information

Math: Unit Two Vocabulary

Math: Unit Two Vocabulary (2.2) Algorithm- a set of step-by-step instructions for doing something, such as carrying out a computation or solving a problem. (2.8) Ballpark Estimate- A rough estimate 47 + 62 =? Using a ballpark estimate:

More information

Incorporating Evidence in Bayesian networks with the Select Operator

Incorporating Evidence in Bayesian networks with the Select Operator Incorporating Evidence in Bayesian networks with the Select Operator C.J. Butz and F. Fang Department of Computer Science, University of Regina Regina, Saskatchewan, Canada SAS 0A2 {butz, fang11fa}@cs.uregina.ca

More information

Bayesian network inference

Bayesian network inference Bayesian network inference Given: Query variables: X Evidence observed variables: E = e Unobserved variables: Y Goal: calculate some useful information about the query variables osterior Xe MA estimate

More information

Lecture 2: October 24

Lecture 2: October 24 Machine Learning: Foundations Fall Semester, 2 Lecture 2: October 24 Lecturer: Yishay Mansour Scribe: Shahar Yifrah, Roi Meron 2. Bayesian Inference - Overview This lecture is going to describe the basic

More information

Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80)

Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you

More information