# CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization

Save this PDF as:

Size: px
Start display at page:

Download "CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization"

## Transcription

1 Full Joint Probability Distributions CS 331: Artificial Intelligence Fundamentals of Probability II Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true Catch means the dentist s probe catches in my teeth Thanks to Andrew Moore for some course material 1 This cell means Toothache = true, Cavity = true, Catch = true = Full Joint Probability Distributions Joint Probability Distribution Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true From the full joint probability distribution, we can calculate any probability involving the three random variables in this world eg. Toothache = true OR Cavity = true = Toothache=true true, Cavity=false, Catch=false + Toothache=true, Cavity=false, Catch=true + Toothache=false, Cavity=true, Catch=false + Toothache=false, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false + Toothache=true, Cavity = true, Catch=true + = = 0.28 The probabilities in the last column sum to 1 3 Marginalization We can even calculate marginal probabilities (the probability distribution over a subset of the variables eg: Toothache=true true, Cavity=true true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false = = 0.12 Marginalization Or even: Cavity=true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false Toothache=false, Cavity=true, Catch=true + Toothache=false, Cavity=true, Catch=false = =

2 Marginalization The general marginalization rule for any sets of variables Y and Z: Y Y, z or z Y Y z z z z is over all possible combinations of values of Z (remember Z is a set Marginalization For continuous variables, marginalization involves taking the integral: Y Y, z dz 7 8 Normalization Cavity true Toothache true Cavity true, Toothache true Toothache true Cavity false Toothache true Cavity false, Toothache true Toothache true Note that t 1/Toothache=true h t remains constant in the two equations. Normalization In fact, 1/toothache can be viewed as a normalization constant for Cavity toothache, ensuring it adds up to 1 We will refer to normalization constants with the symbol Cavity toothache Cavity, toothache 10 Inference Suppose you get a query such as Cavity = true Toothache = true Toothache is called the evidence variable because we observe it. More generally, it s a set of variables. Cavity is called the query variable (we ll assume it s a single variable for now There are also unobserved (aka hidden variables like Catch 11 Inference We will write the query as X e This is a probability distribution hence the boldface X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables 12 2

3 Inference We will write the query as X e P ( X e X, e X, e, y Summation is over all possible combinations of values of the unobserved variables Y X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables y Inference P ( X e X, e X, e, y Computing X e involves going through all possible entries of the full joint probability distribution and adding up probabilities with X=x i, E=e, and Y=y Suppose you have a domain with n Boolean variables. What is the space and time complexity of computing X e? y 14 How do you avoid the exponential space and time complexity of inference? Use independence (aka factoring Suppose the full joint distribution now consists of four variables: Toothache = {true, false} Catch ={true {true, false} Cavity = {true, false} Weather = {sunny, rain, cloudy, snow} There are now 32 entries in the full joint distribution table Does the weather influence one s dental problems? Is Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy? In other words, is Weather independent of Toothache, Catch and Cavity? We say that variables X and Y are independent if any of the following hold: (note that they are all equivalent P ( X Y P ( X or Y X Y or X, Y X Y

4 Why is independence useful? Assume that Weather is independent of toothache, catch, cavity ie. Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Why is independence useful? Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity Now we can calculate: l Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity * toothache, catch, cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity 19 This table has 4 values This table has 8 values You now need to store 12 values to calculate Weather, Toothache, Catch, Cavity If Weather was not independent of Toothache, Catch, and Cavity then you would have needed 32 values 20 Another example: Suppose you have n coin flips and you want to calculate the joint distribution C 1,, C n If the coin flips are not independent, you need 2 n values in the table If the coin flips are independent, then n 1,..., Cn C i i1 C Each C i table has 2 entries and there are n of them for a total of 2n values is powerful! It required extra domain knowledge. A different kind of knowledge than numerical probabilities. It needed an understanding of causation Bayes Rule The product rule can be written in two ways: A, B = A BB A, B = B Bayes Rule More generally, the following is known as Bayes Rule: B A B B Note that these are distributions Sometimes, you can treat B as a normalization constant α You can combine the equations above to get: A B B B A B B

5 More General Forms of Bayes Rule Bayes Rule Example If A takes 2 values: A B If A takes k values: Av B i Av Av n A k1 i Av Av k i k 25 Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m (0.5( m s s 0.05 Bayes Rule Example Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m P m s s Note: Even though s m = 0.5, m s = (0.5( ( When is Bayes Rule Useful? Sometimes it s easier to get X Y than Y X. Information is typically available in the form effect cause rather than cause effect For example, symptom disease is easy to measure empirically but obtaining disease symptom is harder 28 How is Bayes Rule Used In machine learning, we use Bayes rule in the following way: Bayes Rule With More Than One Piece of Evidence Suppose you now have 2 evidence variables Toothache = true and Catch = catch (note that Cavity is uninstantiated below Likelihood of the data Prior probability D h h h D D Posterior probability h = hypothesis D = data 29 Cavity Toothache = true, Catch = true = α Toothache h h = true, Catch = true Cavity Cavity i In order to calculate Toothache = true, Catch = true Cavity, you need a table of 4 probability values. With N evidence variables, you need 2 N probability values. 30 5

6 Conditional Are Toothache and Catch independent? No if probe catches in the tooth, it likely has a cavity which causes the toothache. But given the presence or absence of the cavity, they are independent d (since they are directly caused dby the cavity but don t have a direct effect on each other Conditional independence: Toothache = true, Catch = catch Cavity = Toothache = true Cavity * Catch = true Cavity 31 Conditional General form: A, B C A C B C Or equivalently: A B, C A C B A, C B C and How to think about conditional independence: In A B, C = A C: if knowing C tells me everything about A, I don t gain anything by knowing B 32 Conditional 7 independent values in table (have to sum to 1 Toothache, Catch, Cavity = Toothache, (, Catch Cavity Cavity ( = Toothache Cavity Catch Cavity Cavity What You Should Know How to do inference in joint probability distributions How to use Bayes Rule Why independence and conditional independence is useful 2 independent values in table 2 independent values in table 1 independent value in table Conditional independence permits probabilistic systems to scale up!

### 13.3 Inference Using Full Joint Distribution

191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent

### Outline. Probability. Random Variables. Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule.

Probability 1 Probability Random Variables Outline Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule Inference Independence 2 Inference in Ghostbusters A ghost

### Artificial Intelligence

Artificial Intelligence ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #13 Uncertainty Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty

### / Informatica. Intelligent Systems. Agents dealing with uncertainty. Example: driving to the airport. Why logic fails

Intelligent Systems Prof. dr. Paul De Bra Technische Universiteit Eindhoven debra@win.tue.nl Agents dealing with uncertainty Uncertainty Probability Syntax and Semantics Inference Independence and Bayes'

### Outline. Uncertainty. Uncertainty. Making decisions under uncertainty. Probability. Methods for handling uncertainty

Outline Uncertainty Chapter 13 Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty Let action A t = leave for airport t minutes before flight Will A t get me

### Logic, Probability and Learning

Logic, Probability and Learning Luc De Raedt luc.deraedt@cs.kuleuven.be Overview Logic Learning Probabilistic Learning Probabilistic Logic Learning Closely following : Russell and Norvig, AI: a modern

### Basics of Probability

Basics of Probability 1 Sample spaces, events and probabilities Begin with a set Ω the sample space e.g., 6 possible rolls of a die. ω Ω is a sample point/possible world/atomic event A probability space

### 22c:145 Artificial Intelligence

22c:145 Artificial Intelligence Fall 2005 Uncertainty Cesare Tinelli The University of Iowa Copyright 2001-05 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material and may not be used

### Artificial Intelligence. Conditional probability. Inference by enumeration. Independence. Lesson 11 (From Russell & Norvig)

Artificial Intelligence Conditional probability Conditional or posterior probabilities e.g., cavity toothache) = 0.8 i.e., given that toothache is all I know tation for conditional distributions: Cavity

### Uncertainty. Chapter 13. Chapter 13 1

Uncertainty Chapter 13 Chapter 13 1 Attribution Modified from Stuart Russell s slides (Berkeley) Chapter 13 2 Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes Rule

### CS 188: Artificial Intelligence. Probability recap

CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability

### 15-381: Artificial Intelligence. Probabilistic Reasoning and Inference

5-38: Artificial Intelligence robabilistic Reasoning and Inference Advantages of probabilistic reasoning Appropriate for complex, uncertain, environments - Will it rain tomorrow? Applies naturally to many

### Attribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)

Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision

### Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

### Lecture 2: Introduction to belief (Bayesian) networks

Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (I-maps) January 7, 2008 1 COMP-526 Lecture 2 Recall from last time: Conditional

### CS 771 Artificial Intelligence. Reasoning under uncertainty

CS 771 Artificial Intelligence Reasoning under uncertainty Today Representing uncertainty is useful in knowledge bases Probability provides a coherent framework for uncertainty Review basic concepts in

### Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify

### Probability Review. ICPSR Applied Bayesian Modeling

Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How

### Marginal Independence and Conditional Independence

Marginal Independence and Conditional Independence Computer Science cpsc322, Lecture 26 (Textbook Chpt 6.1-2) March, 19, 2010 Recap with Example Marginalization Conditional Probability Chain Rule Lecture

### Reasoning Component Architecture

Architecture of a Spam Filter Application By Avi Pfeffer A spam filter consists of two components. In this article, based on my book Practical Probabilistic Programming, first describe the architecture

### Section 6.1 Joint Distribution Functions

Section 6.1 Joint Distribution Functions We often care about more than one random variable at a time. DEFINITION: For any two random variables X and Y the joint cumulative probability distribution function

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### Introduction to Game Theory IIIii. Payoffs: Probability and Expected Utility

Introduction to Game Theory IIIii Payoffs: Probability and Expected Utility Lecture Summary 1. Introduction 2. Probability Theory 3. Expected Values and Expected Utility. 1. Introduction We continue further

### Chapter 4. Probability and Probability Distributions

Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

### General Naïve Bayes. CS 188: Artificial Intelligence Fall Example: Spam Filtering. Example: OCR. Generalization and Overfitting

CS 88: Artificial Intelligence Fall 7 Lecture : Perceptrons //7 General Naïve Bayes A general naive Bayes model: C x E n parameters C parameters n x E x C parameters E E C E n Dan Klein UC Berkeley We

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Querying Joint Probability Distributions

Querying Joint Probability Distributions Sargur Srihari srihari@cedar.buffalo.edu 1 Queries of Interest Probabilistic Graphical Models (BNs and MNs) represent joint probability distributions over multiple

### Part III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing

CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and

### 1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error.

1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error. 8.5 Goodness of Fit Test Suppose we want to make an inference about

### Statistical Natural Language Processing

Statistical Natural Language Processing Prasad Tadepalli CS430 lecture Natural Language Processing Some subproblems are partially solved Spelling correction, grammar checking Information retrieval with

### Machine Learning

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 8, 2011 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

### 4.4 Equations of the Form ax + b = cx + d

4.4 Equations of the Form ax + b = cx + d We continue our study of equations in which the variable appears on both sides of the equation. Suppose we are given the equation: 3x + 4 = 5x! 6 Our first step

### 1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016

and P (B X). In order to do find those conditional probabilities, we ll use Bayes formula. We can easily compute the reverse probabilities A short introduction to Bayesian statistics, part I Math 18, Mathematical

### Graphical Models. CS 343: Artificial Intelligence Bayesian Networks. Raymond J. Mooney. Conditional Probability Tables.

Graphical Models CS 343: Artificial Intelligence Bayesian Networks Raymond J. Mooney University of Texas at Austin 1 If no assumption of independence is made, then an exponential number of parameters must

### MAT 1000. Mathematics in Today's World

MAT 1000 Mathematics in Today's World We talked about Cryptography Last Time We will talk about probability. Today There are four rules that govern probabilities. One good way to analyze simple probabilities

### 5.7 Literal Equations

5.7 Literal Equations Now that we have learned to solve a variety of different equations (linear equations in chapter 2, polynomial equations in chapter 4, and rational equations in the last section) we

### The Foundations: Logic and Proofs. Chapter 1, Part III: Proofs

The Foundations: Logic and Proofs Chapter 1, Part III: Proofs Rules of Inference Section 1.6 Section Summary Valid Arguments Inference Rules for Propositional Logic Using Rules of Inference to Build Arguments

### Bayesian Networks. Mausam (Slides by UW-AI faculty)

Bayesian Networks Mausam (Slides by UW-AI faculty) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation & inference BNs provide

Section 5.1 Climbing an Infinite Ladder Suppose we have an infinite ladder and the following capabilities: 1. We can reach the first rung of the ladder. 2. If we can reach a particular rung of the ladder,

### 1. Chi-Squared Tests

1. Chi-Squared Tests We'll now look at how to test statistical hypotheses concerning nominal data, and specifically when nominal data are summarized as tables of frequencies. The tests we will considered

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1

Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields

### CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam)

CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam) Problem Points Possible 1. Perceptrons and SVMs 20 2. Probabilistic Models 20 3. Markov Models

### Probability, Conditional Independence

Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms

### FOR INTERNAL SCRUTINY (date of this version: 13/3/2009) UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS

UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS COGNITIVE MODELLING (LEVEL 10) COGNITIVE MODELLING (LEVEL 11) Friday 1 April 2005 00:00 to 00:00 Year 4 Courses Convener:

### Bayes Rule. How is this rule derived? Using Bayes rule for probabilistic inference: Evidence Cause) Evidence)

Bayes Rule P ( A B = B A A B ( Rev. Thomas Bayes (1702-1761 How is this rule derived? Using Bayes rule for probabilistic inference: P ( Cause Evidence = Evidence Cause Cause Evidence Cause Evidence: diagnostic

### Bayesian probability theory

Bayesian probability theory Bruno A. Olshausen arch 1, 2004 Abstract Bayesian probability theory provides a mathematical framework for peforming inference, or reasoning, using probability. The foundations

### Statistical Inference

Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this

### Chi Square for Contingency Tables

2 x 2 Case Chi Square for Contingency Tables A test for p 1 = p 2 We have learned a confidence interval for p 1 p 2, the difference in the population proportions. We want a hypothesis testing procedure

### CS 341 Homework 9 Languages That Are and Are Not Regular

CS 341 Homework 9 Languages That Are and Are Not Regular 1. Show that the following are not regular. (a) L = {ww R : w {a, b}*} (b) L = {ww : w {a, b}*} (c) L = {ww' : w {a, b}*}, where w' stands for w

### Bayesian Networks Chapter 14. Mausam (Slides by UW-AI faculty & David Page)

Bayesian Networks Chapter 14 Mausam (Slides by UW-AI faculty & David Page) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation

### Artificial Intelligence II

Artificial Intelligence II Some supplementary notes on probability Sean B Holden, 2010-14 1 Introduction These notes provide a reminder of some simple manipulations that turn up a great deal when dealing

### Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to

### Bayesian networks. Data Mining. Abraham Otero. Data Mining. Agenda. Bayes' theorem Naive Bayes Bayesian networks Bayesian networks (example)

Bayesian networks 1/40 Agenda Introduction Bayes' theorem Bayesian networks Quick reference 2/40 1 Introduction Probabilistic methods: They are backed by a strong mathematical formalism. They can capture

### The rule for computing conditional property can be interpreted different. In Question 2, P B

Question 4: What is the product rule for probability? The rule for computing conditional property can be interpreted different. In Question 2, P A and B we defined the conditional probability PA B. If

### Probability for AI students who have seen some probability before but didn t understand any of it

Probability for AI students who have seen some probability before but didn t understand any of it I am inflicting these proofs on you for two reasons: 1. These kind of manipulations will need to be second

### reductio ad absurdum null hypothesis, alternate hypothesis

Chapter 10 s Using a Single Sample 10.1: Hypotheses & Test Procedures Basics: In statistics, a hypothesis is a statement about a population characteristic. s are based on an reductio ad absurdum form of

### Coin Flip Questions. Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT.

Coin Flip Questions Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT. 1 How many ways can you get exactly 1 head? 2 How many ways can you get exactly 2 heads?

### Module 7: Hypothesis Testing I Statistics (OA3102)

Module 7: Hypothesis Testing I Statistics (OA3102) Professor Ron Fricker Naval Postgraduate School Monterey, California Reading assignment: WM&S chapter 10.1-10.5 Revision: 2-12 1 Goals for this Module

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### People have thought about, and defined, probability in different ways. important to note the consequences of the definition:

PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models

### + Section 6.2 and 6.3

Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities

### Lecture 9: Bayesian hypothesis testing

Lecture 9: Bayesian hypothesis testing 5 November 27 In this lecture we ll learn about Bayesian hypothesis testing. 1 Introduction to Bayesian hypothesis testing Before we go into the details of Bayesian

### Economics 304 Fall 2014

Economics 304 Fall 014 Country-Analysis Project Part 4: Economic Growth Analysis Introduction In this part of the project, you will analyze the economic growth performance of your country over the available

### Probabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm

Probabilistic Graphical Models 10-708 Homework 1: Due January 29, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 1-3. You must complete all four problems to

### LECTURE 4. Conditional Probability and Bayes Theorem

LECTURE 4 Conditional Probability and Bayes Theorem 1 The conditional sample space Physical motivation for the formal mathematical theory 1. Roll a fair die once so the sample space S is given by S {1,

### Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence

### Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23

Probability Theory Uncertainty is key concept in machine learning. Probability provides consistent framework for the quantification and manipulation of uncertainty. Probability of an event is the fraction

### Knowledge-based systems and the need for learning

Knowledge-based systems and the need for learning The implementation of a knowledge-based system can be quite difficult. Furthermore, the process of reasoning with that knowledge can be quite slow. This

### Introductory Probability. MATH 107: Finite Mathematics University of Louisville. March 5, 2014

Introductory Probability MATH 07: Finite Mathematics University of Louisville March 5, 204 What is probability? Counting and probability 2 / 3 Probability in our daily lives We see chances, odds, and probabilities

### The Rate Laws. aa + bb cd + dd. The rate law for the above equation is. rate α [A] x [B] y

The Rate Laws We are interested to know how the rate of reaction depends on the concentrations of reactants. When chemical reaction takes place, the concentrations of reactants and products change continuously

### STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012)

STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012) TA: Zhen (Alan) Zhang zhangz19@stt.msu.edu Office hour: (C500 WH) 1:45 2:45PM Tuesday (office tel.: 432-3342) Help-room: (A102 WH) 11:20AM-12:30PM,

### Hypothesis Testing & Data Analysis. Statistics. Descriptive Statistics. What is the difference between descriptive and inferential statistics?

2 Hypothesis Testing & Data Analysis 5 What is the difference between descriptive and inferential statistics? Statistics 8 Tools to help us understand our data. Makes a complicated mess simple to understand.

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### Proof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems.

Math 232 - Discrete Math 2.1 Direct Proofs and Counterexamples Notes Axiom: Proposition that is assumed to be true. Proof: A logical argument establishing the truth of the theorem given the truth of the

### Bayesian Networks. Read R&N Ch. 14.1-14.2. Next lecture: Read R&N 18.1-18.4

Bayesian Networks Read R&N Ch. 14.1-14.2 Next lecture: Read R&N 18.1-18.4 You will be expected to know Basic concepts and vocabulary of Bayesian networks. Nodes represent random variables. Directed arcs

### Lecture 3: Summations and Analyzing Programs with Loops

high school algebra If c is a constant (does not depend on the summation index i) then ca i = c a i and (a i + b i )= a i + b i There are some particularly important summations, which you should probably

### Chapter 7 Part 2. Hypothesis testing Power

Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship

### Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment,

Uncertainty Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment, E.g., loss of sensory information such as vision Incorrectness in

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### CS 441 Discrete Mathematics for CS Lecture 2. Propositional logic. CS 441 Discrete mathematics for CS. Course administration

CS 441 Discrete Mathematics for CS Lecture 2 Propositional logic Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Course administration Homework 1 First homework assignment is out today will be posted

### Markov Chains. Ben Langmead. Department of Computer Science

Markov Chains Ben Langmead Department of Computer Science You are free to use these slides. If you do, please sign the guestbook (www.langmead-lab.org/teaching-materials), or email me (ben.langmead@gmail.com)

### Math 58. Rumbos Fall 2008 1. Solutions to Assignment #3

Math 58. Rumbos Fall 2008 1 Solutions to Assignment #3 1. Use randomization to test the null hypothesis that there is no difference between calcium supplementation and a placebo for the experimental data

### Chapter 5 - Probability

Chapter 5 - Probability 5.1 Basic Ideas An experiment is a process that, when performed, results in exactly one of many observations. These observations are called the outcomes of the experiment. The set

### Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to convert between odds and probability. 2. Be able to update prior odds to posterior odds

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,

### EXAM. Exam #3. Math 1430, Spring 2002. April 21, 2001 ANSWERS

EXAM Exam #3 Math 1430, Spring 2002 April 21, 2001 ANSWERS i 60 pts. Problem 1. A city has two newspapers, the Gazette and the Journal. In a survey of 1, 200 residents, 500 read the Journal, 700 read the

### Philosophical argument

Michael Lacewing Philosophical argument At the heart of philosophy is philosophical argument. Arguments are different from assertions. Assertions are simply stated; arguments always involve giving reasons.

### Joint Distribution and Correlation

Joint Distribution and Correlation Michael Ash Lecture 3 Reminder: Start working on the Problem Set Mean and Variance of Linear Functions of an R.V. Linear Function of an R.V. Y = a + bx What are the properties

### Midterm Examination 1 with Solutions - Math 574, Frank Thorne Thursday, February 9, 2012

Midterm Examination 1 with Solutions - Math 574, Frank Thorne (thorne@math.sc.edu) Thursday, February 9, 2012 1. (3 points each) For each sentence below, say whether it is logically equivalent to the sentence

### Machine Learning. CS 188: Artificial Intelligence Naïve Bayes. Example: Digit Recognition. Other Classification Tasks

CS 188: Artificial Intelligence Naïve Bayes Machine Learning Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters

### The hardest logic puzzle ever becomes even tougher

The hardest logic puzzle ever becomes even tougher Abstract The hardest logic puzzle ever presented by George Boolos became a target for philosophers and logicians who tried to modify it and make it even

### New Zealand Mathematical Olympiad Committee. Induction

New Zealand Mathematical Olympiad Committee Induction Chris Tuffley 1 Proof by dominoes Mathematical induction is a useful tool for proving statements like P is true for all natural numbers n, for example

### Math: Unit Two Vocabulary

(2.2) Algorithm- a set of step-by-step instructions for doing something, such as carrying out a computation or solving a problem. (2.8) Ballpark Estimate- A rough estimate 47 + 62 =? Using a ballpark estimate:

### Incorporating Evidence in Bayesian networks with the Select Operator

Incorporating Evidence in Bayesian networks with the Select Operator C.J. Butz and F. Fang Department of Computer Science, University of Regina Regina, Saskatchewan, Canada SAS 0A2 {butz, fang11fa}@cs.uregina.ca

### Bayesian network inference

Bayesian network inference Given: Query variables: X Evidence observed variables: E = e Unobserved variables: Y Goal: calculate some useful information about the query variables osterior Xe MA estimate