CS 331: Artificial Intelligence Fundamentals of Probability II. Joint Probability Distribution. Marginalization. Marginalization


 Adela Hensley
 1 years ago
 Views:
Transcription
1 Full Joint Probability Distributions CS 331: Artificial Intelligence Fundamentals of Probability II Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true Catch means the dentist s probe catches in my teeth Thanks to Andrew Moore for some course material 1 This cell means Toothache = true, Cavity = true, Catch = true = Full Joint Probability Distributions Joint Probability Distribution Toothache Cavity Catch Toothache, Cavity, Catch false false false false false true false true false false true true true false false true false true true true false true true true From the full joint probability distribution, we can calculate any probability involving the three random variables in this world eg. Toothache = true OR Cavity = true = Toothache=true true, Cavity=false, Catch=false + Toothache=true, Cavity=false, Catch=true + Toothache=false, Cavity=true, Catch=false + Toothache=false, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false + Toothache=true, Cavity = true, Catch=true + = = 0.28 The probabilities in the last column sum to 1 3 Marginalization We can even calculate marginal probabilities (the probability distribution over a subset of the variables eg: Toothache=true true, Cavity=true true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false = = 0.12 Marginalization Or even: Cavity=true = Toothache=true, Cavity=true, Catch=true + Toothache=true, Cavity=true, Catch=false Toothache=false, Cavity=true, Catch=true + Toothache=false, Cavity=true, Catch=false = =
2 Marginalization The general marginalization rule for any sets of variables Y and Z: Y Y, z or z Y Y z z z z is over all possible combinations of values of Z (remember Z is a set Marginalization For continuous variables, marginalization involves taking the integral: Y Y, z dz 7 8 Normalization Cavity true Toothache true Cavity true, Toothache true Toothache true Cavity false Toothache true Cavity false, Toothache true Toothache true Note that t 1/Toothache=true h t remains constant in the two equations. Normalization In fact, 1/toothache can be viewed as a normalization constant for Cavity toothache, ensuring it adds up to 1 We will refer to normalization constants with the symbol Cavity toothache Cavity, toothache 10 Inference Suppose you get a query such as Cavity = true Toothache = true Toothache is called the evidence variable because we observe it. More generally, it s a set of variables. Cavity is called the query variable (we ll assume it s a single variable for now There are also unobserved (aka hidden variables like Catch 11 Inference We will write the query as X e This is a probability distribution hence the boldface X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables 12 2
3 Inference We will write the query as X e P ( X e X, e X, e, y Summation is over all possible combinations of values of the unobserved variables Y X = Query variable (a single variable for now E = Set of evidence variables e = the set of observed values for the evidence variables Y = Unobserved variables y Inference P ( X e X, e X, e, y Computing X e involves going through all possible entries of the full joint probability distribution and adding up probabilities with X=x i, E=e, and Y=y Suppose you have a domain with n Boolean variables. What is the space and time complexity of computing X e? y 14 How do you avoid the exponential space and time complexity of inference? Use independence (aka factoring Suppose the full joint distribution now consists of four variables: Toothache = {true, false} Catch ={true {true, false} Cavity = {true, false} Weather = {sunny, rain, cloudy, snow} There are now 32 entries in the full joint distribution table Does the weather influence one s dental problems? Is Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy? In other words, is Weather independent of Toothache, Catch and Cavity? We say that variables X and Y are independent if any of the following hold: (note that they are all equivalent P ( X Y P ( X or Y X Y or X, Y X Y
4 Why is independence useful? Assume that Weather is independent of toothache, catch, cavity ie. Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Why is independence useful? Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity Now we can calculate: l Weather=cloudy, Toothache = toothache, Catch = catch, Cavity = cavity = Weather=cloudy Toothache = toothache, Catch = catch, Cavity = cavity * toothache, catch, cavity = Weather=cloudy * Toothache = toothache, Catch = catch, Cavity = cavity 19 This table has 4 values This table has 8 values You now need to store 12 values to calculate Weather, Toothache, Catch, Cavity If Weather was not independent of Toothache, Catch, and Cavity then you would have needed 32 values 20 Another example: Suppose you have n coin flips and you want to calculate the joint distribution C 1,, C n If the coin flips are not independent, you need 2 n values in the table If the coin flips are independent, then n 1,..., Cn C i i1 C Each C i table has 2 entries and there are n of them for a total of 2n values is powerful! It required extra domain knowledge. A different kind of knowledge than numerical probabilities. It needed an understanding of causation Bayes Rule The product rule can be written in two ways: A, B = A BB A, B = B Bayes Rule More generally, the following is known as Bayes Rule: B A B B Note that these are distributions Sometimes, you can treat B as a normalization constant α You can combine the equations above to get: A B B B A B B
5 More General Forms of Bayes Rule Bayes Rule Example If A takes 2 values: A B If A takes k values: Av B i Av Av n A k1 i Av Av k i k 25 Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m (0.5( m s s 0.05 Bayes Rule Example Meningitis causes stiff necks with probability 0.5. The prior probability of having meningitis is The prior probability of having a stiff neck is What is the probability of having meningitis given that you have a stiff neck? Let m = patient has meningitis Let s = patient has stiff neck s m = 0.5 m = s = 0.05 s m m P m s s Note: Even though s m = 0.5, m s = (0.5( ( When is Bayes Rule Useful? Sometimes it s easier to get X Y than Y X. Information is typically available in the form effect cause rather than cause effect For example, symptom disease is easy to measure empirically but obtaining disease symptom is harder 28 How is Bayes Rule Used In machine learning, we use Bayes rule in the following way: Bayes Rule With More Than One Piece of Evidence Suppose you now have 2 evidence variables Toothache = true and Catch = catch (note that Cavity is uninstantiated below Likelihood of the data Prior probability D h h h D D Posterior probability h = hypothesis D = data 29 Cavity Toothache = true, Catch = true = α Toothache h h = true, Catch = true Cavity Cavity i In order to calculate Toothache = true, Catch = true Cavity, you need a table of 4 probability values. With N evidence variables, you need 2 N probability values. 30 5
6 Conditional Are Toothache and Catch independent? No if probe catches in the tooth, it likely has a cavity which causes the toothache. But given the presence or absence of the cavity, they are independent d (since they are directly caused dby the cavity but don t have a direct effect on each other Conditional independence: Toothache = true, Catch = catch Cavity = Toothache = true Cavity * Catch = true Cavity 31 Conditional General form: A, B C A C B C Or equivalently: A B, C A C B A, C B C and How to think about conditional independence: In A B, C = A C: if knowing C tells me everything about A, I don t gain anything by knowing B 32 Conditional 7 independent values in table (have to sum to 1 Toothache, Catch, Cavity = Toothache, (, Catch Cavity Cavity ( = Toothache Cavity Catch Cavity Cavity What You Should Know How to do inference in joint probability distributions How to use Bayes Rule Why independence and conditional independence is useful 2 independent values in table 2 independent values in table 1 independent value in table Conditional independence permits probabilistic systems to scale up!
13.3 Inference Using Full Joint Distribution
191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent
More informationOutline. Probability. Random Variables. Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule.
Probability 1 Probability Random Variables Outline Joint and Marginal Distributions Conditional Distribution Product Rule, Chain Rule, Bayes Rule Inference Independence 2 Inference in Ghostbusters A ghost
More informationArtificial Intelligence
Artificial Intelligence ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #13 Uncertainty Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty
More information/ Informatica. Intelligent Systems. Agents dealing with uncertainty. Example: driving to the airport. Why logic fails
Intelligent Systems Prof. dr. Paul De Bra Technische Universiteit Eindhoven debra@win.tue.nl Agents dealing with uncertainty Uncertainty Probability Syntax and Semantics Inference Independence and Bayes'
More informationOutline. Uncertainty. Uncertainty. Making decisions under uncertainty. Probability. Methods for handling uncertainty
Outline Uncertainty Chapter 13 Uncertainty Probability Syntax and Semantics Inference Independence and Bayes' Rule Uncertainty Let action A t = leave for airport t minutes before flight Will A t get me
More informationLogic, Probability and Learning
Logic, Probability and Learning Luc De Raedt luc.deraedt@cs.kuleuven.be Overview Logic Learning Probabilistic Learning Probabilistic Logic Learning Closely following : Russell and Norvig, AI: a modern
More informationBasics of Probability
Basics of Probability 1 Sample spaces, events and probabilities Begin with a set Ω the sample space e.g., 6 possible rolls of a die. ω Ω is a sample point/possible world/atomic event A probability space
More information22c:145 Artificial Intelligence
22c:145 Artificial Intelligence Fall 2005 Uncertainty Cesare Tinelli The University of Iowa Copyright 200105 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material and may not be used
More informationArtificial Intelligence. Conditional probability. Inference by enumeration. Independence. Lesson 11 (From Russell & Norvig)
Artificial Intelligence Conditional probability Conditional or posterior probabilities e.g., cavity toothache) = 0.8 i.e., given that toothache is all I know tation for conditional distributions: Cavity
More informationUncertainty. Chapter 13. Chapter 13 1
Uncertainty Chapter 13 Chapter 13 1 Attribution Modified from Stuart Russell s slides (Berkeley) Chapter 13 2 Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes Rule
More informationCS 188: Artificial Intelligence. Probability recap
CS 188: Artificial Intelligence Bayes Nets Representation and Independence Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Conditional probability
More information15381: Artificial Intelligence. Probabilistic Reasoning and Inference
538: Artificial Intelligence robabilistic Reasoning and Inference Advantages of probabilistic reasoning Appropriate for complex, uncertain, environments  Will it rain tomorrow? Applies naturally to many
More informationAttribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)
Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision
More informationArtificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =
Artificial Intelligence 15381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information
More informationLecture 2: Introduction to belief (Bayesian) networks
Lecture 2: Introduction to belief (Bayesian) networks Conditional independence What is a belief network? Independence maps (Imaps) January 7, 2008 1 COMP526 Lecture 2 Recall from last time: Conditional
More informationCS 771 Artificial Intelligence. Reasoning under uncertainty
CS 771 Artificial Intelligence Reasoning under uncertainty Today Representing uncertainty is useful in knowledge bases Probability provides a coherent framework for uncertainty Review basic concepts in
More informationBayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify
More informationProbability Review. ICPSR Applied Bayesian Modeling
Probability Review ICPSR Applied Bayesian Modeling Random Variables Flip a coin. Will it be heads or tails? The outcome of a single event is random, or unpredictable What if we flip a coin 10 times? How
More informationMarginal Independence and Conditional Independence
Marginal Independence and Conditional Independence Computer Science cpsc322, Lecture 26 (Textbook Chpt 6.12) March, 19, 2010 Recap with Example Marginalization Conditional Probability Chain Rule Lecture
More informationReasoning Component Architecture
Architecture of a Spam Filter Application By Avi Pfeffer A spam filter consists of two components. In this article, based on my book Practical Probabilistic Programming, first describe the architecture
More informationSection 6.1 Joint Distribution Functions
Section 6.1 Joint Distribution Functions We often care about more than one random variable at a time. DEFINITION: For any two random variables X and Y the joint cumulative probability distribution function
More informationDiscrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According
More informationIntroduction to Game Theory IIIii. Payoffs: Probability and Expected Utility
Introduction to Game Theory IIIii Payoffs: Probability and Expected Utility Lecture Summary 1. Introduction 2. Probability Theory 3. Expected Values and Expected Utility. 1. Introduction We continue further
More informationChapter 4. Probability and Probability Distributions
Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the
More informationGeneral Naïve Bayes. CS 188: Artificial Intelligence Fall Example: Spam Filtering. Example: OCR. Generalization and Overfitting
CS 88: Artificial Intelligence Fall 7 Lecture : Perceptrons //7 General Naïve Bayes A general naive Bayes model: C x E n parameters C parameters n x E x C parameters E E C E n Dan Klein UC Berkeley We
More informationThe Basics of Graphical Models
The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures
More informationQuerying Joint Probability Distributions
Querying Joint Probability Distributions Sargur Srihari srihari@cedar.buffalo.edu 1 Queries of Interest Probabilistic Graphical Models (BNs and MNs) represent joint probability distributions over multiple
More informationPart III: Machine Learning. CS 188: Artificial Intelligence. Machine Learning This Set of Slides. Parameter Estimation. Estimation: Smoothing
CS 188: Artificial Intelligence Lecture 20: Dynamic Bayes Nets, Naïve Bayes Pieter Abbeel UC Berkeley Slides adapted from Dan Klein. Part III: Machine Learning Up until now: how to reason in a model and
More information1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error.
1. Rejecting a true null hypothesis is classified as a error. 2. Failing to reject a false null hypothesis is classified as a error. 8.5 Goodness of Fit Test Suppose we want to make an inference about
More informationStatistical Natural Language Processing
Statistical Natural Language Processing Prasad Tadepalli CS430 lecture Natural Language Processing Some subproblems are partially solved Spelling correction, grammar checking Information retrieval with
More informationMachine Learning
Machine Learning 10701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 8, 2011 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies
More information4.4 Equations of the Form ax + b = cx + d
4.4 Equations of the Form ax + b = cx + d We continue our study of equations in which the variable appears on both sides of the equation. Suppose we are given the equation: 3x + 4 = 5x! 6 Our first step
More information1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016
and P (B X). In order to do find those conditional probabilities, we ll use Bayes formula. We can easily compute the reverse probabilities A short introduction to Bayesian statistics, part I Math 18, Mathematical
More informationGraphical Models. CS 343: Artificial Intelligence Bayesian Networks. Raymond J. Mooney. Conditional Probability Tables.
Graphical Models CS 343: Artificial Intelligence Bayesian Networks Raymond J. Mooney University of Texas at Austin 1 If no assumption of independence is made, then an exponential number of parameters must
More informationMAT 1000. Mathematics in Today's World
MAT 1000 Mathematics in Today's World We talked about Cryptography Last Time We will talk about probability. Today There are four rules that govern probabilities. One good way to analyze simple probabilities
More information5.7 Literal Equations
5.7 Literal Equations Now that we have learned to solve a variety of different equations (linear equations in chapter 2, polynomial equations in chapter 4, and rational equations in the last section) we
More informationThe Foundations: Logic and Proofs. Chapter 1, Part III: Proofs
The Foundations: Logic and Proofs Chapter 1, Part III: Proofs Rules of Inference Section 1.6 Section Summary Valid Arguments Inference Rules for Propositional Logic Using Rules of Inference to Build Arguments
More informationBayesian Networks. Mausam (Slides by UWAI faculty)
Bayesian Networks Mausam (Slides by UWAI faculty) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation & inference BNs provide
More informationClimbing an Infinite Ladder
Section 5.1 Climbing an Infinite Ladder Suppose we have an infinite ladder and the following capabilities: 1. We can reach the first rung of the ladder. 2. If we can reach a particular rung of the ladder,
More information1. ChiSquared Tests
1. ChiSquared Tests We'll now look at how to test statistical hypotheses concerning nominal data, and specifically when nominal data are summarized as tables of frequencies. The tests we will considered
More information7 Probability. Copyright Cengage Learning. All rights reserved.
7 Probability Copyright Cengage Learning. All rights reserved. 7.1 Sample Spaces and Events Copyright Cengage Learning. All rights reserved. Sample Spaces 3 Sample Spaces At the beginning of a football
More informationA crash course in probability and Naïve Bayes classification
Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More informationCIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam)
CIS 391 Introduction to Artificial Intelligence Practice Midterm II With Solutions (From old CIS 521 exam) Problem Points Possible 1. Perceptrons and SVMs 20 2. Probabilistic Models 20 3. Markov Models
More informationProbability, Conditional Independence
Probability, Conditional Independence June 19, 2012 Probability, Conditional Independence Probability Sample space Ω of events Each event ω Ω has an associated measure Probability of the event P(ω) Axioms
More informationFOR INTERNAL SCRUTINY (date of this version: 13/3/2009) UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS COGNITIVE MODELLING (LEVEL 10) COGNITIVE MODELLING (LEVEL 11) Friday 1 April 2005 00:00 to 00:00 Year 4 Courses Convener:
More informationBayes Rule. How is this rule derived? Using Bayes rule for probabilistic inference: Evidence Cause) Evidence)
Bayes Rule P ( A B = B A A B ( Rev. Thomas Bayes (17021761 How is this rule derived? Using Bayes rule for probabilistic inference: P ( Cause Evidence = Evidence Cause Cause Evidence Cause Evidence: diagnostic
More informationBayesian probability theory
Bayesian probability theory Bruno A. Olshausen arch 1, 2004 Abstract Bayesian probability theory provides a mathematical framework for peforming inference, or reasoning, using probability. The foundations
More informationStatistical Inference
Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this
More informationChi Square for Contingency Tables
2 x 2 Case Chi Square for Contingency Tables A test for p 1 = p 2 We have learned a confidence interval for p 1 p 2, the difference in the population proportions. We want a hypothesis testing procedure
More informationCS 341 Homework 9 Languages That Are and Are Not Regular
CS 341 Homework 9 Languages That Are and Are Not Regular 1. Show that the following are not regular. (a) L = {ww R : w {a, b}*} (b) L = {ww : w {a, b}*} (c) L = {ww' : w {a, b}*}, where w' stands for w
More informationBayesian Networks Chapter 14. Mausam (Slides by UWAI faculty & David Page)
Bayesian Networks Chapter 14 Mausam (Slides by UWAI faculty & David Page) Bayes Nets In general, joint distribution P over set of variables (X 1 x... x X n ) requires exponential space for representation
More informationArtificial Intelligence II
Artificial Intelligence II Some supplementary notes on probability Sean B Holden, 201014 1 Introduction These notes provide a reminder of some simple manipulations that turn up a great deal when dealing
More informationProbability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to
More informationBayesian networks. Data Mining. Abraham Otero. Data Mining. Agenda. Bayes' theorem Naive Bayes Bayesian networks Bayesian networks (example)
Bayesian networks 1/40 Agenda Introduction Bayes' theorem Bayesian networks Quick reference 2/40 1 Introduction Probabilistic methods: They are backed by a strong mathematical formalism. They can capture
More informationThe rule for computing conditional property can be interpreted different. In Question 2, P B
Question 4: What is the product rule for probability? The rule for computing conditional property can be interpreted different. In Question 2, P A and B we defined the conditional probability PA B. If
More informationProbability for AI students who have seen some probability before but didn t understand any of it
Probability for AI students who have seen some probability before but didn t understand any of it I am inflicting these proofs on you for two reasons: 1. These kind of manipulations will need to be second
More informationreductio ad absurdum null hypothesis, alternate hypothesis
Chapter 10 s Using a Single Sample 10.1: Hypotheses & Test Procedures Basics: In statistics, a hypothesis is a statement about a population characteristic. s are based on an reductio ad absurdum form of
More informationCoin Flip Questions. Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT.
Coin Flip Questions Suppose you flip a coin five times and write down the sequence of results, like HHHHH or HTTHT. 1 How many ways can you get exactly 1 head? 2 How many ways can you get exactly 2 heads?
More informationModule 7: Hypothesis Testing I Statistics (OA3102)
Module 7: Hypothesis Testing I Statistics (OA3102) Professor Ron Fricker Naval Postgraduate School Monterey, California Reading assignment: WM&S chapter 10.110.5 Revision: 212 1 Goals for this Module
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationPeople have thought about, and defined, probability in different ways. important to note the consequences of the definition:
PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models
More information+ Section 6.2 and 6.3
Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities
More informationLecture 9: Bayesian hypothesis testing
Lecture 9: Bayesian hypothesis testing 5 November 27 In this lecture we ll learn about Bayesian hypothesis testing. 1 Introduction to Bayesian hypothesis testing Before we go into the details of Bayesian
More informationEconomics 304 Fall 2014
Economics 304 Fall 014 CountryAnalysis Project Part 4: Economic Growth Analysis Introduction In this part of the project, you will analyze the economic growth performance of your country over the available
More informationProbabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm
Probabilistic Graphical Models 10708 Homework 1: Due January 29, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 13. You must complete all four problems to
More informationLECTURE 4. Conditional Probability and Bayes Theorem
LECTURE 4 Conditional Probability and Bayes Theorem 1 The conditional sample space Physical motivation for the formal mathematical theory 1. Roll a fair die once so the sample space S is given by S {1,
More informationConditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence
More informationProbability Theory. Elementary rules of probability Sum rule. Product rule. p. 23
Probability Theory Uncertainty is key concept in machine learning. Probability provides consistent framework for the quantification and manipulation of uncertainty. Probability of an event is the fraction
More informationKnowledgebased systems and the need for learning
Knowledgebased systems and the need for learning The implementation of a knowledgebased system can be quite difficult. Furthermore, the process of reasoning with that knowledge can be quite slow. This
More informationIntroductory Probability. MATH 107: Finite Mathematics University of Louisville. March 5, 2014
Introductory Probability MATH 07: Finite Mathematics University of Louisville March 5, 204 What is probability? Counting and probability 2 / 3 Probability in our daily lives We see chances, odds, and probabilities
More informationThe Rate Laws. aa + bb cd + dd. The rate law for the above equation is. rate α [A] x [B] y
The Rate Laws We are interested to know how the rate of reaction depends on the concentrations of reactants. When chemical reaction takes place, the concentrations of reactants and products change continuously
More informationSTT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012)
STT 200 LECTURE 1, SECTION 2,4 RECITATION 7 (10/16/2012) TA: Zhen (Alan) Zhang zhangz19@stt.msu.edu Office hour: (C500 WH) 1:45 2:45PM Tuesday (office tel.: 4323342) Helproom: (A102 WH) 11:20AM12:30PM,
More informationHypothesis Testing & Data Analysis. Statistics. Descriptive Statistics. What is the difference between descriptive and inferential statistics?
2 Hypothesis Testing & Data Analysis 5 What is the difference between descriptive and inferential statistics? Statistics 8 Tools to help us understand our data. Makes a complicated mess simple to understand.
More informationCourse: Model, Learning, and Inference: Lecture 5
Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.
More informationProof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems.
Math 232  Discrete Math 2.1 Direct Proofs and Counterexamples Notes Axiom: Proposition that is assumed to be true. Proof: A logical argument establishing the truth of the theorem given the truth of the
More informationBayesian Networks. Read R&N Ch. 14.114.2. Next lecture: Read R&N 18.118.4
Bayesian Networks Read R&N Ch. 14.114.2 Next lecture: Read R&N 18.118.4 You will be expected to know Basic concepts and vocabulary of Bayesian networks. Nodes represent random variables. Directed arcs
More informationLecture 3: Summations and Analyzing Programs with Loops
high school algebra If c is a constant (does not depend on the summation index i) then ca i = c a i and (a i + b i )= a i + b i There are some particularly important summations, which you should probably
More informationChapter 7 Part 2. Hypothesis testing Power
Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship
More informationProblems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment,
Uncertainty Problems often have a certain amount of uncertainty, possibly due to: Incompleteness of information about the environment, E.g., loss of sensory information such as vision Incorrectness in
More informationSummary of Probability
Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible
More informationCS 441 Discrete Mathematics for CS Lecture 2. Propositional logic. CS 441 Discrete mathematics for CS. Course administration
CS 441 Discrete Mathematics for CS Lecture 2 Propositional logic Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Course administration Homework 1 First homework assignment is out today will be posted
More informationMarkov Chains. Ben Langmead. Department of Computer Science
Markov Chains Ben Langmead Department of Computer Science You are free to use these slides. If you do, please sign the guestbook (www.langmeadlab.org/teachingmaterials), or email me (ben.langmead@gmail.com)
More informationMath 58. Rumbos Fall 2008 1. Solutions to Assignment #3
Math 58. Rumbos Fall 2008 1 Solutions to Assignment #3 1. Use randomization to test the null hypothesis that there is no difference between calcium supplementation and a placebo for the experimental data
More informationChapter 5  Probability
Chapter 5  Probability 5.1 Basic Ideas An experiment is a process that, when performed, results in exactly one of many observations. These observations are called the outcomes of the experiment. The set
More informationBayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
1 Learning Goals Bayesian Updating: Odds Class 12, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to convert between odds and probability. 2. Be able to update prior odds to posterior odds
More informationMarkov chains and Markov Random Fields (MRFs)
Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume
More informationCHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES
CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,
More informationEXAM. Exam #3. Math 1430, Spring 2002. April 21, 2001 ANSWERS
EXAM Exam #3 Math 1430, Spring 2002 April 21, 2001 ANSWERS i 60 pts. Problem 1. A city has two newspapers, the Gazette and the Journal. In a survey of 1, 200 residents, 500 read the Journal, 700 read the
More informationPhilosophical argument
Michael Lacewing Philosophical argument At the heart of philosophy is philosophical argument. Arguments are different from assertions. Assertions are simply stated; arguments always involve giving reasons.
More informationJoint Distribution and Correlation
Joint Distribution and Correlation Michael Ash Lecture 3 Reminder: Start working on the Problem Set Mean and Variance of Linear Functions of an R.V. Linear Function of an R.V. Y = a + bx What are the properties
More informationMidterm Examination 1 with Solutions  Math 574, Frank Thorne Thursday, February 9, 2012
Midterm Examination 1 with Solutions  Math 574, Frank Thorne (thorne@math.sc.edu) Thursday, February 9, 2012 1. (3 points each) For each sentence below, say whether it is logically equivalent to the sentence
More informationMachine Learning. CS 188: Artificial Intelligence Naïve Bayes. Example: Digit Recognition. Other Classification Tasks
CS 188: Artificial Intelligence Naïve Bayes Machine Learning Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters
More informationThe hardest logic puzzle ever becomes even tougher
The hardest logic puzzle ever becomes even tougher Abstract The hardest logic puzzle ever presented by George Boolos became a target for philosophers and logicians who tried to modify it and make it even
More informationNew Zealand Mathematical Olympiad Committee. Induction
New Zealand Mathematical Olympiad Committee Induction Chris Tuffley 1 Proof by dominoes Mathematical induction is a useful tool for proving statements like P is true for all natural numbers n, for example
More informationMath: Unit Two Vocabulary
(2.2) Algorithm a set of stepbystep instructions for doing something, such as carrying out a computation or solving a problem. (2.8) Ballpark Estimate A rough estimate 47 + 62 =? Using a ballpark estimate:
More informationIncorporating Evidence in Bayesian networks with the Select Operator
Incorporating Evidence in Bayesian networks with the Select Operator C.J. Butz and F. Fang Department of Computer Science, University of Regina Regina, Saskatchewan, Canada SAS 0A2 {butz, fang11fa}@cs.uregina.ca
More informationBayesian network inference
Bayesian network inference Given: Query variables: X Evidence observed variables: E = e Unobserved variables: Y Goal: calculate some useful information about the query variables osterior Xe MA estimate
More informationLecture 2: October 24
Machine Learning: Foundations Fall Semester, 2 Lecture 2: October 24 Lecturer: Yishay Mansour Scribe: Shahar Yifrah, Roi Meron 2. Bayesian Inference  Overview This lecture is going to describe the basic
More informationDiscrete Math in Computer Science Homework 7 Solutions (Max Points: 80)
Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you
More information