15-381: Artificial Intelligence. Probabilistic Reasoning and Inference

Size: px

Start display at page:

Download "15-381: Artificial Intelligence. Probabilistic Reasoning and Inference"

Abel Aldous Joseph
10 years ago
Views:

1 5-38: Artificial Intelligence robabilistic Reasoning and Inference

2 Advantages of probabilistic reasoning Appropriate for complex, uncertain, environments - Will it rain tomorrow? Applies naturally to many domains - Robot predicting the direction of road, biology, Word paper clip Allows to generalize acquired knowledge and incorporate prior belief - Medical diagnosis Easy to integrate different information sources - Robot s sensors

Applies naturally to many domains - Robot predicting the direction of road, biology, Word

3 Unmanned vehicles Examples

4 Examples: Speech processing

5 Example: Biological data ATGAAGCTACTGTCTTCTATCGAACAAGCATGCG ATATTTGCCGACTTAAAAAGCTCAAG TGCTCCAAAGAAAAACCGAAGTGCGCCAAGTGT CTGAAGAACAACTGGGAGTGTCGCTAC TCTCCCAAAACCAAAAGGTCTCCGCTGACTAGG GCACATCTGACAGAAGTGGAATCAAGG CTAGAAAGACTGGAACAGCTATTTCTACTGATTT TTCCTCGAGAAGACCTTGACATGATT

CTGAAGAACAACTGGGAGTGTCGCTAC TCTCCCAAAACCAAAAGGTCTCCGCTGACTAGG

6 Basic notations Random variable - referring to an element / event whose status is unknown: A it will rain tomorrow Domain usually denoted by Ω - The set of values a random variable can take: - A The stock market will go up this year : Binary - A Number of Steelers wins in 007 : Discrete - A % change in Google stock in 007 : Continuous

random variable can take: - A The stock market will go up this year : Binary - A

7 Axioms of probability Kolmogorov s axioms A variety of useful facts can be derived from just three axioms:. 0 A. true, false 0 3. A B A + B A B

8 Axioms of probability Kolmogorov s axioms A variety of useful facts can be derived from just three axioms:. 0 A. true, false 0 3. A B A + B A B Steelers win the season

9 Axioms of probability Kolmogorov s axioms A variety of useful facts can be derived from just three axioms:. 0 A. true, false 0 3. A B A + B A B

10 Axioms of probability Kolmogorov s axioms A variety of useful facts can be derived from just three axioms:. 0 A. true, false 0 3. A B A + B A B

11 Axioms of probability Kolmogorov s axioms A variety of useful facts can be derived from just three axioms:. 0 A. true, false 0 3. A B A + B A B There have been several other attempts to provide a foundation for probability theory. Kolmogorov s axioms are the most widely used.

A B A + B A B There have been several other attempts to provide a

12 Example of using the axioms Assume a probability for dice as follows: ? {heads,tails}? heads+-tails?

13 Using the axioms How can we use the axioms to prove that: A A?

14 riors Degree of belief in an event in the absence of any other information No rain Rain rain tomorrow 0. no rain tomorrow 0.8

15 Conditional probablity What is the probability of an event given knowledge of another event Example: - raining sunny - raining cloudy - raining cloudy, cold

16 Conditional probability A B : The fraction of cases where A is true if B is true A 0. A B 0.5

17 Conditional probability In some cases, given knowledge of one or more random variables we can improve upon our prior belief of another random variable For example: pslept in movie 0.5 pslept in movie liked movie /3 pdidn t sleep in movie liked movie /3 Liked movie 0 0 Slept

variable For example: pslept in movie 0.

18 Joint distributions The probability that a set of random variables will take a specific value is their joint distribution. Notation: A B or A,B Example: liked movie, slept Liked movie 0 0 Slept

19 Joint distribution cont class size > summer /3 class size > 0, summer? Time regular, summer Evaluation of classes Class size Evaluation

20 Joint distribution cont class size > summer /3 class size > 0, summer 0 Time regular, summer Evaluation of classes Class size Evaluation

21 Joint distribution cont class size > eval /9 class size > 0, eval /9 Evaluation of classes Time regular, summer Class size Evaluation

22 Chain rule The joint distribution can be specified in terms of conditional probability: A,B A B*B Together with Bayes rule which is actually derived from it this is one of the most powerful rules in probabilistic reasoning

23 Bayes rule One of the most important rules for AI usage. Derived from the chain rule: A,B A BB B AA Thus, A B B A A B Thomas Bayes was an English clergyman who set out his theory of probability in 764.

24 Bayes rule cont Often it would be useful to derive the rule a bit further:! A A A B A A B B A A B A B This results from: B A B,A A B A B B,A B,A0

25 Using Bayes rule Cards game: lace your bet on the location of the King!

26 Using Bayes rule Cards game: Do you want to change your bet?

27 Using Bayes rule selb k C k C selb selb k C Computing the posterior probability: C k selb A B C C C selb k C k C selb k C k C selb Bayes rule

28 Using Bayes rule A B C Ck selb / /3 selb C selb C k C k C k + selb k C 0 C 0 /3 / /3 / /3

29 Important points Random variables Chain rule Bayes rule Joint distribution, independence, conditional independence

30 Joint distributions The probability that a set of random variables will take a specific value is their joint distribution. Requires a joint probability table to specify the possible assignments The table can grow very rapidly Liked movie 0 0 Slept How can we decrease the number of columns in the table?

10-601. Machine Learning. http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html

10-601. Machine Learning. http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html 10-601 Machine Learning http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html Course data All up-to-date info is on the course web page: http://www.cs.cmu.edu/afs/cs/academic/class/10601-f10/index.html