# Think Bayes. Bayesian Statistics Made Simple. Version 1.0.5

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Think Bayes Bayesian Statistics Made Simple Version 1.0.5

2

3 Think Bayes Bayesian Statistics Made Simple Version Allen B. Downey Green Tea Press Needham, Massachusetts

4 Copyright 2012 Allen B. Downey. Green Tea Press 9 Washburn Ave Needham MA Permission is granted to copy, distribute, and/or modify this document under the terms of the Creative Commons Attribution-NonCommercial 3.0 Unported License, which is available at licenses/by-nc/3.0/.

5 Preface 0.1 My theory, which is mine The premise of this book, and the other books in the Think X series, is that if you know how to program, you can use that skill to learn other topics. Most books on Bayesian statistics use mathematical notation and present ideas in terms of mathematical concepts like calculus. This book uses Python code instead of math, and discrete approximations instead of continuous mathematics. As a result, what would be an integral in a math book becomes a summation, and most operations on probability distributions are simple loops. I think this presentation is easier to understand, at least for people with programming skills. It is also more general, because when we make modeling decisions, we can choose the most appropriate model without worrying too much about whether the model lends itself to conventional analysis. Also, it provides a smooth development path from simple examples to realworld problems. Chapter 3 is a good example. It starts with a simple example involving dice, one of the staples of basic probability. From there it proceeds in small steps to the locomotive problem, which I borrowed from Mosteller s Fifty Challenging Problems in Probability with Solutions, and from there to the German tank problem, a famously successful application of Bayesian methods during World War II. 0.2 Modeling and approximation Most chapters in this book are motivated by a real-world problem, so they involve some degree of modeling. Before we can apply Bayesian methods (or any other analysis), we have to make decisions about which parts of the

8 viii Chapter 0. Preface 0.4 Code style Experienced Python programmers will notice that the code in this book does not comply with PEP 8, which is the most common style guide for Python ( Specifically, PEP 8 calls for lowercase function names with underscores between words, like_this. In this book and the accompanying code, function and method names begin with a capital letter and use camel case, LikeThis. I broke this rule because I developed some of the code while I was a Visiting Scientist at Google, so I followed the Google style guide, which deviates from PEP 8 in a few places. Once I got used to Google style, I found that I liked it. And at this point, it would be too much trouble to change. Also on the topic of style, I write Bayes s theorem with an s after the apostrophe, which is preferred in some style guides and deprecated in others. I don t have a strong preference. I had to choose one, and this is the one I chose. And finally one typographical note: throughout the book, I use PMF and CDF for the mathematical concept of a probability mass function or cumulative distribution function, and Pmf and Cdf to refer to the Python objects I use to represent them. 0.5 Prerequisites There are several excellent modules for doing Bayesian statistics in Python, including pymc and OpenBUGS. I chose not to use them for this book because you need a fair amount of background knowledge to get started with these modules, and I want to keep the prerequisites minimal. If you know Python and a little bit about probability, you are ready to start this book. Chapter 1 is about probability and Bayes s theorem; it has no code. Chapter 2 introduces Pmf, a thinly disguised Python dictionary I use to represent a probability mass function (PMF). Then Chapter 3 introduces Suite, a kind of Pmf that provides a framework for doing Bayesian updates. And that s just about all there is to it. Well, almost. In some of the later chapters, I use analytic distributions including the Gaussian (normal) distribution, the exponential and Poisson distributions, and the beta distribution. In Chapter 15 I break out the lesscommon Dirichlet distribution, but I explain it as I go along. If you are not

9 0.5. Prerequisites ix familiar with these distributions, you can read about them on Wikipedia. You could also read the companion to this book, Think Stats, or an introductory statistics book (although I m afraid most of them take a mathematical approach that is not particularly helpful for practical purposes). Contributor List If you have a suggestion or correction, please send to If I make a change based on your feedback, I will add you to the contributor list (unless you ask to be omitted). If you include at least part of the sentence the error appears in, that makes it easy for me to search. Page and section numbers are fine, too, but not as easy to work with. Thanks! First, I have to acknowledge David MacKay s excellent book, Information Theory, Inference, and Learning Algorithms, which is where I first came to understand Bayesian methods. With his permission, I use several problems from his book as examples. This book also benefited from my interactions with Sanjoy Mahajan, especially in fall 2012, when I audited his class on Bayesian Inference at Olin College. I wrote parts of this book during project nights with the Boston Python User Group, so I would like to thank them for their company and pizza. Jonathan Edwards sent in the first typo. George Purkins found a markup error. Olivier Yiptong sent several helpful suggestions. Yuriy Pasichnyk found several errors. Kristopher Overholt sent a long list of corrections and suggestions. Robert Marcus found a misplaced i. Max Hailperin suggested a clarification in Chapter 1. Markus Dobler pointed out that drawing cookies from a bowl with replacement is an unrealistic scenario. Tom Pollard and Paul A. Giannaros spotted a version problem with some of the numbers in the train example.

10 x Chapter 0. Preface Ram Limbu found a typo and suggested a clarification. In spring 2013, students in my class, Computational Bayesian Statistics, made many helpful corrections and suggestions: Kai Austin, Claire Barnes, Kari Bender, Rachel Boy, Kat Mendoza, Arjun Iyer, Ben Kroop, Nathan Lintz, Kyle McConnaughay, Alec Radford, Brendan Ritter, and Evan Simpson. Greg Marra and Matt Aasted helped me clarify the discussion of The Price is Right problem. Marcus Ogren pointed out that the original statement of the locomotive problem was ambiguous. Jasmine Kwityn and Dan Fauxsmith at O Reilly Media proofread the book and found many opportunities for improvement. James Lawry spotted a math error. Ben Kahle found a reference to the wrong figure. Jeffrey Law found an inconsistency between the text and the code.

11 Contents Preface v 0.1 My theory, which is mine v 0.2 Modeling and approximation v 0.3 Working with the code vii 0.4 Code style viii 0.5 Prerequisites viii 1 Bayes s Theorem Conditional probability Conjoint probability The cookie problem Bayes s theorem The diachronic interpretation The M&M problem The Monty Hall problem Discussion Computational Statistics Distributions The cookie problem

12 xii Contents 2.3 The Bayesian framework The Monty Hall problem Encapsulating the framework The M&M problem Discussion Exercises Estimation The dice problem The locomotive problem What about that prior? An alternative prior Credible intervals Cumulative distribution functions The German tank problem Discussion Exercises More Estimation The Euro problem Summarizing the posterior Swamping the priors Optimization The beta distribution Discussion Exercises

13 Contents xiii 5 Odds and Addends Odds The odds form of Bayes s theorem Oliver s blood Addends Maxima Mixtures Discussion Decision Analysis The Price is Right problem The prior Probability density functions Representing PDFs Modeling the contestants Likelihood Update Optimal bidding Discussion Prediction The Boston Bruins problem Poisson processes The posteriors The distribution of goals The probability of winning Sudden death Discussion Exercises

14 xiv Contents 8 Observer Bias The Red Line problem The model Wait times Predicting wait times Estimating the arrival rate Incorporating uncertainty Decision analysis Discussion Exercises Two Dimensions Paintball The suite Trigonometry Likelihood Joint distributions Conditional distributions Credible intervals Discussion Exercises Approximate Bayesian Computation The Variability Hypothesis Mean and standard deviation Update The posterior distribution of CV

15 Contents xv 10.5 Underflow Log-likelihood A little optimization ABC Robust estimation Who is more variable? Discussion Exercises Hypothesis Testing Back to the Euro problem Making a fair comparison The triangle prior Discussion Exercises Evidence Interpreting SAT scores The scale The prior Posterior A better model Calibration Posterior distribution of efficacy Predictive distribution Discussion

16 xvi Contents 13 Simulation The Kidney Tumor problem A simple model A more general model Implementation Caching the joint distribution Conditional distributions Serial Correlation Discussion A Hierarchical Model The Geiger counter problem Start simple Make it hierarchical A little optimization Extracting the posteriors Discussion Exercises Dealing with Dimensions Belly button bacteria Lions and tigers and bears The hierarchical version Random sampling Optimization Collapsing the hierarchy One more problem

17 Contents xvii 15.8 We re not done yet The belly button data Predictive distributions Joint posterior Coverage Discussion

18 xviii Contents

19 Chapter 1 Bayes s Theorem 1.1 Conditional probability The fundamental idea behind all Bayesian statistics is Bayes s theorem, which is surprisingly easy to derive, provided that you understand conditional probability. So we ll start with probability, then conditional probability, then Bayes s theorem, and on to Bayesian statistics. A probability is a number between 0 and 1 (including both) that represents a degree of belief in a fact or prediction. The value 1 represents certainty that a fact is true, or that a prediction will come true. The value 0 represents certainty that the fact is false. Intermediate values represent degrees of certainty. The value 0.5, often written as 50%, means that a predicted outcome is as likely to happen as not. For example, the probability that a tossed coin lands face up is very close to 50%. A conditional probability is a probability based on some background information. For example, I want to know the probability that I will have a heart attack in the next year. According to the CDC, Every year about 785,000 Americans have a first coronary attack. ( heartdisease/facts.htm) The U.S. population is about 311 million, so the probability that a randomly chosen American will have a heart attack in the next year is roughly 0.3%. But I am not a randomly chosen American. Epidemiologists have identified many factors that affect the risk of heart attacks; depending on those factors, my risk might be higher or lower than average.

20 2 Chapter 1. Bayes s Theorem I am male, 45 years old, and I have borderline high cholesterol. Those factors increase my chances. However, I have low blood pressure and I don t smoke, and those factors decrease my chances. Plugging everything into the online calculator at nih.gov/calculator.asp, I find that my risk of a heart attack in the next year is about 0.2%, less than the national average. That value is a conditional probability, because it is based on a number of factors that make up my condition. The usual notation for conditional probability is p(a B), which is the probability of A given that B is true. In this example, A represents the prediction that I will have a heart attack in the next year, and B is the set of conditions I listed. 1.2 Conjoint probability Conjoint probability is a fancy way to say the probability that two things are true. I write p(a and B) to mean the probability that A and B are both true. If you learned about probability in the context of coin tosses and dice, you might have learned the following formula: p(a and B) = p(a) p(b) WARNING: not always true For example, if I toss two coins, and A means the first coin lands face up, and B means the second coin lands face up, then p(a) = p(b) = 0.5, and sure enough, p(a and B) = p(a) p(b) = But this formula only works because in this case A and B are independent; that is, knowing the outcome of the first event does not change the probability of the second. Or, more formally, p(b A) = p(b). Here is a different example where the events are not independent. Suppose that A means that it rains today and B means that it rains tomorrow. If I know that it rained today, it is more likely that it will rain tomorrow, so p(b A) > p(b). In general, the probability of a conjunction is p(a and B) = p(a) p(b A)

23 1.5. The diachronic interpretation 5 which reduces to 3/5. So the vanilla cookie is evidence in favor of the hypothesis that we chose Bowl 1, because vanilla cookies are more likely to come from Bowl 1. This example demonstrates one use of Bayes s theorem: it provides a strategy to get from p(b A) to p(a B). This strategy is useful in cases, like the cookie problem, where it is easier to compute the terms on the right side of Bayes s theorem than the term on the left. 1.5 The diachronic interpretation There is another way to think of Bayes s theorem: it gives us a way to update the probability of a hypothesis, H, in light of some body of data, D. This way of thinking about Bayes s theorem is called the diachronic interpretation. Diachronic means that something is happening over time; in this case the probability of the hypotheses changes, over time, as we see new data. Rewriting Bayes s theorem with H and D yields: p(h D) = p(h) p(d H) p(d) In this interpretation, each term has a name: p(h) is the probability of the hypothesis before we see the data, called the prior probability, or just prior. p(h D) is what we want to compute, the probability of the hypothesis after we see the data, called the posterior. p(d H) is the probability of the data under the hypothesis, called the likelihood. p(d) is the probability of the data under any hypothesis, called the normalizing constant. Sometimes we can compute the prior based on background information. For example, the cookie problem specifies that we choose a bowl at random with equal probability.

24 6 Chapter 1. Bayes s Theorem In other cases the prior is subjective; that is, reasonable people might disagree, either because they use different background information or because they interpret the same information differently. The likelihood is usually the easiest part to compute. In the cookie problem, if we know which bowl the cookie came from, we find the probability of a vanilla cookie by counting. The normalizing constant can be tricky. It is supposed to be the probability of seeing the data under any hypothesis at all, but in the most general case it is hard to nail down what that means. Most often we simplify things by specifying a set of hypotheses that are Mutually exclusive: At most one hypothesis in the set can be true, and Collectively exhaustive: There are no other possibilities; at least one of the hypotheses has to be true. I use the word suite for a set of hypotheses that has these properties. In the cookie problem, there are only two hypotheses the cookie came from Bowl 1 or Bowl 2 and they are mutually exclusive and collectively exhaustive. In that case we can compute p(d) using the law of total probability, which says that if there are two exclusive ways that something might happen, you can add up the probabilities like this: p(d) = p(b 1 ) p(d B 1 ) + p(b 2 ) p(d B 2 ) Plugging in the values from the cookie problem, we have p(d) = (1/2) (3/4) + (1/2) (1/2) = 5/8 which is what we computed earlier by mentally combining the two bowls. 1.6 The M&M problem M&M s are small candy-coated chocolates that come in a variety of colors. Mars, Inc., which makes M&M s, changes the mixture of colors from time to time.

25 1.6. The M&M problem 7 In 1995, they introduced blue M&M s. Before then, the color mix in a bag of plain M&M s was 30% Brown, 20% Yellow, 20% Red, 10% Green, 10% Orange, 10% Tan. Afterward it was 24% Blue, 20% Green, 16% Orange, 14% Yellow, 13% Red, 13% Brown. Suppose a friend of mine has two bags of M&M s, and he tells me that one is from 1994 and one from He won t tell me which is which, but he gives me one M&M from each bag. One is yellow and one is green. What is the probability that the yellow one came from the 1994 bag? This problem is similar to the cookie problem, with the twist that I draw one sample from each bowl/bag. This problem also gives me a chance to demonstrate the table method, which is useful for solving problems like this on paper. In the next chapter we will solve them computationally. The first step is to enumerate the hypotheses. The bag the yellow M&M came from I ll call Bag 1; I ll call the other Bag 2. So the hypotheses are: A: Bag 1 is from 1994, which implies that Bag 2 is from B: Bag 1 is from 1996 and Bag 2 from Now we construct a table with a row for each hypothesis and a column for each term in Bayes s theorem: Prior Likelihood Posterior p(h) p(d H) p(h) p(d H) p(h D) A 1/2 (20)(20) /27 B 1/2 (14)(10) 70 7/27 The first column has the priors. Based on the statement of the problem, it is reasonable to choose p(a) = p(b) = 1/2. The second column has the likelihoods, which follow from the information in the problem. For example, if A is true, the yellow M&M came from the 1994 bag with probability 20%, and the green came from the 1996 bag with probability 20%. If B is true, the yellow M&M came from the 1996 bag with probability 14%, and the green came from the 1994 bag with probability 10%. Because the selections are independent, we get the conjoint probability by multiplying. The third column is just the product of the previous two. The sum of this column, 270, is the normalizing constant. To get the last column, which

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### BNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I

BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential

### Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify

### Book Review of Rosenhouse, The Monty Hall Problem. Leslie Burkholder 1

Book Review of Rosenhouse, The Monty Hall Problem Leslie Burkholder 1 The Monty Hall Problem, Jason Rosenhouse, New York, Oxford University Press, 2009, xii, 195 pp, US \$24.95, ISBN 978-0-19-5#6789-8 (Source

### Decision Theory. 36.1 Rational prospecting

36 Decision Theory Decision theory is trivial, apart from computational details (just like playing chess!). You have a choice of various actions, a. The world may be in one of many states x; which one

### Unit 19: Probability Models

Unit 19: Probability Models Summary of Video Probability is the language of uncertainty. Using statistics, we can better predict the outcomes of random phenomena over the long term from the very complex,

### fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson Contents What Are These Demos About? How to Use These Demos If This Is Your First Time Using Fathom Tutorial: An Extended Example

### CONTENTS OF DAY 2. II. Why Random Sampling is Important 9 A myth, an urban legend, and the real reason NOTES FOR SUMMER STATISTICS INSTITUTE COURSE

1 2 CONTENTS OF DAY 2 I. More Precise Definition of Simple Random Sample 3 Connection with independent random variables 3 Problems with small populations 8 II. Why Random Sampling is Important 9 A myth,

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### WEEK #22: PDFs and CDFs, Measures of Center and Spread

WEEK #22: PDFs and CDFs, Measures of Center and Spread Goals: Explore the effect of independent events in probability calculations. Present a number of ways to represent probability distributions. Textbook

### Probabilities and Random Variables

Probabilities and Random Variables This is an elementary overview of the basic concepts of probability theory. 1 The Probability Space The purpose of probability theory is to model random experiments so

### Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22

CS 70 Discrete Mathematics for CS Fall 2006 Papadimitriou & Vazirani Lecture 22 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice, roulette

### 8. You have three six-sided dice (one red, one green, and one blue.) The numbers of the dice are

Probability Warm up problem. Alice and Bob each pick, at random, a real number between 0 and 10. Call Alice s number A and Bob s number B. The is the probability that A B? What is the probability that

### Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80)

Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you

### The Assumption(s) of Normality

The Assumption(s) of Normality Copyright 2000, 2011, J. Toby Mordkoff This is very complicated, so I ll provide two versions. At a minimum, you should know the short one. It would be great if you knew

### AP Stats - Probability Review

AP Stats - Probability Review Multiple Choice Identify the choice that best completes the statement or answers the question. 1. I toss a penny and observe whether it lands heads up or tails up. Suppose

### Bayesian Analysis for the Social Sciences

Bayesian Analysis for the Social Sciences Simon Jackman Stanford University http://jackman.stanford.edu/bass November 9, 2012 Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November

### 4. Introduction to Statistics

Statistics for Engineers 4-1 4. Introduction to Statistics Descriptive Statistics Types of data A variate or random variable is a quantity or attribute whose value may vary from one unit of investigation

### Statistics and Probability

Statistics and Probability TABLE OF CONTENTS 1 Posing Questions and Gathering Data. 2 2 Representing Data. 7 3 Interpreting and Evaluating Data 13 4 Exploring Probability..17 5 Games of Chance 20 6 Ideas

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice,

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

### Using simulation to calculate the NPV of a project

Using simulation to calculate the NPV of a project Marius Holtan Onward Inc. 5/31/2002 Monte Carlo simulation is fast becoming the technology of choice for evaluating and analyzing assets, be it pure financial

### LOGNORMAL MODEL FOR STOCK PRICES

LOGNORMAL MODEL FOR STOCK PRICES MICHAEL J. SHARPE MATHEMATICS DEPARTMENT, UCSD 1. INTRODUCTION What follows is a simple but important model that will be the basis for a later study of stock prices as

### Lecture 14. Chapter 7: Probability. Rule 1: Rule 2: Rule 3: Nancy Pfenning Stats 1000

Lecture 4 Nancy Pfenning Stats 000 Chapter 7: Probability Last time we established some basic definitions and rules of probability: Rule : P (A C ) = P (A). Rule 2: In general, the probability of one event

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Problem sets for BUEC 333 Part 1: Probability and Statistics

Problem sets for BUEC 333 Part 1: Probability and Statistics I will indicate the relevant exercises for each week at the end of the Wednesday lecture. Numbered exercises are back-of-chapter exercises from

### Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

### MONEY MANAGEMENT. Guy Bower delves into a topic every trader should endeavour to master - money management.

MONEY MANAGEMENT Guy Bower delves into a topic every trader should endeavour to master - money management. Many of us have read Jack Schwager s Market Wizards books at least once. As you may recall it

### Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Be able to explain the difference between the p-value and a posterior

### Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive Amounts

### 6.042/18.062J Mathematics for Computer Science. Expected Value I

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

### The Monty Hall Problem

Gateway to Exploring Mathematical Sciences (GEMS) 9 November, 2013 Lecture Notes The Monty Hall Problem Mark Huber Fletcher Jones Foundation Associate Professor of Mathematics and Statistics and George

### Massachusetts Institute of Technology

n (i) m m (ii) n m ( (iii) n n n n (iv) m m Massachusetts Institute of Technology 6.0/6.: Probabilistic Systems Analysis (Quiz Solutions Spring 009) Question Multiple Choice Questions: CLEARLY circle the

### Life Insurance Modelling: Notes for teachers. Overview

Life Insurance Modelling: Notes for teachers In this mathematics activity, students model the following situation: An investment fund is set up Investors pay a set amount at the beginning of 20 years.

### Likelihood: Frequentist vs Bayesian Reasoning

"PRINCIPLES OF PHYLOGENETICS: ECOLOGY AND EVOLUTION" Integrative Biology 200B University of California, Berkeley Spring 2009 N Hallinan Likelihood: Frequentist vs Bayesian Reasoning Stochastic odels and

### UNIT AUTHOR: Elizabeth Hume, Colonial Heights High School, Colonial Heights City Schools

Money & Finance I. UNIT OVERVIEW & PURPOSE: The purpose of this unit is for students to learn how savings accounts, annuities, loans, and credit cards work. All students need a basic understanding of how

### TRANSCRIPT: In this lecture, we will talk about both theoretical and applied concepts related to hypothesis testing.

This is Dr. Chumney. The focus of this lecture is hypothesis testing both what it is, how hypothesis tests are used, and how to conduct hypothesis tests. 1 In this lecture, we will talk about both theoretical

### ECON 459 Game Theory. Lecture Notes Auctions. Luca Anderlini Spring 2015

ECON 459 Game Theory Lecture Notes Auctions Luca Anderlini Spring 2015 These notes have been used before. If you can still spot any errors or have any suggestions for improvement, please let me know. 1

### Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University

Lecture 1 Introduction Properties of Probability Methods of Enumeration Asrat Temesgen Stockholm University 1 Chapter 1 Probability 1.1 Basic Concepts In the study of statistics, we consider experiments

### MINITAB ASSISTANT WHITE PAPER

MINITAB ASSISTANT WHITE PAPER This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. One-Way

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### The equivalence of logistic regression and maximum entropy models

The equivalence of logistic regression and maximum entropy models John Mount September 23, 20 Abstract As our colleague so aptly demonstrated ( http://www.win-vector.com/blog/20/09/the-simplerderivation-of-logistic-regression/

### Appendix D: The Monty Hall Controversy

Appendix D: The Monty Hall Controversy Appendix D: The Monty Hall Controversy - Page 1 Let's Make a Deal Prepared by Rich Williams, Spring 1991 Last Modified Fall, 2004 You are playing Let's Make a Deal

### GCSE Statistics Revision notes

GCSE Statistics Revision notes Collecting data Sample This is when data is collected from part of the population. There are different methods for sampling Random sampling, Stratified sampling, Systematic

### Business Analytics. Methods, Models, and Decisions. James R. Evans : University of Cincinnati PEARSON

Business Analytics Methods, Models, and Decisions James R. Evans : University of Cincinnati PEARSON Boston Columbus Indianapolis New York San Francisco Upper Saddle River Amsterdam Cape Town Dubai London

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

### A Concrete Introduction. to the Abstract Concepts. of Integers and Algebra using Algebra Tiles

A Concrete Introduction to the Abstract Concepts of Integers and Algebra using Algebra Tiles Table of Contents Introduction... 1 page Integers 1: Introduction to Integers... 3 2: Working with Algebra Tiles...

### DECISION MAKING UNDER UNCERTAINTY:

DECISION MAKING UNDER UNCERTAINTY: Models and Choices Charles A. Holloway Stanford University TECHNISCHE HOCHSCHULE DARMSTADT Fachbereich 1 Gesamtbibliothek Betrtebswirtscrtaftslehre tnventar-nr. :...2>2&,...S'.?S7.

### Week 2: Conditional Probability and Bayes formula

Week 2: Conditional Probability and Bayes formula We ask the following question: suppose we know that a certain event B has occurred. How does this impact the probability of some other A. This question

### Probability Models.S1 Introduction to Probability

Probability Models.S1 Introduction to Probability Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard The stochastic chapters of this book involve random variability. Decisions are

### People have thought about, and defined, probability in different ways. important to note the consequences of the definition:

PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models

### Math 141. Lecture 2: More Probability! Albyn Jones 1. jones@reed.edu www.people.reed.edu/ jones/courses/141. 1 Library 304. Albyn Jones Math 141

Math 141 Lecture 2: More Probability! Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Outline Law of total probability Bayes Theorem the Multiplication Rule, again Recall

### Bayesian Tutorial (Sheet Updated 20 March)

Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that

### Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010

Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010 Week 1 Week 2 14.0 Students organize and describe distributions of data by using a number of different

### Assignment 2: Option Pricing and the Black-Scholes formula The University of British Columbia Science One CS 2015-2016 Instructor: Michael Gelbart

Assignment 2: Option Pricing and the Black-Scholes formula The University of British Columbia Science One CS 2015-2016 Instructor: Michael Gelbart Overview Due Thursday, November 12th at 11:59pm Last updated

### Multivariate Normal Distribution

Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

### Programming Using Python

Introduction to Computation and Programming Using Python Revised and Expanded Edition John V. Guttag The MIT Press Cambridge, Massachusetts London, England CONTENTS PREFACE xiii ACKNOWLEDGMENTS xv 1 GETTING

### VISUAL ALGEBRA FOR COLLEGE STUDENTS. Laurie J. Burton Western Oregon University

VISUAL ALGEBRA FOR COLLEGE STUDENTS Laurie J. Burton Western Oregon University VISUAL ALGEBRA FOR COLLEGE STUDENTS TABLE OF CONTENTS Welcome and Introduction 1 Chapter 1: INTEGERS AND INTEGER OPERATIONS

### Minitab Guide. This packet contains: A Friendly Guide to Minitab. Minitab Step-By-Step

Minitab Guide This packet contains: A Friendly Guide to Minitab An introduction to Minitab; including basic Minitab functions, how to create sets of data, and how to create and edit graphs of different

### image used without permission from: random%5crandom_08.jpg

Name: Midterm Exam 01 (Part 1 in class: open book, notes, laptop, calculators. It is closed internet, wireless, other people.) (Part 2 take home solo submission: open everything and everyone worldwide.

### Using Laws of Probability. Sloan Fellows/Management of Technology Summer 2003

Using Laws of Probability Sloan Fellows/Management of Technology Summer 2003 Uncertain events Outline The laws of probability Random variables (discrete and continuous) Probability distribution Histogram

### Session 8 Probability

Key Terms for This Session Session 8 Probability Previously Introduced frequency New in This Session binomial experiment binomial probability model experimental probability mathematical probability outcome

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Name: Section: I pledge my honor that I have not violated the Honor Code Signature: This exam has 34 pages. You have 3 hours to complete this

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### Introduction and Overview

Introduction and Overview Probability and Statistics is a topic that is quickly growing, has become a major part of our educational program, and has a substantial role in the NCTM Standards. While covering

### 3.2 Roulette and Markov Chains

238 CHAPTER 3. DISCRETE DYNAMICAL SYSTEMS WITH MANY VARIABLES 3.2 Roulette and Markov Chains In this section we will be discussing an application of systems of recursion equations called Markov Chains.

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### If A is divided by B the result is 2/3. If B is divided by C the result is 4/7. What is the result if A is divided by C?

Problem 3 If A is divided by B the result is 2/3. If B is divided by C the result is 4/7. What is the result if A is divided by C? Suggested Questions to ask students about Problem 3 The key to this question

### 14:440:127 Introduction to Computers for Engineers. Notes for Lecture 06

14:440:127 Introduction to Computers for Engineers Notes for Lecture 06 Rutgers University, Spring 2010 Instructor- Blase E. Ur 1 Loop Examples 1.1 Example- Sum Primes Let s say we wanted to sum all 1,

### MATH 10: Elementary Statistics and Probability Chapter 3: Probability Topics

MATH 10: Elementary Statistics and Probability Chapter 3: Probability Topics Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of slides, you should

### Stat 20: Intro to Probability and Statistics

Stat 20: Intro to Probability and Statistics Lecture 16: More Box Models Tessa L. Childers-Day UC Berkeley 22 July 2014 By the end of this lecture... You will be able to: Determine what we expect the sum

### Chi Square Distribution

17. Chi Square A. Chi Square Distribution B. One-Way Tables C. Contingency Tables D. Exercises Chi Square is a distribution that has proven to be particularly useful in statistics. The first section describes

### MONT 107N Understanding Randomness Solutions For Final Examination May 11, 2010

MONT 07N Understanding Randomness Solutions For Final Examination May, 00 Short Answer (a) (0) How are the EV and SE for the sum of n draws with replacement from a box computed? Solution: The EV is n times

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### Artificial Intelligence Mar 27, Bayesian Networks 1 P (T D)P (D) + P (T D)P ( D) =

Artificial Intelligence 15-381 Mar 27, 2007 Bayesian Networks 1 Recap of last lecture Probability: precise representation of uncertainty Probability theory: optimal updating of knowledge based on new information

### MATH 201. Final ANSWERS August 12, 2016

MATH 01 Final ANSWERS August 1, 016 Part A 1. 17 points) A bag contains three different types of dice: four 6-sided dice, five 8-sided dice, and six 0-sided dice. A die is drawn from the bag and then rolled.

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Basic Probability. Probability: The part of Mathematics devoted to quantify uncertainty

AMS 5 PROBABILITY Basic Probability Probability: The part of Mathematics devoted to quantify uncertainty Frequency Theory Bayesian Theory Game: Playing Backgammon. The chance of getting (6,6) is 1/36.

### Chapter 4. Probability Distributions

Chapter 4 Probability Distributions Lesson 4-1/4-2 Random Variable Probability Distributions This chapter will deal the construction of probability distribution. By combining the methods of descriptive

### Probabilities. Probability of a event. From Random Variables to Events. From Random Variables to Events. Probability Theory I

Victor Adamchi Danny Sleator Great Theoretical Ideas In Computer Science Probability Theory I CS 5-25 Spring 200 Lecture Feb. 6, 200 Carnegie Mellon University We will consider chance experiments with

### PROBABILITY 14.3. section. The Probability of an Event

4.3 Probability (4-3) 727 4.3 PROBABILITY In this section In the two preceding sections we were concerned with counting the number of different outcomes to an experiment. We now use those counting techniques

### Probability. A random sample is selected in such a way that every different sample of size n has an equal chance of selection.

1 3.1 Sample Spaces and Tree Diagrams Probability This section introduces terminology and some techniques which will eventually lead us to the basic concept of the probability of an event. The Rare Event

### Chapter 5 A Survey of Probability Concepts

Chapter 5 A Survey of Probability Concepts True/False 1. Based on a classical approach, the probability of an event is defined as the number of favorable outcomes divided by the total number of possible

### 9. Sampling Distributions

9. Sampling Distributions Prerequisites none A. Introduction B. Sampling Distribution of the Mean C. Sampling Distribution of Difference Between Means D. Sampling Distribution of Pearson's r E. Sampling

### Forecast Confidence Level and Portfolio Optimization

and Portfolio Optimization by Richard O. Michaud and Robert O. Michaud New Frontier Advisors Newsletter July 2004 Abstract This report focuses on the role and importance of the uncertainty in forecast

### psychology and economics

psychology and economics lecture 9: biases in statistical reasoning tomasz strzalecki failures of Bayesian updating how people fail to update in a Bayesian way how Bayes law fails to describe how people

### Programming Your Calculator Casio fx-7400g PLUS

Programming Your Calculator Casio fx-7400g PLUS Barry Kissane Programming Your Calculator: Casio fx-7400g PLUS Published by Shriro Australia Pty Limited 72-74 Gibbes Street, Chatswood NSW 2067, Australia

### Sampling and Hypothesis Testing

Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

### Penalized regression: Introduction

Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihood

### Test of Hypotheses. Since the Neyman-Pearson approach involves two statistical hypotheses, one has to decide which one

Test of Hypotheses Hypothesis, Test Statistic, and Rejection Region Imagine that you play a repeated Bernoulli game: you win \$1 if head and lose \$1 if tail. After 10 plays, you lost \$2 in net (4 heads

### Introduction to Hypothesis Testing

I. Terms, Concepts. Introduction to Hypothesis Testing A. In general, we do not know the true value of population parameters - they must be estimated. However, we do have hypotheses about what the true

### AMS 5 CHANCE VARIABILITY

AMS 5 CHANCE VARIABILITY The Law of Averages When tossing a fair coin the chances of tails and heads are the same: 50% and 50%. So if the coin is tossed a large number of times, the number of heads and

### RUTHERFORD HIGH SCHOOL Rutherford, New Jersey COURSE OUTLINE STATISTICS AND PROBABILITY

RUTHERFORD HIGH SCHOOL Rutherford, New Jersey COURSE OUTLINE STATISTICS AND PROBABILITY I. INTRODUCTION According to the Common Core Standards (2010), Decisions or predictions are often based on data numbers