# 1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016

Save this PDF as:

Size: px
Start display at page:

Download "1 A simple example. A short introduction to Bayesian statistics, part I Math 218, Mathematical Statistics D Joyce, Spring 2016"

## Transcription

1 and P (B X). In order to do find those conditional probabilities, we ll use Bayes formula. We can easily compute the reverse probabilities A short introduction to Bayesian statistics, part I Math 18, Mathematical Statistics D Joyce, Spring 016 I ll try to make this introduction to Bayesian statistics clear and short. First we ll look as a specific example, then the general setting, then Bayesian statistics for the Bernoulli process, for the Poisson process, and for normal distributions. 1 A simple example Suppose we have two identical urns urn A with 5 red balls and 10 green balls, and urn B with 10 red balls and 5 green balls. We ll select randomly one of the two urns, then sample with replacement that urn to help determine whether we chose A or B. ) n k P (X A) ( 1 k ( ) P (X B) ( 1 n k ( ) so by Bayes formula we derive the posterior probabilities P (A X) ) k P (X A)P (A) P (X A)P (A) + P (X B)P (B) ( 1 ) k ( ) n k 1 ( 1 ) k ( ) n k 1 + ( 1 n k n k + k P (B X) 1 P (A X) k n k + k ) n k ( ) k 1 For example, suppose that in n 10 trials we got k 4 red balls. The posterior probabilities would become P (A X) 4 5 and P (B X) 1 5. Before sampling we ll suppose that we have prior probabilities of 1, that is, P (A) 1 and P (B) 1. Let s take a sample X (X 1, X,..., X n ) of size n, and suppose that k of the n balls we select with replacement are red. We want to use that information to help determine which of the two urns, A or B, we chose. That is, we ll compute P (A X) 1

2 Before the experiment we chose the two urns each with probability 1, that is, the probability of choosing a red ball was either p 1 or p each with probability 1. That s shown in the prior graph on the left. After drawing n 10 balls out of that urn (with replacement) and getting k 4 red balls, we update the probabilities. That s shown in the posterior graph on the right. How this example generalizes. In the example we had a discrete distribution on p, the probability that we d chose a red ball. This parameter p could take two values: p could be 1 with probability 1 when we chose urn A, or p could be with probability 1 when we chose urn B. We actually had a prior distribution on the parameter p. After taking into consideration the outcome k of an experiment, we had a different distribution on p. It was a conditional distribution p k. In general, we won t have only two different values on a parameter, but infinitely many; we ll have a continuous distribution on the parameter instead of a discrete one. The basic principle The setting for Bayesian statistics is a family of distributions parametrized by one or more parameters along with a prior distribution for those parameters. In the example above we had a Bernoulli process parametrized by one parameter p the probability of success. In the example the prior distribution for p was discrete and had only two values, 1 and each with probability 1. A sample X is taken, and a posterior distribution for the parameters is computed. Let s clarify the situation and introduce terminology and notation in the general case where X is a discrete random variable, and there is only one discrete parameter θ. (In practice, θ is a continuous parameter, but in the example above it was discrete, and for this introduction, let s take θ to be discrete). In statistics, we don t know what the value of θ is; our job is to make inferences about θ. The way to find out about θ is to perform many trials and see what happens, that is, to select a random sample from the distribution, X (X 1, X,..., X n ), where each random variable X i has the given distribution. The actual outcomes that are observed I ll denote x (x 1, x,..., x n ). Now, different values of θ lead to different probabilities of the outcome x, that is, P (Xx θ) varies with θ. In the so-called classical statistics, this probability is called the likelihood of θ given the outcome x, denoted L(θ x). The reason the word likelihood is used is the suggestion that the real value of θ is likely to be one with a higher probability P (Xx θ). But this likelihood L(θ x) is not a probability about θ. (Note that classical statistics is much younger than Bayesian statistics and probably should have some other name.) What Bayesian statistics does is replace this concept of likelihood by a real probability. In order to do that, we ll treat the parameter θ as a random variable rather than an unknown constant. Since it s a random variable, I ll use an uppercase Θ. This random variable Θ itself has a probability mass function, which I ll denote f Θ (θ) P (Θθ). This f Θ is called the prior distribution on Θ. It s the probability you have before considering the information in X, the results of an observation. The symbol P (Xx θ) really is a conditional probability now, and it should properly be written P (Xx Θθ), but I ll abbreviate it simply as P (x θ) and leave out the references to the random variables when the context is clear. Using Bayes law we can invert this conditional probability. In full, it says P (Θθ Xx) but we can abbreviate that as P (θ x) P (Xx Θθ)P (Θθ) P (Xx) P (x θ)p (θ). P (x) This conditional probability P (θ x) is called the posterior distribution on Θ. It s the probability you

3 have after taking into consideration new information from an observation. Note that the denominator P (x) is a constant, so the last equation says that the posterior distribution P (θ x) is proportional to P (x θ)p (θ). I ll write proportions with the traditional symbol so that the last statement can be written as P (θ x) P (x θ)p (θ). Using proportions saves a lot of symbols, and we don t lose any information since the constant of proportionality P (x) is known. When we discuss the three settings Bernoulli, Poisson, and normal the random variable X will be either discrete or continuous, but our parameters will all be continuous, not discrete (unlike the simple example above where our parameter p was discrete and only took the two values 1 and ). That means we ll be working with probability density functions instead of probability mass functions. In the continuous case there are analogous statements. In particular, analogous to the last statement, we have f(θ x) f(x θ)f(θ) where f(θ) is the prior density function on the parameter Θ, f(θ x) is the posterior density function on Θ, and f(x θ) is a conditional probability or a conditional density depending on whether X is a continuous or discrete random variable. The Bernoulli process. A single trial X for a Bernoulli process, called a Bernoulli trial, ends with one of two outcomes success where X 1 and failure where X 0. Success occurs with probability p while failure occurs with probability q 1 p. The term Bernoulli process is just another name for a random sample from a Bernoulli population. Thus, it consists of repeated independent Bernoulli trials X (X 1, X,..., X n ) with the same parameter p. The problem for statistics is determining the value of this parameter p. All we know is that it lies between 0 and 1. We also expect the ratio k/n of the number of successes k to the number trials n to approach p as n approaches, but that s a theoretical result that doesn t say much about what p is when n is small. Let s see what the Bayesian approach says here. We start with a prior density function f(p) on p, and take a random sample x (x 1, x,..., x n ). Then the posterior density function is proportional to a conditional probability times the prior density function f(p x) P (Xx p) f(p). Suppose, now, that there are k successes occur among the n trials x. With our convention that X i 1 means the trial X i ended in success, that means that k x 1 + x + + x n. Then Therefore, P (Xx p) p k (1 p) n k. f(p x) p k (1 p) n k f(p). Thus, we have a formula for determining the posterior density function f(p x) from the prior density function f(p). (In order to know a density function, it s enough to know what it s proportional to, because we also know the integral of a density function is 1.) But what should the prior distribution be? That depends on your state of knowledge. You may already have some knowledge about what p might be. But if you don t, maybe the best thing to do is assume that all values of p are equally probable. Let s do that and see what happens. So, assume now that the prior density function f(p) is uniform on the interval [0, 1]. So f(p) 1 on the interval, 0 off it. Then we can determine the posterior density function. On the interval [0, 1], f(p x) p k (1 p) n k f(p) p k (1 p) n k

4 That s enough to tell us this is the beta distribution Beta(k + 1, n + 1 k) because the probability density function for a beta distribution Beta(α, β) is 1 f(x) B(α, β) xα 1 (1 x) β 1 for 0 x 1, where B(α, β) is a constant, namely, the beta function B evaluated at the arguments α and β. Note that the prior distribution f(p) we chose was uniform on [0, 1], and that s actually the beta distribution Beta(1, 1). Let s suppose you have a large number of balls in an urn, every one of which is either red or green, but you have no idea how many there are or what the fraction p of red balls there are. They could even be all red or all green. You decide to make your prior distribution on p uniform, that is Beta(1, 1). This uniform prior density is shaded green in the first figure. green, and the probability is shifted back towards the center. Now f P (p) 6p(1 p). It s shaded blue in the figure. Suppose the next three are red, red, and green, in that order. After each one, we can update the distribution. Next will be Beta(, ), then Beta(4, ), and then Beta(4, ) They appear in the next figure. The first green, second pink, and third blue. Now you choose one ball at random and put it back. If it was red, your new distribution on p is Beta(, 1). The density function of this distribution is f P (p) p. It s shaded pink in the figure. The probability is now more dense near 1 and less near 0. Let s suppose we do it again and get a green ball. Now we ve got Beta(, ). So far, one red and one With each new piece of information the slowly narrows. We can t say much yet with only 5 drawings, reds and greens. A sample of size 5 doesn t say much. Even with so little information, we can still pretty much rule out p being less than 0.05 or greater than Let s see what the distribution would look like with more data. Take three more cases. First, when n 10 and we ve drawn red balls 6 times. Then when n 0 and we ve gotten 14 red balls. And finally when n 50 and we got reds. Those have the three distributions Beta(7, 5) graphed in green, Beta(15, 7) graphed in red, and Beta(4, 18) graphed in blue. These are much skinnier distributions, so we ll squeeze the vertical scale. 4

5 Even after 50 trials, about all we can say is that p is with high probability between 0.4 and We can actually compute that high probability as well, since we have a distribution on p. Math 18 Home Page at 5

### Sufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Sufficient Statistics and Exponential Family 1 Statistics and Sufficient Statistics Suppose we have a random sample X 1,, X n taken from a distribution

### Common probability distributionsi Math 217/218 Probability and Statistics Prof. D. Joyce, 2016

Introduction. ommon probability distributionsi Math 7/8 Probability and Statistics Prof. D. Joyce, 06 I summarize here some of the more common distributions used in probability and statistics. Some are

### An introduction to the Beta-Binomial model

An introduction to the Beta-Binomial model COMPSCI 316: Computational Cognitive Science Dan Navarro & Amy Perfors University of Adelaide Abstract This note provides a detailed discussion of the beta-binomial

### Stat260: Bayesian Modeling and Inference Lecture Date: February 1, Lecture 3

Stat26: Bayesian Modeling and Inference Lecture Date: February 1, 21 Lecture 3 Lecturer: Michael I. Jordan Scribe: Joshua G. Schraiber 1 Decision theory Recall that decision theory provides a quantification

### Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify

### WHERE DOES THE 10% CONDITION COME FROM?

1 WHERE DOES THE 10% CONDITION COME FROM? The text has mentioned The 10% Condition (at least) twice so far: p. 407 Bernoulli trials must be independent. If that assumption is violated, it is still okay

### 1 Prior Probability and Posterior Probability

Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which

### Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS Part 4: Geometric Distribution Negative Binomial Distribution Hypergeometric Distribution Sections 3-7, 3-8 The remaining discrete random

### Pooling and Meta-analysis. Tony O Hagan

Pooling and Meta-analysis Tony O Hagan Pooling Synthesising prior information from several experts 2 Multiple experts The case of multiple experts is important When elicitation is used to provide expert

### Joint distributions Math 217 Probability and Statistics Prof. D. Joyce, Fall 2014

Joint distributions Math 17 Probability and Statistics Prof. D. Joyce, Fall 14 Today we ll look at joint random variables and joint distributions in detail. Product distributions. If Ω 1 and Ω are sample

### Maximum Likelihood Estimation

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

### Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Conditional Probability, Independence and Bayes Theorem Class 3, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of conditional probability and independence

More Quadratic Equations Math 99 N1 Chapter 8 1 Quadratic Equations We won t discuss quadratic inequalities. Quadratic equations are equations where the unknown appears raised to second power, and, possibly

### Chapter 4. Probability and Probability Distributions

Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

### Sampling Central Limit Theorem Proportions. Outline. 1 Sampling. 2 Central Limit Theorem. 3 Proportions

Outline 1 Sampling 2 Central Limit Theorem 3 Proportions Outline 1 Sampling 2 Central Limit Theorem 3 Proportions Populations and samples When we use statistics, we are trying to find out information about

### Series Convergence Tests Math 122 Calculus III D Joyce, Fall 2012

Some series converge, some diverge. Series Convergence Tests Math 22 Calculus III D Joyce, Fall 202 Geometric series. We ve already looked at these. We know when a geometric series converges and what it

### MATH 10: Elementary Statistics and Probability Chapter 9: Hypothesis Testing with One Sample

MATH 10: Elementary Statistics and Probability Chapter 9: Hypothesis Testing with One Sample Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note 11

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao,David Tse Note Conditional Probability A pharmaceutical company is marketing a new test for a certain medical condition. According

### MITES Physics III Summer Introduction 1. 3 Π = Product 2. 4 Proofs by Induction 3. 5 Problems 5

MITES Physics III Summer 010 Sums Products and Proofs Contents 1 Introduction 1 Sum 1 3 Π Product 4 Proofs by Induction 3 5 Problems 5 1 Introduction These notes will introduce two topics: A notation which

### Probability distributions

Probability distributions (Notes are heavily adapted from Harnett, Ch. 3; Hayes, sections 2.14-2.19; see also Hayes, Appendix B.) I. Random variables (in general) A. So far we have focused on single events,

### 4. Introduction to Statistics

Statistics for Engineers 4-1 4. Introduction to Statistics Descriptive Statistics Types of data A variate or random variable is a quantity or attribute whose value may vary from one unit of investigation

### Lecture 7: Continuous Random Variables

Lecture 7: Continuous Random Variables 21 September 2005 1 Our First Continuous Random Variable The back of the lecture hall is roughly 10 meters across. Suppose it were exactly 10 meters, and consider

### An Introduction to Basic Statistics and Probability

An Introduction to Basic Statistics and Probability Shenek Heyward NCSU An Introduction to Basic Statistics and Probability p. 1/4 Outline Basic probability concepts Conditional probability Discrete Random

### Probability OPRE 6301

Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS. Part 3: Discrete Uniform Distribution Binomial Distribution

Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS Part 3: Discrete Uniform Distribution Binomial Distribution Sections 3-5, 3-6 Special discrete random variable distributions we will cover

### Activities/ Resources for Unit V: Proportions, Ratios, Probability, Mean and Median

Activities/ Resources for Unit V: Proportions, Ratios, Probability, Mean and Median 58 What is a Ratio? A ratio is a comparison of two numbers. We generally separate the two numbers in the ratio with a

### EXAM. Exam #3. Math 1430, Spring 2002. April 21, 2001 ANSWERS

EXAM Exam #3 Math 1430, Spring 2002 April 21, 2001 ANSWERS i 60 pts. Problem 1. A city has two newspapers, the Gazette and the Journal. In a survey of 1, 200 residents, 500 read the Journal, 700 read the

### The Dirichlet-Multinomial and Dirichlet-Categorical models for Bayesian inference

The Dirichlet-Multinomial and Dirichlet-Categorical models for Bayesian inference Stephen Tu tu.stephenl@gmail.com 1 Introduction This document collects in one place various results for both the Dirichlet-multinomial

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Chapter 3 RANDOM VARIATE GENERATION

Chapter 3 RANDOM VARIATE GENERATION In order to do a Monte Carlo simulation either by hand or by computer, techniques must be developed for generating values of random variables having known distributions.

### Ch. 12.1: Permutations

Ch. 12.1: Permutations The Mathematics of Counting The field of mathematics concerned with counting is properly known as combinatorics. Whenever we ask a question such as how many different ways can we

### 3.4 Limits at Infinity - Asymptotes

3.4 Limits at Infinity - Asymptotes Definition 3.3. If f is a function defined on some interval (a, ), then f(x) = L means that values of f(x) are very close to L (keep getting closer to L) as x. The line

Some Tips for Using WebAssign in Calculus The problems you see on your WebAssign homework are generally questions taken from your textbook but sometimes randomized so that the numbers and functions may

### Chapter 5. Random variables

Random variables random variable numerical variable whose value is the outcome of some probabilistic experiment; we use uppercase letters, like X, to denote such a variable and lowercase letters, like

### Elements of probability theory

The role of probability theory in statistics We collect data so as to provide evidentiary support for answers we give to our many questions about the world (and in our particular case, about the business

### Examination 110 Probability and Statistics Examination

Examination 0 Probability and Statistics Examination Sample Examination Questions The Probability and Statistics Examination consists of 5 multiple-choice test questions. The test is a three-hour examination

### Statistical Inference

Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this

### Principle of Data Reduction

Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then

### Sampling and Hypothesis Testing

Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

### Lecture 19: Chapter 8, Section 1 Sampling Distributions: Proportions

Lecture 19: Chapter 8, Section 1 Sampling Distributions: Proportions Typical Inference Problem Definition of Sampling Distribution 3 Approaches to Understanding Sampling Dist. Applying 68-95-99.7 Rule

### Computational Statistics and Data Analysis

Computational Statistics and Data Analysis 53 (2008) 17 26 Contents lists available at ScienceDirect Computational Statistics and Data Analysis journal homepage: www.elsevier.com/locate/csda Coverage probability

### Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Be able to explain the difference between the p-value and a posterior

### 1. The algebra of exponents 1.1. Natural Number Powers. It is easy to say what is meant by a n a (raised to) to the (power) n if n N.

CHAPTER 3: EXPONENTS AND POWER FUNCTIONS 1. The algebra of exponents 1.1. Natural Number Powers. It is easy to say what is meant by a n a (raised to) to the (power) n if n N. For example: In general, if

### Chapter 4 Lecture Notes

Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,

### MATH 10: Elementary Statistics and Probability Chapter 5: Continuous Random Variables

MATH 10: Elementary Statistics and Probability Chapter 5: Continuous Random Variables Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of slides,

### Sampling Distribution of a Normal Variable

Ismor Fischer, 5/9/01 5.-1 5. Formal Statement and Examples Comments: Sampling Distribution of a Normal Variable Given a random variable. Suppose that the population distribution of is known to be normal,

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### Change of Continuous Random Variable

Change of Continuous Random Variable All you are responsible for from this lecture is how to implement the Engineer s Way (see page 4) to compute how the probability density function changes when we make

### Master s Theory Exam Spring 2006

Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

### Math 461 Fall 2006 Test 2 Solutions

Math 461 Fall 2006 Test 2 Solutions Total points: 100. Do all questions. Explain all answers. No notes, books, or electronic devices. 1. [105+5 points] Assume X Exponential(λ). Justify the following two

### THE MULTINOMIAL DISTRIBUTION. Throwing Dice and the Multinomial Distribution

THE MULTINOMIAL DISTRIBUTION Discrete distribution -- The Outcomes Are Discrete. A generalization of the binomial distribution from only 2 outcomes to k outcomes. Typical Multinomial Outcomes: red A area1

### Nonparametric adaptive age replacement with a one-cycle criterion

Nonparametric adaptive age replacement with a one-cycle criterion P. Coolen-Schrijner, F.P.A. Coolen Department of Mathematical Sciences University of Durham, Durham, DH1 3LE, UK e-mail: Pauline.Schrijner@durham.ac.uk

### Continuous Random Variables. and Probability Distributions. Continuous Random Variables and Probability Distributions ( ) ( ) Chapter 4 4.

UCLA STAT 11 A Applied Probability & Statistics for Engineers Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology Teaching Assistant: Neda Farzinnia, UCLA Statistics University of California,

### Linear Equations and Inequalities

Linear Equations and Inequalities Section 1.1 Prof. Wodarz Math 109 - Fall 2008 Contents 1 Linear Equations 2 1.1 Standard Form of a Linear Equation................ 2 1.2 Solving Linear Equations......................

### Hypothesis Testing Level I Quantitative Methods. IFT Notes for the CFA exam

Hypothesis Testing 2014 Level I Quantitative Methods IFT Notes for the CFA exam Contents 1. Introduction... 3 2. Hypothesis Testing... 3 3. Hypothesis Tests Concerning the Mean... 10 4. Hypothesis Tests

### Worked examples Basic Concepts of Probability Theory

Worked examples Basic Concepts of Probability Theory Example 1 A regular tetrahedron is a body that has four faces and, if is tossed, the probability that it lands on any face is 1/4. Suppose that one

### Margin of Error When Estimating a Population Proportion

Margin of Error When Estimating a Population Proportion Student Outcomes Students use data from a random sample to estimate a population proportion. Students calculate and interpret margin of error in

### The Normal Distribution

Chapter 6 The Normal Distribution 6.1 The Normal Distribution 1 6.1.1 Student Learning Objectives By the end of this chapter, the student should be able to: Recognize the normal probability distribution

Section 5.4 The Quadratic Formula 481 5.4 The Quadratic Formula Consider the general quadratic function f(x) = ax + bx + c. In the previous section, we learned that we can find the zeros of this function

### 1 Sufficient statistics

1 Sufficient statistics A statistic is a function T = rx 1, X 2,, X n of the random sample X 1, X 2,, X n. Examples are X n = 1 n s 2 = = X i, 1 n 1 the sample mean X i X n 2, the sample variance T 1 =

### m (t) = e nt m Y ( t) = e nt (pe t + q) n = (pe t e t + qe t ) n = (qe t + p) n

1. For a discrete random variable Y, prove that E[aY + b] = ae[y] + b and V(aY + b) = a 2 V(Y). Solution: E[aY + b] = E[aY] + E[b] = ae[y] + b where each step follows from a theorem on expected value from

### Preliminary Mathematics

Preliminary Mathematics The purpose of this document is to provide you with a refresher over some topics that will be essential for what we do in this class. We will begin with fractions, decimals, and

### For a partition B 1,..., B n, where B i B j = for i. A = (A B 1 ) (A B 2 ),..., (A B n ) and thus. P (A) = P (A B i ) = P (A B i )P (B i )

Probability Review 15.075 Cynthia Rudin A probability space, defined by Kolmogorov (1903-1987) consists of: A set of outcomes S, e.g., for the roll of a die, S = {1, 2, 3, 4, 5, 6}, 1 1 2 1 6 for the roll

### Basic Probability Theory I

A Probability puzzler!! Basic Probability Theory I Dr. Tom Ilvento FREC 408 Our Strategy with Probability Generally, we want to get to an inference from a sample to a population. In this case the population

### Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to

### 8.7 Exponential Growth and Decay

Section 8.7 Exponential Growth and Decay 847 8.7 Exponential Growth and Decay Exponential Growth Models Recalling the investigations in Section 8.3, we started by developing a formula for discrete compound

### MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions...

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo c Copyright 2004-2012 by Jenny A. Baglivo. All Rights Reserved. Contents 3 MT426 Notebook 3 3 3.1 Definitions............................................

### Foundations of Statistics Frequentist and Bayesian

Mary Parker, http://www.austincc.edu/mparker/stat/nov04/ page 1 of 13 Foundations of Statistics Frequentist and Bayesian Statistics is the science of information gathering, especially when the information

### Continuous Functions, Smooth Functions and the Derivative

UCSC AMS/ECON 11A Supplemental Notes # 4 Continuous Functions, Smooth Functions and the Derivative c 2004 Yonatan Katznelson 1. Continuous functions One of the things that economists like to do with mathematical

### Chapter 3 Random Variables and Probability Distributions

Math 322 Probabilit and Statistical Methods Chapter 3 Random Variables and Probabilit Distributions In statistics we deal with random variables- variables whose observed value is determined b chance. Random

### A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution

A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September

### Recursive Estimation

Recursive Estimation Raffaello D Andrea Spring 04 Problem Set : Bayes Theorem and Bayesian Tracking Last updated: March 8, 05 Notes: Notation: Unlessotherwisenoted,x, y,andz denoterandomvariables, f x

### Section 6.2 Definition of Probability

Section 6.2 Definition of Probability Probability is a measure of the likelihood that an event occurs. For example, if there is a 20% chance of rain tomorrow, that means that the probability that it will

### Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma

Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma Shinto Eguchi a, and John Copas b a Institute of Statistical Mathematics and Graduate University of Advanced Studies, Minami-azabu

### Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### CHAPTER 6: Continuous Uniform Distribution: 6.1. Definition: The density function of the continuous random variable X on the interval [A, B] is.

Some Continuous Probability Distributions CHAPTER 6: Continuous Uniform Distribution: 6. Definition: The density function of the continuous random variable X on the interval [A, B] is B A A x B f(x; A,

### Slide 1 Math 1520, Lecture 23. This lecture covers mean, median, mode, odds, and expected value.

Slide 1 Math 1520, Lecture 23 This lecture covers mean, median, mode, odds, and expected value. Slide 2 Mean, Median and Mode Mean, Median and mode are 3 concepts used to get a sense of the central tendencies

### Contents. Introduction and Notes pages 2-3 (These are important and it s only 2 pages ~ please take the time to read them!)

Page Contents Introduction and Notes pages 2-3 (These are important and it s only 2 pages ~ please take the time to read them!) Systematic Search for a Change of Sign (Decimal Search) Method Explanation

### To give it a definition, an implicit function of x and y is simply any relationship that takes the form:

2 Implicit function theorems and applications 21 Implicit functions The implicit function theorem is one of the most useful single tools you ll meet this year After a while, it will be second nature to

### Jan 17 Homework Solutions Math 151, Winter 2012. Chapter 2 Problems (pages 50-54)

Jan 17 Homework Solutions Math 11, Winter 01 Chapter Problems (pages 0- Problem In an experiment, a die is rolled continually until a 6 appears, at which point the experiment stops. What is the sample

### REPEATED TRIALS. The probability of winning those k chosen times and losing the other times is then p k q n k.

REPEATED TRIALS Suppose you toss a fair coin one time. Let E be the event that the coin lands heads. We know from basic counting that p(e) = 1 since n(e) = 1 and 2 n(s) = 2. Now suppose we play a game

### Bayes and Naïve Bayes. cs534-machine Learning

Bayes and aïve Bayes cs534-machine Learning Bayes Classifier Generative model learns Prediction is made by and where This is often referred to as the Bayes Classifier, because of the use of the Bayes rule

### Bayesian probability: P. State of the World: X. P(X your information I)

Bayesian probability: P State of the World: X P(X your information I) 1 First example: bag of balls Every probability is conditional to your background knowledge I : P(A I) What is the (your) probability

### Jan 31 Homework Solutions Math 151, Winter 2012. Chapter 3 Problems (pages 102-110)

Jan 31 Homework Solutions Math 151, Winter 01 Chapter 3 Problems (pages 10-110) Problem 61 Genes relating to albinism are denoted by A and a. Only those people who receive the a gene from both parents

### STT315 Chapter 4 Random Variables & Probability Distributions KM. Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables

Chapter 4.5, 6, 8 Probability Distributions for Continuous Random Variables Discrete vs. continuous random variables Examples of continuous distributions o Uniform o Exponential o Normal Recall: A random

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Math 370, Spring 2008 Prof. A.J. Hildebrand. Practice Test 1 Solutions

Math 70, Spring 008 Prof. A.J. Hildebrand Practice Test Solutions About this test. This is a practice test made up of a random collection of 5 problems from past Course /P actuarial exams. Most of the

### Decimal Notations for Fractions Number and Operations Fractions /4.NF

Decimal Notations for Fractions Number and Operations Fractions /4.NF Domain: Cluster: Standard: 4.NF Number and Operations Fractions Understand decimal notation for fractions, and compare decimal fractions.

### #1 Make sense of problems and persevere in solving them.

#1 Make sense of problems and persevere in solving them. 1. Make sense of problems and persevere in solving them. Interpret and make meaning of the problem looking for starting points. Analyze what is

### Likelihood: Frequentist vs Bayesian Reasoning

"PRINCIPLES OF PHYLOGENETICS: ECOLOGY AND EVOLUTION" Integrative Biology 200B University of California, Berkeley Spring 2009 N Hallinan Likelihood: Frequentist vs Bayesian Reasoning Stochastic odels and

### Two-sample inference: Continuous data

Two-sample inference: Continuous data Patrick Breheny April 5 Patrick Breheny STA 580: Biostatistics I 1/32 Introduction Our next two lectures will deal with two-sample inference for continuous data As

### Taylor Polynomials and Taylor Series Math 126

Taylor Polynomials and Taylor Series Math 26 In many problems in science and engineering we have a function f(x) which is too complicated to answer the questions we d like to ask. In this chapter, we will