# Bayesian Analysis for the Social Sciences

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Bayesian Analysis for the Social Sciences Simon Jackman Stanford University November 9, 2012 Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

2 Introduction to Bayesian Inference Bayesian inference relies exclusively on Bayes Theorem: p(h data) p(h) p(data h) h is a usually a parameter (but could also be a data point, a model, a hypothesis) p are probability densities (or probability mass functions in the case of discrete h and/or discrete data) p(h) a prior density; p(data h) the likelihood or conditional density of the data given h p(h data) is the posterior density for h given the data. Gives rise to the Bayesian mantra: a posterior density is proportional to the prior times the likelihood Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

3 Probability Densities as Representations of Beliefs Definition (Probability Density Function (informal)) Let h be a unknown quantity, h H R. A function p(h) is a proper probability density function if 1 p(h) 0 h. 2 p(h)dh = 1. H N(0,1) N(0,2) θ θ Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

4 Probability Densities as Representations of Beliefs Definition (Probability Density Function (informal)) Let h be a unknown quantity, h H R. A function p(h) is a proper probability density function if 1 p(h) 0 h. 2 p(h)dh = 1. H Unif(0,1) Beta(2,3) p(θ) 1.0 p(θ) θ θ Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

5 Probability Mass Function Definition (Probability Mass Function) If h is a discrete random variable, taking values in a countable space H R, then a function p : H [0, 1] is a probability mass function if 1 p(h) = 0 h R \ H 2 h H p(h) = trinomial 1.0 Binomial(2/3,5) 1.0 Poisson(4) red green blue θ θ θ Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

6 Introduction to Bayesian Inference p(h data) p(h) p(data h) Bayesian inference involves computing, summarizing and communicating summaries of the posterior density p(h data). How to do this is what this class is about. Depending on the problem, doing all this is easy or hard; we solve hard with computing power. We re working with densities (or sometimes, mass functions). Bayesian point estimates are a single number summary of a posterior density Uncertainty assessed/communicated in various ways: e.g., the standard deviation of the posterior, width of interval spanning 2.5th to 97.5th percentiles of the posterior, etc. Sometimes, can just draw a picture; details, examples coming. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

7 Introduction to Bayesian Inference p(h data) p(h) p(data h) Bayes Theorem tells us how to update beliefs about h in light of evidence ( data ) a general method for induction or for learning from data : prior - data - posterior Bayes Theorem is itself uncontroversial: follows from widely accepted axioms of probability theory (e.g., Kolmogorov) and the definition of conditional probability Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

8 Why Be Bayesian? conceptual simplicity: say what you mean and mean what you say (subjective probability) a foundation for inference that does not rest on the thought experiment of repeated sampling uniformity of application: no special tweeks for this or that data analysis. Apply Bayes Rule. modern computing makes Bayesian inference easy and nearly universally applicable Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

9 Conceptual Simplicity p(h data) p(h) p(data h) the posterior density (or mass function) p(h data) is a complete characterization of beliefs after looking at data as such it contains everything we need for making inferences Examples: the posterior probability that a regression coefficient is positive, negative or lies in a particular interval; the posterior probability that a subject belongs to a particular latent class; the posterior probability that a hypothesis is true; or, the posterior probabilities that a particular statistical model is true model among a family of statistical models. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

10 Contrast Frequentist Inference Model for data: y f (h). Estimate h: e.g., least squares, MLE, etc, to yield h h(y) null hypothesis e.g., H 0 : h H0 = 0 Inference via the sampling distribution of h conditional on H 0 : e.g., assuming H 0, over repeated applications of the sampling process, how frequently would we observe a result at least as extreme as the one we obtained? At least as extreme? Assessed via a test statistic, e.g., t(y) = (h H0 - h)/ var(h h = h H0 ) how frequently? The p-value, relative frequency with which we see t > t(y) in repeated applications of the sampling process. Often t(y) d N(0, 1). Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

11 Contrast Frequentist Inference null hypothesis e.g., H 0 : h H0 = 0 test-statistic: Often t(y) d N(0, 1). t(y) = (h H0 - h)/ var(h h = h H0 ) p-value is a statement about the plausibility of the statistic h relative to what we might have observed in random sampling assuming H 0 : h H0 = 0 one more step need to reject/fail-to-reject H 0. Is p sufficiently small? frequentist p-value is a summary of the distribution of h under H 0 Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

12 Contrast Frequentist Inference n.b., frequentist inference treats h as a random variable h is a fixed but unknown feature of the population from which data is being (randomly) sampled Bayesian inference: h is fixed, a function of the data available for analysis Bayesian inference: h is a random variable, subject to (subjective) uncertainty Bayesian Frequentist h random fixed but unknown h fixed random random-ness subjective sampling distribution of interest posterior sampling distribution p(h y) p(h(y) h = h H0 ) Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

13 Subjective Uncertainty how do we do statistical inference in situations where repeated sampling is infeasible? inference when we have the entire population and hence no uncertainty due to sampling: e.g., parts of comparative political economy. Bayesians rely on a notion of subjective uncertainty e.g., h is a random variable because we don t know its value Bayes Theorem tells us how to manage that uncertainty, how to update beliefs about h in light of data Contrast objectivist notion of probability: probability as a property of the object under study (e.g., coins, decks of cards, roulette wheels, people, groups, societies). Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

14 Subjective Uncertainty Many Bayesians regard objectivist probability as metaphysical nonsense. de Finetti: PROBABILITY DOES NOT EXIST The abandonment of superstitious beliefs about...fairies and Witches was an essential step along the road to scientific thinking. Probability, too, if regarded as something endowed with some kind of objective existence, is not less a misleading misconception, an illusory attempt to exteriorize or materialize our true probabilistic beliefs. In investigating the reasonableness of our own modes of thought and behaviour under uncertainty, all we require, and all that we are reasonably entitled to, is consistency among these beliefs, and their reasonable relation to any kind of relevant objective data ( relevant in as much as subjectively deemed to be so). This is Probability Theory. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

15 Subjective Uncertainty Bayesian probability statements are thus about states of mind over states of the world, and not about states of the world per sé. Borel: one can guess the outcome of a coin toss while the coin is still in the air and its movement is perfectly determined, or even after the coin has landed but before one reviews the result. i.e., subjective uncertainty obtains irrespective of objective uncertainty (however conceived) not just any subjective uncertainty: beliefs must conform to the rules of probability: e.g., p(h) should be proper: i.e., H p(h)dh = 1, p(h) 0 h H. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

16 Bayes Theorem Conditional probability: Let A and B be events with P(B) > 0. Then the conditional probability of A given B is P(A B) = P(A B) P(B) = P(A, B) P(B). Multiplication rule: P(A B) = P(A, B) = P(A B)P(B) = P(B A)P(A) Law of Total Probability: P(B) = P(A B) + P( A B) = P(B A)P(A) + P(B A)P( A) Bayes Theorem: If A and B are events with P(B) > 0, then P(A B) = P(B A)P(A) P(B) Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

17 Bayes Theorem, Example case, drug-testing work suggests that about 3% of the subject pool (elite athletes) uses a particular prohibited drug. H U : test subject uses the prohibited substance. p(h U ) =.03. E (evidence) is a positive test result. Test has a false negative rate of.05; i.e., P( E H U ) =.05 P(E H U ) =.95. Test has a false positive rate of.10: i.e., P(E H U ) =.10. Bayes Theorem: P(H U E) = P(H U )P(E H U ) i {U, U} P(H i)p(e H i ) =.03~.95 (.03~.95) + (.97~.10) = =.23 Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

18 Bayes Theorem, Continuous Parameter Bayes Theorem: p(h y) = p(y h)p(h) p(y h)p(h)dh Proof: by the definition of conditional probability p(h, y) = p(h y)p(y) = p(y h)p(h), (1) where all these densities are assumed to exist and have the properties p(z) > 0 and p(z)dz = 1 (i.e., are proper probability densities. The result follows by re-arranging the quantities in equation equation 1 and noting that p(y) = p(y, h)dh = p(y h)p(h)dh. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

19 and Densities, Continuous Parameter Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

20 , and s: less standard cases θ θ θ θ θ θ Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

21 Cromwell s Rule: the dangers of dogmatism p(h data) p(h) p(data h) p(h data) = 0 hs.t.p(h) = 0. Cromwell s Rule: After the English deposed, tried and executed Charles I in 1649, the Scots invited Charles son, Charles II, to become king. The English regarded this as a hostile act, and Oliver Cromwell led an army north. to the outbreak of hostilities, Cromwell wrote to the synod of the Church of Scotland, I beseech you, in the bowels of Christ, consider it possible that you are mistaken. a dogmatic prior that assigns zero probability to a hypothesis can never be revised likewise, a hypothesis with prior weight of 1.0 can never be refuted. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

22 Cromwell s Rule Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

23 Bayesian Point Estimates Bayes estimates: single number summary of a posterior density but which one?: e.g., mode, median, mean, some quantile(s)? different loss functions rationalize different point estimate Loss: Let H be a set of possible states of nature h, and let a A be actions availble to the researcher. Then define l(h, a) as the loss to the researcher from taking action a when the state of nature is h. expected loss: Given a posterior distribution for h, p(h y), the posterior expected loss of an action a is m(p(h y), a) = H l(h, a)p(h y)dh. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

24 Mean as Bayes Estimator Under Quadratic Loss quadratic loss: If h H is a parameter of interest, and ~ h is an estimate of h, then l(h, ~ h) = (h - ~ h) 2 is the quadratic loss arising from the use of the estimate ~ h instead of h. Mean as Bayes Estimate Under Quadratic Loss: E(h y) = ~ h = H h p(h y)dh. Proof: Quadratic loss implies that the posterior expected loss is m(h, ~ h) = (h - ~ h) 2 p(h y)dh. Expanding the quadratic yields H m(h, ~ h) = H h2 p(h y)dh + ~ h 2-2 ~ he(h y). Differentiate with respect to ~ h, noting that the first term does not involve ~ h. Solve the 1st order condition for ~ h and the result follows. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

25 Bayes Estimates Quadratic Loss: mean of the posterior density, E(h y) = h p(h y)dh Symmetric Linear Loss: median of the posterior density, n.b., only well-defined for h H R, in which case ~ h is defined such that.5 All-or-nothing Loss: mode of the posterior density H = ~ h = argmax p(h y) h H Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

26 Credible Region; HPD region Definition (Credible Region) A region C X such that p(h)dh = 1 - α, 0 α 1 is a 100(1 - α)% C credible region for h. For single-parameter problems (i.e., X R), if C is not a set of disjoint intervals, then C is a credible interval. If p(h) is a (prior/posterior) density, then C is a (prior/posterior) credible region. Definition (Highest Probability Density Region) A region C X is a 100(1 - α)% highest probability density region for h under p(h) if 1 P(h C) = 1 - α 2 P(h 1 ) P(h 2 ), h 1 C, h 2 C Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

27 HPD intervals A 100(1 - α)% HPD region for a symmetric, unimodal density is unique and symmetric around the mode; e.g., a normal density. Cf skewed distributions; a HPD differs from simply reading off the quantiles. N(0,1) χ 2 4 df 25% 75% Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

28 HPD intervals HPDs can be a series of disjoint intervals, e.g., a bimodal density these are uncommon; but in such a circumstance, presenting a picture of the density might be the reasonable thing to do. See Example 1.7, p28: y i N(0, R), subject to extreme missingness. The posterior density of q(r) = r 12 / r 11 r 22 : Correlation Coefficient Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

29 Bayesian Consistency for anything other than a dogmatic/degenerate prior (see the earlier discussion of Cromwell s Rule), more and more data will overwhelm the prior. Bayesian asymptotics: with an arbitrarily large amount of sample information relative to prior information, the posterior density tends to the likelihood (normalized to be a density over h). central limit arguments: since likelihoods are usually approximately normal in large samples, then so too are posterior densities. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

30 Bayesian Consistency The prior remains fixed across the sequence, as sample size increases and h * is held constant. In this example, n = 6, 30, 90, 450 across the four columns. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

31 Bayesian Consistency The prior remains fixed across the sequence, as sample size increases and h * is held constant. In this example, n = 6, 30, 150, 1500 across the four columns. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

32 Other topics from Chapter One 1.8. Bayesian hypothesis testing Exchangeability. de Finetti s Representation Theorem. Simon Jackman (Stanford) Bayesian Analysis for the Social Sciences November 9, / 32

### 15.0 More Hypothesis Testing

15.0 More Hypothesis Testing 1 Answer Questions Type I and Type II Error Power Calculation Bayesian Hypothesis Testing 15.1 Type I and Type II Error In the philosophy of hypothesis testing, the null hypothesis

### Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1

Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

### MS&E 226: Small Data

MS&E 226: Small Data Lecture 16: Bayesian inference (v3) Ramesh Johari ramesh.johari@stanford.edu 1 / 35 Priors 2 / 35 Frequentist vs. Bayesian inference Frequentists treat the parameters as fixed (deterministic).

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### Bayesian Statistics in One Hour. Patrick Lam

Bayesian Statistics in One Hour Patrick Lam Outline Introduction Bayesian Models Applications Missing Data Hierarchical Models Outline Introduction Bayesian Models Applications Missing Data Hierarchical

### People have thought about, and defined, probability in different ways. important to note the consequences of the definition:

PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models

### Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010

Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010 Week 1 Week 2 14.0 Students organize and describe distributions of data by using a number of different

### Part 2: One-parameter models

Part 2: One-parameter models Bernoilli/binomial models Return to iid Y 1,...,Y n Bin(1, θ). The sampling model/likelihood is p(y 1,...,y n θ) =θ P y i (1 θ) n P y i When combined with a prior p(θ), Bayes

### Sampling and Hypothesis Testing

Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

### Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals Bayesian Updating with Discrete Priors Class 11, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1. Be able to apply Bayes theorem to compute probabilities. 2. Be able to identify

### 1 Prior Probability and Posterior Probability

Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which

### Senior Secondary Australian Curriculum

Senior Secondary Australian Curriculum Mathematical Methods Glossary Unit 1 Functions and graphs Asymptote A line is an asymptote to a curve if the distance between the line and the curve approaches zero

### Hypothesis Testing. 1 Introduction. 2 Hypotheses. 2.1 Null and Alternative Hypotheses. 2.2 Simple vs. Composite. 2.3 One-Sided and Two-Sided Tests

Hypothesis Testing 1 Introduction This document is a simple tutorial on hypothesis testing. It presents the basic concepts and definitions as well as some frequently asked questions associated with hypothesis

### Fairfield Public Schools

Mathematics Fairfield Public Schools AP Statistics AP Statistics BOE Approved 04/08/2014 1 AP STATISTICS Critical Areas of Focus AP Statistics is a rigorous course that offers advanced students an opportunity

### Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Comparison of frequentist and Bayesian inference. Class 20, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Be able to explain the difference between the p-value and a posterior

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

The result of the bayesian analysis is the probability distribution of every possible hypothesis H, given one real data set D. This prestatistical approach to our problem was the standard approach of Laplace

### An Introduction to Bayesian Statistics

An Introduction to Bayesian Statistics Robert Weiss Department of Biostatistics UCLA School of Public Health robweiss@ucla.edu April 2011 Robert Weiss (UCLA) An Introduction to Bayesian Statistics UCLA

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### Lecture 9: Bayesian hypothesis testing

Lecture 9: Bayesian hypothesis testing 5 November 27 In this lecture we ll learn about Bayesian hypothesis testing. 1 Introduction to Bayesian hypothesis testing Before we go into the details of Bayesian

### Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.5 The t Test 9.6

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

### Elements of statistics (MATH0487-1)

Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium December 10, 2012 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis -

### Bayes and Naïve Bayes. cs534-machine Learning

Bayes and aïve Bayes cs534-machine Learning Bayes Classifier Generative model learns Prediction is made by and where This is often referred to as the Bayes Classifier, because of the use of the Bayes rule

### INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

### Statistics in Geophysics: Introduction and Probability Theory

Statistics in Geophysics: Introduction and Steffen Unkel Department of Statistics Ludwig-Maximilians-University Munich, Germany Winter Term 2013/14 1/32 What is Statistics? Introduction Statistics is the

### Inference of Probability Distributions for Trust and Security applications

Inference of Probability Distributions for Trust and Security applications Vladimiro Sassone Based on joint work with Mogens Nielsen & Catuscia Palamidessi Outline 2 Outline Motivations 2 Outline Motivations

### Variance of OLS Estimators and Hypothesis Testing. Randomness in the model. GM assumptions. Notes. Notes. Notes. Charlie Gibbons ARE 212.

Variance of OLS Estimators and Hypothesis Testing Charlie Gibbons ARE 212 Spring 2011 Randomness in the model Considering the model what is random? Y = X β + ɛ, β is a parameter and not random, X may be

EC151.02 Statistics for Business and Economics (MWF 8:00-8:50) Instructor: Chiu Yu Ko Office: 462D, 21 Campenalla Way Phone: 2-6093 Email: kocb@bc.edu Office Hours: by appointment Description This course

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### Bayesian Methods. 1 The Joint Posterior Distribution

Bayesian Methods Every variable in a linear model is a random variable derived from a distribution function. A fixed factor becomes a random variable with possibly a uniform distribution going from a lower

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson Contents What Are These Demos About? How to Use These Demos If This Is Your First Time Using Fathom Tutorial: An Extended Example

### The Big 50 Revision Guidelines for S1

The Big 50 Revision Guidelines for S1 If you can understand all of these you ll do very well 1. Know what is meant by a statistical model and the Modelling cycle of continuous refinement 2. Understand

### Probability and Statistics Vocabulary List (Definitions for Middle School Teachers)

Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence

### 2DI36 Statistics. 2DI36 Part II (Chapter 7 of MR)

2DI36 Statistics 2DI36 Part II (Chapter 7 of MR) What Have we Done so Far? Last time we introduced the concept of a dataset and seen how we can represent it in various ways But, how did this dataset came

### Tutorial 5: Hypothesis Testing

Tutorial 5: Hypothesis Testing Rob Nicholls nicholls@mrc-lmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction................................ 1 2 Testing distributional assumptions....................

### Bayesian Statistical Methods. David Kaplan Sarah Depaoli Department of Educational Psychology University of Wisconsin Madison

Bayesian Statistical Methods 1 Running head: BAYESIAN STATISTICAL METHODS Bayesian Statistical Methods David Kaplan Sarah Depaoli Department of Educational Psychology University of Wisconsin Madison To

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab

Lecture 2 Probability BMIR Lecture Series in Probability and Statistics Ching-Han Hsu, BMES, National Tsing Hua University c 2014 by Ching-Han Hsu, Ph.D., BMIR Lab 2.1 1 Sample Spaces and Events Random

### Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett

Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.

### Posterior probability!

Posterior probability! P(x θ): old name direct probability It gives the probability of contingent events (i.e. observed data) for a given hypothesis (i.e. a model with known parameters θ) L(θ)=P(x θ):

### Continuous Random Variables

Continuous Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Continuous Random Variables 2 11 Introduction 2 12 Probability Density Functions 3 13 Transformations 5 2 Mean, Variance and Quantiles

### Exploratory Data Analysis

Exploratory Data Analysis Johannes Schauer johannes.schauer@tugraz.at Institute of Statistics Graz University of Technology Steyrergasse 17/IV, 8010 Graz www.statistics.tugraz.at February 12, 2008 Introduction

### MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

### Persuasion by Cheap Talk - Online Appendix

Persuasion by Cheap Talk - Online Appendix By ARCHISHMAN CHAKRABORTY AND RICK HARBAUGH Online appendix to Persuasion by Cheap Talk, American Economic Review Our results in the main text concern the case

### 2. Information Economics

2. Information Economics In General Equilibrium Theory all agents had full information regarding any variable of interest (prices, commodities, state of nature, cost function, preferences, etc.) In many

### What is the purpose of this document? What is in the document? How do I send Feedback?

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Statistics

### What is Bayesian statistics and why everything else is wrong

What is Bayesian statistics and why everything else is wrong 1 Michael Lavine ISDS, Duke University, Durham, North Carolina Abstract We use a single example to explain (1), the Likelihood Principle, (2)

There are three kinds of people in the world those who are good at math and those who are not. PSY 511: Advanced Statistics for Psychological and Behavioral Research 1 Positive Views The record of a month

### Discrete Mathematics & Mathematical Reasoning Chapter 7 (section 7.3): Conditional Probability & Bayes Theorem

Discrete Mathematics & Mathematical Reasoning Chapter 7 (section 7.3): Conditional Probability & Bayes Theorem Kousha Etessami U. of Edinburgh, UK Kousha Etessami (U. of Edinburgh, UK) Discrete Mathematics

### Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics.

Business Course Text Bowerman, Bruce L., Richard T. O'Connell, J. B. Orris, and Dawn C. Porter. Essentials of Business, 2nd edition, McGraw-Hill/Irwin, 2008, ISBN: 978-0-07-331988-9. Required Computing

### A Few Basics of Probability

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

### Economic Order Quantity and Economic Production Quantity Models for Inventory Management

Economic Order Quantity and Economic Production Quantity Models for Inventory Management Inventory control is concerned with minimizing the total cost of inventory. In the U.K. the term often used is stock

### Final Mathematics 5010, Section 1, Fall 2004 Instructor: D.A. Levin

Final Mathematics 51, Section 1, Fall 24 Instructor: D.A. Levin Name YOU MUST SHOW YOUR WORK TO RECEIVE CREDIT. A CORRECT ANSWER WITHOUT SHOWING YOUR REASONING WILL NOT RECEIVE CREDIT. Problem Points Possible

### 38. STATISTICS. 38. Statistics 1. 38.1. Fundamental concepts

38. Statistics 1 38. STATISTICS Revised September 2013 by G. Cowan (RHUL). This chapter gives an overview of statistical methods used in high-energy physics. In statistics, we are interested in using a

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Lecture 3 : Hypothesis testing and model-fitting

Lecture 3 : Hypothesis testing and model-fitting These dark lectures energy puzzle Lecture 1 : basic descriptive statistics Lecture 2 : searching for correlations Lecture 3 : hypothesis testing and model-fitting

### Lecture 2: Statistical Estimation and Testing

Bioinformatics: In-depth PROBABILITY AND STATISTICS Spring Semester 2012 Lecture 2: Statistical Estimation and Testing Stefanie Muff (stefanie.muff@ifspm.uzh.ch) 1 Problems in statistics 2 The three main

### Using pivots to construct confidence intervals. In Example 41 we used the fact that

Using pivots to construct confidence intervals In Example 41 we used the fact that Q( X, µ) = X µ σ/ n N(0, 1) for all µ. We then said Q( X, µ) z α/2 with probability 1 α, and converted this into a statement

### AP Statistics 1998 Scoring Guidelines

AP Statistics 1998 Scoring Guidelines These materials are intended for non-commercial use by AP teachers for course and exam preparation; permission for any other use must be sought from the Advanced Placement

### Null Hypothesis Significance Testing: a short tutorial

Null Hypothesis Significance Testing: a short tutorial Although thoroughly criticized, null hypothesis significance testing is the statistical method of choice in biological, biomedical and social sciences

### Final Exam Practice Problem Answers

Final Exam Practice Problem Answers The following data set consists of data gathered from 77 popular breakfast cereals. The variables in the data set are as follows: Brand: The brand name of the cereal

### Some Essential Statistics The Lure of Statistics

Some Essential Statistics The Lure of Statistics Data Mining Techniques, by M.J.A. Berry and G.S Linoff, 2004 Statistics vs. Data Mining..lie, damn lie, and statistics mining data to support preconceived

### 3. Data Analysis, Statistics, and Probability

3. Data Analysis, Statistics, and Probability Data and probability sense provides students with tools to understand information and uncertainty. Students ask questions and gather and use data to answer

### CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression

Opening Example CHAPTER 13 SIMPLE LINEAR REGREION SIMPLE LINEAR REGREION! Simple Regression! Linear Regression Simple Regression Definition A regression model is a mathematical equation that descries the

### Basic Bayesian Methods

6 Basic Bayesian Methods Mark E. Glickman and David A. van Dyk Summary In this chapter, we introduce the basics of Bayesian data analysis. The key ingredients to a Bayesian analysis are the likelihood

### An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics

Slide 1 An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Dr. Christian Asseburg Centre for Health Economics Part 1 Slide 2 Talk overview Foundations of Bayesian statistics

### MAS2317/3317. Introduction to Bayesian Statistics. More revision material

MAS2317/3317 Introduction to Bayesian Statistics More revision material Dr. Lee Fawcett, 2014 2015 1 Section A style questions 1. Describe briefly the frequency, classical and Bayesian interpretations

### Generalized Linear Models

Generalized Linear Models We have previously worked with regression models where the response variable is quantitative and normally distributed. Now we turn our attention to two types of models where the

### More details on the inputs, functionality, and output can be found below.

Overview: The SMEEACT (Software for More Efficient, Ethical, and Affordable Clinical Trials) web interface (http://research.mdacc.tmc.edu/smeeactweb) implements a single analysis of a two-armed trial comparing

### Hypothesis Testing COMP 245 STATISTICS. Dr N A Heard. 1 Hypothesis Testing 2 1.1 Introduction... 2 1.2 Error Rates and Power of a Test...

Hypothesis Testing COMP 45 STATISTICS Dr N A Heard Contents 1 Hypothesis Testing 1.1 Introduction........................................ 1. Error Rates and Power of a Test.............................

### Homework 3. Part 1. Name: Score: / null

Name: Score: / Homework 3 Part 1 null 1 For the following sample of scores, the standard deviation is. Scores: 7, 2, 4, 6, 4, 7, 3, 7 Answer Key: 2 2 For any set of data, the sum of the deviation scores

### Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2015 CS 551, Fall 2015

### 4. Continuous Random Variables, the Pareto and Normal Distributions

4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random

### Review of Random Variables

Chapter 1 Review of Random Variables Updated: January 16, 2015 This chapter reviews basic probability concepts that are necessary for the modeling and statistical analysis of financial data. 1.1 Random

### Bayes Rule and Bayesian Probability

Eric F. Lock UMN Division of Biostatistics, SPH elock@umn.edu 01/19/2014 Basic probability Think of Probability as a function that assigns a real number to an event For events E and F in probability space

### Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

### STA 371G: Statistics and Modeling

STA 371G: Statistics and Modeling Decision Making Under Uncertainty: Probability, Betting Odds and Bayes Theorem Mingyuan Zhou McCombs School of Business The University of Texas at Austin http://mingyuanzhou.github.io/sta371g

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

### E205 Final: Version B

Name: Class: Date: E205 Final: Version B Multiple Choice Identify the choice that best completes the statement or answers the question. 1. The owner of a local nightclub has recently surveyed a random

### Prentice Hall Algebra 2 2011 Correlated to: Colorado P-12 Academic Standards for High School Mathematics, Adopted 12/2009

Content Area: Mathematics Grade Level Expectations: High School Standard: Number Sense, Properties, and Operations Understand the structure and properties of our number system. At their most basic level

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 1 9/3/2008 PROBABILISTIC MODELS AND PROBABILITY MEASURES Contents 1. Probabilistic experiments 2. Sample space 3. Discrete probability

### Machine Learning: Fundamentals

Machine Learning: Fundamentals A model is a formal description of a belief about the world. Learning is the construction and/or revision of a model in response to observations of the world. The mathematical/statistical

### Chapter 2: Data quantifiers: sample mean, sample variance, sample standard deviation Quartiles, percentiles, median, interquartile range Dot diagrams

Review for Final Chapter 2: Data quantifiers: sample mean, sample variance, sample standard deviation Quartiles, percentiles, median, interquartile range Dot diagrams Histogram Boxplots Chapter 3: Set

### International College of Economics and Finance Syllabus Probability Theory and Introductory Statistics

International College of Economics and Finance Syllabus Probability Theory and Introductory Statistics Lecturer: Mikhail Zhitlukhin. 1. Course description Probability Theory and Introductory Statistics

### Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Pearson Education, Inc.

2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Theorem 8: Let A be a square matrix. Then the following statements are equivalent. That is, for a given A, the statements are either all true

### Statistics I for QBIC. Contents and Objectives. Chapters 1 7. Revised: August 2013

Statistics I for QBIC Text Book: Biostatistics, 10 th edition, by Daniel & Cross Contents and Objectives Chapters 1 7 Revised: August 2013 Chapter 1: Nature of Statistics (sections 1.1-1.6) Objectives

### m (t) = e nt m Y ( t) = e nt (pe t + q) n = (pe t e t + qe t ) n = (qe t + p) n

1. For a discrete random variable Y, prove that E[aY + b] = ae[y] + b and V(aY + b) = a 2 V(Y). Solution: E[aY + b] = E[aY] + E[b] = ae[y] + b where each step follows from a theorem on expected value from

### HYPOTHESIS TESTING (ONE SAMPLE) - CHAPTER 7 1. used confidence intervals to answer questions such as...

HYPOTHESIS TESTING (ONE SAMPLE) - CHAPTER 7 1 PREVIOUSLY used confidence intervals to answer questions such as... You know that 0.25% of women have red/green color blindness. You conduct a study of men

### MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.

Open book and note Calculator OK Multiple Choice 1 point each MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Find the mean for the given sample data.