CSC 412/2506: Probabilistic Learning and Reasoning
|
|
- Moris Wheeler
- 1 years ago
- Views:
Transcription
1 CSC 412/2506: Probabilistic Learning and Reasoning Week 5-2/2: Sampling II Murat A. Erdogdu University of Toronto Prob Learning (UofT) CSC412-Week 5-2/2 1 / 21
2 Overview Gibbs sampling Hamiltonian Monte Carlo MCMC diagnostics Prob Learning (UofT) CSC412-Week 5-2/2 2 / 21
3 Gibbs Sampling Suppose the parameter vector x has been divided into d components x = (x 1,..., x d ) T At each iteration, the Gibbs Sampler, cycles through the components of x, drawing each subset conditional on the value of all others. This means we perform d steps at each sampling iteration t to obtain x (t+1) No accept/reject, only accept. Prob Learning (UofT) CSC412-Week 5-2/2 3 / 21
4 Gibbs Sampling Procedure At iteration t: choose an ordering j of d sub-vectors of x For j = 1 to j = d: Sample x t j from the conditional distribution given all the other components: x t j p(x j x t 1 j ) Where x t 1 j represents all the components of x except for x j at their current values: x t 1 j = (x t 1, x t 2,..., x t j 1, x t 1 j+1,..., xt 1 d ) Prob Learning (UofT) CSC412-Week 5-2/2 4 / 21
5 Gibbs Sampling Example Consider a single observation (y 1, y 2 ) from a bivariate normal, with unknown [ ] mean µ = (µ 1, µ 2 ) and known covariance matrix: 1 ρ Σ = with a standard Gaussian prior distribution on µ ρ 1 The posterior takes the form: ( ) (( ) ) µ1 y1 y N, Σ µ 2 Although it is simple to draw from this posterior we can alternatively use the Gibbs sampler. To do that we must first deterimine the conditional posterior distributions for µ 1 and µ 2 y 2 Prob Learning (UofT) CSC412-Week 5-2/2 5 / 21
6 Gibbs Sampling Example Using the properties of the multivariate normal distribution we have: µ 1 µ 2, y N(y 1 + ρ(µ 2 y 2 ), 1 ρ 2 ) µ 2 µ 1, y N(y 2 + ρ(µ 1 y 1 ), 1 ρ 2 ) Then given some previous (possibly initial) value of µ (t 1), the sampling would be: µ (t) 1 N(y 1 + ρ(µ (t 1) 2 y 2 ), 1 ρ 2 ) µ (t) 2 N(y 2 + ρ(µ (t) 1 y 1), 1 ρ 2 ) Prob Learning (UofT) CSC412-Week 5-2/2 6 / 21
7 Gibbs Sampling Example 1 1 From Bayesian Data Analysis Third edition by Gelman, Carlin, Stern, Dunson, Vehtari, Rubin Prob Learning (UofT) CSC412-Week 5-2/2 7 / 21
8 Hamiltonian Monte Carlo This is essentially a Metropolis-Hastings algorithm with a specialized proposal mechanism. Algorithm uses a physical analogy to make proposals. Given the position x, the potential energy is E(x) Construct a distribution p(x) e E(x), with E(x) = log( p(x)) where p(x) is the unnormalized density we can evaluate. Prob Learning (UofT) CSC412-Week 5-2/2 8 / 21
9 Hamiltonian Monte Carlo Construct a distribution p(x) e E(x), with E(x) = log( p(x)) where p(x) is the unnormalized density we can evaluate. Introduce momentum v carrying the kinetic energy K(v) = v 2 /2 Total energy or Hamiltonian: H = E(x) + K(v). Energy is preserved: Frictionless ball rolling (x, v) (x, v ) H(x, v) = H(x, v ). Ideal Hamiltonian dynamics are reversible: reverse v and the ball will return to its start point! Prob Learning (UofT) CSC412-Week 5-2/2 9 / 21
10 Hamiltonian Monte Carlo The joint distribution: p(x, v) e E(x) e K(v) = e E(x) K(v) = e H(x,v) Momentum is Gaussian, and independent of the position. MCMC procedure Sample the momentum Simulate Hamiltonian dynamics, flip sign of velocity Hamiltonian dynamics is reversible. Energy is constant p(x, v) = p(x, v ). How to simulate Hamiltonian dynamics? dx dt = H v dv dt = H x Prob Learning (UofT) CSC412-Week 5-2/2 10 / 21
11 Leap-frog integrator A numerical approximation: H is not conserved. Dynamics are still deterministic (and reversible) Acceptance probability : min{1, exp(h(x, v) H(x, v ))} Prob Learning (UofT) CSC412-Week 5-2/2 11 / 21
12 HMC algorithm The HMC algorithm (run until it mixes): Current position: x Sample momentum: v N (0, I). Run Leapfrog integrator for L steps and reach (x, v ) Accept new position x with probability: min{1, exp(h(x, v) H(x, v ))} Low energy points are favored. Prob Learning (UofT) CSC412-Week 5-2/2 12 / 21
13 MCMC Inference Sample from unnormalized posterior Estimate statistics from simulated values of x mean median quantiles Posterior predictive density of unobserved outcomes can be obtained by further simulation conditional on drawn values of x All of this however requires some care, as MCMC is not without problems Prob Learning (UofT) CSC412-Week 5-2/2 13 / 21
14 MCMC diagnostics How do we know we have ran the algorithm long enough? What if we started very far from where our distribution is? Since there is correlation between each item of the chain (autocorrelation), what is the effective number of samples? Prob Learning (UofT) CSC412-Week 5-2/2 14 / 21
15 Good Ideas for MCMC Parallel computation is cheap - we can run multiple chains in parallel starting at different points We should discard some initial number of samples - warm-up or burn-in We should examine how well the chain is mixed. (No need to memorize any of the formulas below) Prob Learning (UofT) CSC412-Week 5-2/2 15 / 21
16 R hat Start with m/2 chains of 2n samples (length of the chain) each, with a warm-up period of n. Split them in half so that we have m chains total (half of which are burn-in) of length n each. Label each scalar estimand with x i,j with (i = 1,..., n; j = 1,..., m) The between sequence variance B is: where: and: B = n m 1 x.j = 1 n x.. = 1 m m ( x.j x.. ) j=1 Prob Learning (UofT) CSC412-Week 5-2/2 16 / 21 n i=1 m j=1 x ij x.j
17 R hat The within sequence variance W is: where: W = 1 m s 2 j = 1 n 1 j=1 s 2 j n (x ij x.j ) 2 i=1 For any finite n, W will underestimate the true variance, since the chains have not had time to explore the entire possible range of values Prob Learning (UofT) CSC412-Week 5-2/2 17 / 21
18 R hat We can estimate the marginal posterior variance of x by a weighted average of W and B: var + (x) = n 1 n W + 1 n B This quantity overestimates the marginal posterior variance assuming the starting distribution is overdispersed, but is unbiased under stationarity or in the limit n We estimate the factor by which the scale of the current distribution for x might be reduced if we were to continue to infinity by: var ˆR + (x) = W If chains have not mixed well, R-hat is larger than 1 Prob Learning (UofT) CSC412-Week 5-2/2 18 / 21
19 Effective Sample Size Since our observations are not independent of each other, we de facto gain less information One way to quantify the effective sample size is to consider statistical efficiency of x.. as an estimate of E[x] ( ) lim mn var( x..) = ρ t var(x) n t=1 Where ρ t is the autocorrelation of the sequence x at lag t If the draws were completely independent we would have var( x.. ) = 1 mnvar(x) and the effective sample size would be mn Prob Learning (UofT) CSC412-Week 5-2/2 19 / 21
20 Autocorrelations We define the effective sample size to be: n eff = mn t=1 ρ t ρ t are unknown, so we estimate them by where V t = 1 m(n t) ˆρ t = 1 m V t 2 var + j=1 i=t+1 n (x i,j x i t,j ) 2 Prob Learning (UofT) CSC412-Week 5-2/2 20 / 21
21 Diagnostics Summary Once ˆR is near 1, and ˆn eff is more than 10 per chain for all scalar estimands we collect the mn simulations, (excluding the burn-in) We can then draw inference based on our samples. However: Even if the iterative simulations appear to have converged, passed all tests etc. It may still be far from convergence! When we declare convergence - we mean that all chains appear stationary and well mixed. Non of the checks we learned today are hypothesis test. There are no p-values, and no statistical significance. Prob Learning (UofT) CSC412-Week 5-2/2 21 / 21
Gaussian Processes to Speed up Hamiltonian Monte Carlo
Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo
More informationTutorial on Markov Chain Monte Carlo
Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,
More informationMCMC Using Hamiltonian Dynamics
5 MCMC Using Hamiltonian Dynamics Radford M. Neal 5.1 Introduction Markov chain Monte Carlo (MCMC) originated with the classic paper of Metropolis et al. (1953), where it was used to simulate the distribution
More informationIntroduction to Markov Chain Monte Carlo
Introduction to Markov Chain Monte Carlo Monte Carlo: sample from a distribution to estimate the distribution to compute max, mean Markov Chain Monte Carlo: sampling using local information Generic problem
More informationComputational Statistics for Big Data
Lancaster University Computational Statistics for Big Data Author: 1 Supervisors: Paul Fearnhead 1 Emily Fox 2 1 Lancaster University 2 The University of Washington September 1, 2015 Abstract The amount
More informationBayesian Statistics: Indian Buffet Process
Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note
More informationParallelization Strategies for Multicore Data Analysis
Parallelization Strategies for Multicore Data Analysis Wei-Chen Chen 1 Russell Zaretzki 2 1 University of Tennessee, Dept of EEB 2 University of Tennessee, Dept. Statistics, Operations, and Management
More informationValidation of Software for Bayesian Models Using Posterior Quantiles
Validation of Software for Bayesian Models Using Posterior Quantiles Samantha R. COOK, Andrew GELMAN, and Donald B. RUBIN This article presents a simulation-based method designed to establish the computational
More informationValidation of Software for Bayesian Models using Posterior Quantiles. Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT
Validation of Software for Bayesian Models using Posterior Quantiles Samantha R. Cook Andrew Gelman Donald B. Rubin DRAFT Abstract We present a simulation-based method designed to establish that software
More informationIntroduction to Monte Carlo. Astro 542 Princeton University Shirley Ho
Introduction to Monte Carlo Astro 542 Princeton University Shirley Ho Agenda Monte Carlo -- definition, examples Sampling Methods (Rejection, Metropolis, Metropolis-Hasting, Exact Sampling) Markov Chains
More informationStatistics Graduate Courses
Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.
More informationIntroduction to Markov Chain Monte Carlo
1 Introduction to Markov Chain Monte Carlo Charles J. Geyer 1.1 History Despite a few notable uses of simulation of random processes in the pre-computer era (Hammersley and Handscomb, 1964, Section 1.2;
More informationBayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com
Bayesian Machine Learning (ML): Modeling And Inference in Big Data Zhuhua Cai Google Rice University caizhua@gmail.com 1 Syllabus Bayesian ML Concepts (Today) Bayesian ML on MapReduce (Next morning) Bayesian
More informationParameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 12 04/08/2008. Sven Zenker
Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem Lecture 12 04/08/2008 Sven Zenker Assignment no. 8 Correct setup of likelihood function One fixed set of observation
More informationSTAT3016 Introduction to Bayesian Data Analysis
STAT3016 Introduction to Bayesian Data Analysis Course Description The Bayesian approach to statistics assigns probability distributions to both the data and unknown parameters in the problem. This way,
More informationOptimising and Adapting the Metropolis Algorithm
Optimising and Adapting the Metropolis Algorithm by Jeffrey S. Rosenthal 1 (February 2013; revised March 2013 and May 2013 and June 2013) 1 Introduction Many modern scientific questions involve high dimensional
More informationCentre for Central Banking Studies
Centre for Central Banking Studies Technical Handbook No. 4 Applied Bayesian econometrics for central bankers Andrew Blake and Haroon Mumtaz CCBS Technical Handbook No. 4 Applied Bayesian econometrics
More informationSampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data
Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data (Oxford) in collaboration with: Minjie Xu, Jun Zhu, Bo Zhang (Tsinghua) Balaji Lakshminarayanan (Gatsby) Bayesian
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationBayesian Statistics in One Hour. Patrick Lam
Bayesian Statistics in One Hour Patrick Lam Outline Introduction Bayesian Models Applications Missing Data Hierarchical Models Outline Introduction Bayesian Models Applications Missing Data Hierarchical
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationChenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu)
Paper Author (s) Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu) Lei Zhang, University of Maryland, College Park (lei@umd.edu) Paper Title & Number Dynamic Travel
More informationMISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group
MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could
More informationDirichlet Processes A gentle tutorial
Dirichlet Processes A gentle tutorial SELECT Lab Meeting October 14, 2008 Khalid El-Arini Motivation We are given a data set, and are told that it was generated from a mixture of Gaussian distributions.
More informationBasics of Statistical Machine Learning
CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar
More informationModel-based Synthesis. Tony O Hagan
Model-based Synthesis Tony O Hagan Stochastic models Synthesising evidence through a statistical model 2 Evidence Synthesis (Session 3), Helsinki, 28/10/11 Graphical modelling The kinds of models that
More information11. Time series and dynamic linear models
11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd
More informationImputing Missing Data using SAS
ABSTRACT Paper 3295-2015 Imputing Missing Data using SAS Christopher Yim, California Polytechnic State University, San Luis Obispo Missing data is an unfortunate reality of statistics. However, there are
More informationAPPLIED MISSING DATA ANALYSIS
APPLIED MISSING DATA ANALYSIS Craig K. Enders Series Editor's Note by Todd D. little THE GUILFORD PRESS New York London Contents 1 An Introduction to Missing Data 1 1.1 Introduction 1 1.2 Chapter Overview
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationModeling and Analysis of Call Center Arrival Data: A Bayesian Approach
Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Refik Soyer * Department of Management Science The George Washington University M. Murat Tarimcilar Department of Management Science
More informationE3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
More informationMAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX
MAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX KRISTOFFER P. NIMARK The next section derives the equilibrium expressions for the beauty contest model from Section 3 of the main paper. This is followed by
More informationPITFALLS IN TIME SERIES ANALYSIS. Cliff Hurvich Stern School, NYU
PITFALLS IN TIME SERIES ANALYSIS Cliff Hurvich Stern School, NYU The t -Test If x 1,..., x n are independent and identically distributed with mean 0, and n is not too small, then t = x 0 s n has a standard
More informationOn the mathematical theory of splitting and Russian roulette
On the mathematical theory of splitting and Russian roulette techniques St.Petersburg State University, Russia 1. Introduction Splitting is an universal and potentially very powerful technique for increasing
More informationParametric fractional imputation for missing data analysis
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Biometrika (????,??,?, pp. 1 14 C???? Biometrika Trust Printed in
More informationThe Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables 1 The Monte Carlo Framework Suppose we wish
More informationImproving paired comparison models for NFL point spreads by data transformation. Gregory J. Matthews
Improving paired comparison models for NFL point spreads by data transformation by Gregory J. Matthews A Project Report Submitted to the Faculty of WORCESTER POLYTECHNIC INSTITUTE in partial fulfillment
More informationSections 2.11 and 5.8
Sections 211 and 58 Timothy Hanson Department of Statistics, University of South Carolina Stat 704: Data Analysis I 1/25 Gesell data Let X be the age in in months a child speaks his/her first word and
More informationMonte Carlo Methods in Finance
Author: Yiyang Yang Advisor: Pr. Xiaolin Li, Pr. Zari Rachev Department of Applied Mathematics and Statistics State University of New York at Stony Brook October 2, 2012 Outline Introduction 1 Introduction
More informationIntroduction to Time Series Analysis. Lecture 1.
Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series
More informationMarkov Chain Monte Carlo and Numerical Differential Equations
Markov Chain Monte Carlo and Numerical Differential Equations J.M. Sanz-Serna 1 Introduction This contribution presents a hopefully readable introduction to Markov Chain Monte Carlo methods with particular
More informationBasic Bayesian Methods
6 Basic Bayesian Methods Mark E. Glickman and David A. van Dyk Summary In this chapter, we introduce the basics of Bayesian data analysis. The key ingredients to a Bayesian analysis are the likelihood
More informationSection 5. Stan for Big Data. Bob Carpenter. Columbia University
Section 5. Stan for Big Data Bob Carpenter Columbia University Part I Overview Scaling and Evaluation data size (bytes) 1e18 1e15 1e12 1e9 1e6 Big Model and Big Data approach state of the art big model
More informationGaussian Conjugate Prior Cheat Sheet
Gaussian Conjugate Prior Cheat Sheet Tom SF Haines 1 Purpose This document contains notes on how to handle the multivariate Gaussian 1 in a Bayesian setting. It focuses on the conjugate prior, its Bayesian
More informationGenerating Random Numbers Variance Reduction Quasi-Monte Carlo. Simulation Methods. Leonid Kogan. MIT, Sloan. 15.450, Fall 2010
Simulation Methods Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Simulation Methods 15.450, Fall 2010 1 / 35 Outline 1 Generating Random Numbers 2 Variance Reduction 3 Quasi-Monte
More informationA Latent Variable Approach to Validate Credit Rating Systems using R
A Latent Variable Approach to Validate Credit Rating Systems using R Chicago, April 24, 2009 Bettina Grün a, Paul Hofmarcher a, Kurt Hornik a, Christoph Leitner a, Stefan Pichler a a WU Wien Grün/Hofmarcher/Hornik/Leitner/Pichler
More informationSpatial Statistics Chapter 3 Basics of areal data and areal data modeling
Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data
More informationHandling attrition and non-response in longitudinal data
Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein
More informationAn Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics
Slide 1 An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Dr. Christian Asseburg Centre for Health Economics Part 1 Slide 2 Talk overview Foundations of Bayesian statistics
More informationAPPLIED MATHEMATICS ADVANCED LEVEL
APPLIED MATHEMATICS ADVANCED LEVEL INTRODUCTION This syllabus serves to examine candidates knowledge and skills in introductory mathematical and statistical methods, and their applications. For applications
More informationHandling missing data in large data sets. Agostino Di Ciaccio Dept. of Statistics University of Rome La Sapienza
Handling missing data in large data sets Agostino Di Ciaccio Dept. of Statistics University of Rome La Sapienza The problem Often in official statistics we have large data sets with many variables and
More informationMonte Carlo Simulation
1 Monte Carlo Simulation Stefan Weber Leibniz Universität Hannover email: sweber@stochastik.uni-hannover.de web: www.stochastik.uni-hannover.de/ sweber Monte Carlo Simulation 2 Quantifying and Hedging
More informationMarkov Chain Monte Carlo and Applied Bayesian Statistics: a short course Chris Holmes Professor of Biostatistics Oxford Centre for Gene Function
MCMC Appl. Bayes 1 Markov Chain Monte Carlo and Applied Bayesian Statistics: a short course Chris Holmes Professor of Biostatistics Oxford Centre for Gene Function MCMC Appl. Bayes 2 Objectives of Course
More informationJournal of Statistical Software
JSS Journal of Statistical Software October 2014, Volume 61, Issue 7. http://www.jstatsoft.org/ WebBUGS: Conducting Bayesian Statistical Analysis Online Zhiyong Zhang University of Notre Dame Abstract
More informationSampling for Bayesian computation with large datasets
Sampling for Bayesian computation with large datasets Zaiying Huang Andrew Gelman April 27, 2005 Abstract Multilevel models are extremely useful in handling large hierarchical datasets. However, computation
More informationBig Data need Big Model 1/44
Big Data need Big Model 1/44 Andrew Gelman, Bob Carpenter, Matt Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, Allen Riddell,... Department of Statistics,
More informationEx. 2.1 (Davide Basilio Bartolini)
ECE 54: Elements of Information Theory, Fall 00 Homework Solutions Ex.. (Davide Basilio Bartolini) Text Coin Flips. A fair coin is flipped until the first head occurs. Let X denote the number of flips
More informationEE 570: Location and Navigation
EE 570: Location and Navigation On-Line Bayesian Tracking Aly El-Osery 1 Stephen Bruder 2 1 Electrical Engineering Department, New Mexico Tech Socorro, New Mexico, USA 2 Electrical and Computer Engineering
More informationEstimation and comparison of multiple change-point models
Journal of Econometrics 86 (1998) 221 241 Estimation and comparison of multiple change-point models Siddhartha Chib* John M. Olin School of Business, Washington University, 1 Brookings Drive, Campus Box
More informationMarkov Chain Monte Carlo Simulation Made Simple
Markov Chain Monte Carlo Simulation Made Simple Alastair Smith Department of Politics New York University April2,2003 1 Markov Chain Monte Carlo (MCMC) simualtion is a powerful technique to perform numerical
More informationPREDICTIVE DISTRIBUTIONS OF OUTSTANDING LIABILITIES IN GENERAL INSURANCE
PREDICTIVE DISTRIBUTIONS OF OUTSTANDING LIABILITIES IN GENERAL INSURANCE BY P.D. ENGLAND AND R.J. VERRALL ABSTRACT This paper extends the methods introduced in England & Verrall (00), and shows how predictive
More informationBayesX - Software for Bayesian Inference in Structured Additive Regression
BayesX - Software for Bayesian Inference in Structured Additive Regression Thomas Kneib Faculty of Mathematics and Economics, University of Ulm Department of Statistics, Ludwig-Maximilians-University Munich
More informationAnalysis of Financial Time Series
Analysis of Financial Time Series Analysis of Financial Time Series Financial Econometrics RUEY S. TSAY University of Chicago A Wiley-Interscience Publication JOHN WILEY & SONS, INC. This book is printed
More informationPS 271B: Quantitative Methods II. Lecture Notes
PS 271B: Quantitative Methods II Lecture Notes Langche Zeng zeng@ucsd.edu The Empirical Research Process; Fundamental Methodological Issues 2 Theory; Data; Models/model selection; Estimation; Inference.
More informationVariance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers
Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications
More informationProbabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014
Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about
More informationApplying MCMC Methods to Multi-level Models submitted by William J Browne for the degree of PhD of the University of Bath 1998 COPYRIGHT Attention is drawn tothefactthatcopyright of this thesis rests with
More informationInference on Phase-type Models via MCMC
Inference on Phase-type Models via MCMC with application to networks of repairable redundant systems Louis JM Aslett and Simon P Wilson Trinity College Dublin 28 th June 202 Toy Example : Redundant Repairable
More informationThe Chinese Restaurant Process
COS 597C: Bayesian nonparametrics Lecturer: David Blei Lecture # 1 Scribes: Peter Frazier, Indraneel Mukherjee September 21, 2007 In this first lecture, we begin by introducing the Chinese Restaurant Process.
More information33. STATISTICS. 33. Statistics 1
33. STATISTICS 33. Statistics 1 Revised September 2011 by G. Cowan (RHUL). This chapter gives an overview of statistical methods used in high-energy physics. In statistics, we are interested in using a
More informationL13: cross-validation
Resampling methods Cross validation Bootstrap L13: cross-validation Bias and variance estimation with the Bootstrap Three-way data partitioning CSCE 666 Pattern Analysis Ricardo Gutierrez-Osuna CSE@TAMU
More informationHow To Understand The Theory Of Probability
Graduate Programs in Statistics Course Titles STAT 100 CALCULUS AND MATR IX ALGEBRA FOR STATISTICS. Differential and integral calculus; infinite series; matrix algebra STAT 195 INTRODUCTION TO MATHEMATICAL
More informationBayesian inference for population prediction of individuals without health insurance in Florida
Bayesian inference for population prediction of individuals without health insurance in Florida Neung Soo Ha 1 1 NISS 1 / 24 Outline Motivation Description of the Behavioral Risk Factor Surveillance System,
More informationLecture Notes 1. Brief Review of Basic Probability
Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very
More informationA Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data
A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data Faming Liang University of Florida August 9, 2015 Abstract MCMC methods have proven to be a very powerful tool for analyzing
More informationHIDDEN MARKOV MODELS FOR ALCOHOLISM TREATMENT TRIAL DATA
HIDDEN MARKOV MODELS FOR ALCOHOLISM TREATMENT TRIAL DATA By Kenneth E. Shirley, Dylan S. Small, Kevin G. Lynch, Stephen A. Maisto, and David W. Oslin Columbia University, University of Pennsylvania and
More informationAN INTRODUCTION TO MARKOV CHAIN MONTE CARLO METHODS AND THEIR ACTUARIAL APPLICATIONS. Department of Mathematics and Statistics University of Calgary
AN INTRODUCTION TO MARKOV CHAIN MONTE CARLO METHODS AND THEIR ACTUARIAL APPLICATIONS DAVID P. M. SCOLLNIK Department of Mathematics and Statistics University of Calgary Abstract This paper introduces the
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ Iain Murray http://www.cs.toronto.edu/~murray/ A statistical problem What is the average height of the MLSS
More informationA Tutorial on Probability Theory
Paola Sebastiani Department of Mathematics and Statistics University of Massachusetts at Amherst Corresponding Author: Paola Sebastiani. Department of Mathematics and Statistics, University of Massachusetts,
More informationReview of the Methods for Handling Missing Data in. Longitudinal Data Analysis
Int. Journal of Math. Analysis, Vol. 5, 2011, no. 1, 1-13 Review of the Methods for Handling Missing Data in Longitudinal Data Analysis Michikazu Nakai and Weiming Ke Department of Mathematics and Statistics
More informationOverview of Monte Carlo Simulation, Probability Review and Introduction to Matlab
Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?
More informationMaster s thesis tutorial: part III
for the Autonomous Compliant Research group Tinne De Laet, Wilm Decré, Diederik Verscheure Katholieke Universiteit Leuven, Department of Mechanical Engineering, PMA Division 30 oktober 2006 Outline General
More informationCombining Visual and Auditory Data Exploration for finding structure in high-dimensional data
Combining Visual and Auditory Data Exploration for finding structure in high-dimensional data Thomas Hermann Mark H. Hansen Helge Ritter Faculty of Technology Bell Laboratories Faculty of Technology Bielefeld
More informationBayesian Methods for the Social and Behavioral Sciences
Bayesian Methods for the Social and Behavioral Sciences Jeff Gill Harvard University 2007 ICPSR First Session: June 25-July 20, 9-11 AM. Email: jgill@iq.harvard.edu TA: Yu-Sung Su (ys463@columbia.edu).
More informationApplications of R Software in Bayesian Data Analysis
Article International Journal of Information Science and System, 2012, 1(1): 7-23 International Journal of Information Science and System Journal homepage: www.modernscientificpress.com/journals/ijinfosci.aspx
More informationBayes and Big Data: The Consensus Monte Carlo Algorithm
Bayes and Big Data: The Consensus Monte Carlo Algorithm Steven L. Scott, Alexander W. Blocker, Fernando V. Bonassi, Hugh A. Chipman, Edward I. George 3, and Robert E. McCulloch 4 Google, Inc. Acadia University
More informationDealing with large datasets
Dealing with large datasets (by throwing away most of the data) Alan Heavens Institute for Astronomy, University of Edinburgh with Ben Panter, Rob Tweedie, Mark Bastin, Will Hossack, Keith McKellar, Trevor
More informationBayesian prediction of disability insurance frequencies using economic indicators
Bayesian prediction of disability insurance frequencies using economic indicators Catherine Donnelly Heriot-Watt University, Edinburgh, UK Mario V. Wüthrich ETH Zurich, RisLab, Department of Mathematics,
More information1 Short Introduction to Time Series
ECONOMICS 7344, Spring 202 Bent E. Sørensen January 24, 202 Short Introduction to Time Series A time series is a collection of stochastic variables x,.., x t,.., x T indexed by an integer value t. The
More informationNon Linear Dependence Structures: a Copula Opinion Approach in Portfolio Optimization
Non Linear Dependence Structures: a Copula Opinion Approach in Portfolio Optimization Jean- Damien Villiers ESSEC Business School Master of Sciences in Management Grande Ecole September 2013 1 Non Linear
More informationOnline Model-Based Clustering for Crisis Identification in Distributed Computing
Online Model-Based Clustering for Crisis Identification in Distributed Computing Dawn Woodard School of Operations Research and Information Engineering & Dept. of Statistical Science, Cornell University
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David
More information**BEGINNING OF EXAMINATION** The annual number of claims for an insured has probability function: , 0 < q < 1.
**BEGINNING OF EXAMINATION** 1. You are given: (i) The annual number of claims for an insured has probability function: 3 p x q q x x ( ) = ( 1 ) 3 x, x = 0,1,, 3 (ii) The prior density is π ( q) = q,
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber 2011 1
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2011 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More informationReliability estimators for the components of series and parallel systems: The Weibull model
Reliability estimators for the components of series and parallel systems: The Weibull model Felipe L. Bhering 1, Carlos Alberto de Bragança Pereira 1, Adriano Polpo 2 1 Department of Statistics, University
More information10.2 Series and Convergence
10.2 Series and Convergence Write sums using sigma notation Find the partial sums of series and determine convergence or divergence of infinite series Find the N th partial sums of geometric series and
More information