# Introduction to Markov Chain Monte Carlo

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Introduction to Markov Chain Monte Carlo Monte Carlo: sample from a distribution to estimate the distribution to compute max, mean Markov Chain Monte Carlo: sampling using local information Generic problem solving technique decision/optimization/value problems generic, but not necessarily very efficient Based on - Neal Madras: Lectures on Monte Carlo Methods; AMS 2002

2 Lecture Outline Markov Chains notation & terminology fundamental properties of Markov Chains Sampling from prob. distributions using MCMC uniform desired target distribution Problem solving using MCMC optimization Relevance to Bayesian Networks

3 Markov Chains Notation & Terminology Countable (finite) state space Ω (e.g. N) Sequence of random variables {X t } on Ω for t =0,1,2,... Definition: {X t } is a Markov Chain if P[X t+1 = y X t =x t,...,x 0 =x 0 ] = P[X t+1 =y X t =x t ] Notation: P[X t+1 = i X t = j ] = p ji time-homogeneous

4 Markov Chains Examples Markov Chain Drawing a number from {1,2,3} with replacement. X t = last number seen at time t NOT a Markov Chain Drawing a number from {1,2,3} WITHOUT replacement. X t = last number seen at time t

5 Markov Chains Notation & Terminology Let P = (p ij ) transition probability matrix dimension Ω x Ω Let t (j) = P[X t = j] 0 initial probability distribution Then t (j) = i t-1 (i)p ij = ( t-1 P)(j) = ( o P t )(j) Example: graph vs. matrix representation

6 Markov Chains Fundamental Properties Theorem: Under some conditions (irreducibility and aperiodicity), the limit lim t P t exists and is ij independent of i; call it (j). If Ω is finite, then j (j) = 1 and ( P)(j) = (j) and such is a unique solution to xp=x ( is called a stationary distribution) Nice: no matter where we start, after some time, we will be in any state j with probability ~ (j) DEMO

7 Markov Chains Fundamental Properties Proposition: Assume a Markov Chain with discrete state space Ω. Assume there exist positive distribution on Ω ( (i)>0 and i (i) = 1) and for every i,j: (i)p ij = (j)p ji (detailed balance property) then is the stationary distribution of P Corollary: If transition matrix P is symmetric and Ω finite, then the stationary distribution is (i)=1/ Ω DEMO

8 Markov Chain Monte Carlo Random Walk on {0,1} m Ω={0,1} m generate chain: pick J {1,...,m} uniformly at random and set X t =(z 1,...,1-z J,...,z m ) where (z 1,...,z m )=X t-1 Markov Chain Monte Carlo basic idea: Given a prob. distribution on a set Ω, the problem is to generate random elements of Ω with distribution. MCMC does that by constructing a Markov Chain with stationary distribution and simulating the chain.

9 MCMC: Uniform Sampler Problem: sample elements uniformly at random from set (large but finite) Ω Idea: construct an irreducible symmetric Markov Chain with states Ω and run it for sufficient time by Theorem and Corollary, this will work Example: generate uniformly at random a feasible solution to the Knapsack Problem

10 MCMC: Uniform Sampler Example Knapsack Problem Definition Given: m items and their weight w i and value v i, knapsack with weight limit b Find: what is the most valuable subset of items that will fit into the knapsack? Representation: z=(z 1,...,z m ) {0,1} m, z i means whether we take item i feasible solutions Ω = { z {0,1} m ; i w i z i b} problem: maximize i v i z i subject to z Ω

11 MCMC Example: Knapsack Problem Uniform sampling using MCMC: given current X t =(z 1,...,z m ), generate X t+1 by: (1) choose J {1,...,m} uniformly at random (2) flip z J, i.e. let y = (z 1,...,1-z J,...,z m ) (3) if y is feasible, then set X t+1 = y, else set X t+1 = X t Comments: P ij is symmetric uniform sampling how long should we run it? can we use this to find a good solution?

12 MCMC Example: Knapsack Problem Can we use MCMC to find good solution? Yes: keep generating feasible solutions uniformly at random and remember the best one seen so far. this may take very long time, if number of good solutions is small Better: generate good solutions with higher probability => sample from a distribution where good solutions have higher probabilities (z) = C -1 exp( i v i z i )

13 MCMC: Target Distribution Sampler Let Ω be a countable (finite) state space Let Q be a symmetric transition prob. matrix Let be any prob. distribution on Ω s.t. (i)>0 the target distribution we can define a new Markov Chain {X i } such that its stationary distribution is this allows to sample from Ω according to

14 MCMC: Metropolis Algorithm Given such Ω,,Q creates a new MC {X t }: (1) choose proposal y randomly using Q P[Y=j X t = i ] = q ij (2) let = min{1, (Y)/ (i)} (acceptance probability) (3) accept y with probability, i.e. X t+1 =Y with prob., X t+1 =X t otherwise Resulting p ij : p ij =q ij min{1, (j)/ (i)} for i j p ii = 1 - j i p ij

15 MCMC: Metropolis Algorithm Proposition (Metropolis works): The p ij 's from Metropolis Algorithm satisfy detailed balance property w.r.t i.e. (i)p ij = (j)p ji the new Markov Chain has a stationary distr. Remarks: we only need to know ratios of values of the MC might converge to exponentially slowly

16 MCMC: Metropolis Algorithm Knapsack Problem Target distribution: -1 (z) = C b exp( b i v i z i ) Algorithm: (1) choose J {1,...,m} uniformly at random (2) let y = (z 1,...,1-z J,...,z m ) (3) if y is not feasible, then X t+1 = X t (4) if y is feasible, set X t+1 = y with prob., else X t+1 = X t where = min{1, exp( b i v i (y i -z i )}

17 MCMC: Optimization Metropolis Algorithm theoretically works, but: needs large b to make good states more likely its convergence time may be exponential in b try changing b over time Simulated Annealing for Knapsack Problem: = min{1, exp( b(t) i v i (y i -z i )} b(t) increases slowly with time (e.g. =log(t), =(1.001) t )

18 MCMC: Simulated Annealing General optimization problem: maximize function G(z) on all feasible solutions Ω let Q be again symmetric transition prob. matrix on Ω Simulated Annealing is Metropolis Algorithm with p ij =q ij min{1, exp( b(t) [G(j)-G(i)]) } for i j p ii = 1 - j i p ij Effect of b(t): exploration vs. exploitation trade-off

19 MCMC: Gibbs Sampling Consider a factored state space z Ω is a vector z=(z 1,...,z m ) notation: z -i = (z 1,...,z i-1,z i+1,...,z m ) Assume that target is s.t. P[Z i z -i ] is known Algorithm: (1) pick a component i {1,...,m} (2) sample value of z i from P[Z i z -i ], set X t =(z 1,...,z m ) A special case of generalized Metropolis Sampling (Metropolis-Hastings)

20 MCMC: Relevance to Bayesian Networks In Bayesian Networks, we know P[Z i z -i ] = P[Z i MarkovBlanket(Z i )] BN Inference Problem: compute P[Z i =z i E=e] Possible solution: (1) sample from worlds according to P[Z=z E=e] (2) compute fraction of those worlds where Z i =z i Gibbs Sampler works: let (z) = P[Z=z E=e], then P[Z i z -i ] satisfies detailed balance property w.r.t (z) (z) is stationary

21 MCMC: Inference in BN Example P[H S=true, B=true]

22 MCMC: Inference in BN Example h,l p (h,l) ( h,l) =? h,l h, l h, l Smoking and Breathing difficulties are fixed

23 MCMC: Inference in BN Example P[z i MB(Z i )] P[z i Par(Z i )] Y Chld(Z) P[y Par(Y)] p (h,l) ( h,l) = P[h gets picked].p[ h MB(H)] = ½.P[ h l,s,b] = ½.αP[ h s].p[b h,l]

### Tutorial on Markov Chain Monte Carlo

Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,

### Gaussian Processes to Speed up Hamiltonian Monte Carlo

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo

### EC 6310: Advanced Econometric Theory

EC 6310: Advanced Econometric Theory July 2008 Slides for Lecture on Bayesian Computation in the Nonlinear Regression Model Gary Koop, University of Strathclyde 1 Summary Readings: Chapter 5 of textbook.

### Lecture 6: The Bayesian Approach

Lecture 6: The Bayesian Approach What Did We Do Up to Now? We are given a model Log-linear model, Markov network, Bayesian network, etc. This model induces a distribution P(X) Learning: estimate a set

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

### Introduction to Monte Carlo. Astro 542 Princeton University Shirley Ho

Introduction to Monte Carlo Astro 542 Princeton University Shirley Ho Agenda Monte Carlo -- definition, examples Sampling Methods (Rejection, Metropolis, Metropolis-Hasting, Exact Sampling) Markov Chains

### Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com

Bayesian Machine Learning (ML): Modeling And Inference in Big Data Zhuhua Cai Google Rice University caizhua@gmail.com 1 Syllabus Bayesian ML Concepts (Today) Bayesian ML on MapReduce (Next morning) Bayesian

### Sampling-based optimization

Sampling-based optimization Richard Combes October 11, 2013 The topic of this lecture is a family of mathematical techniques called sampling-based methods. These methods are called sampling mathods because

### 1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let

Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as

### A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data

A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data Faming Liang University of Florida August 9, 2015 Abstract MCMC methods have proven to be a very powerful tool for analyzing

### Bayesian Statistics: Indian Buffet Process

Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note

### LECTURE 4. Last time: Lecture outline

LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random

### Random access protocols for channel access. Markov chains and their stability. Laurent Massoulié.

Random access protocols for channel access Markov chains and their stability laurent.massoulie@inria.fr Aloha: the first random access protocol for channel access [Abramson, Hawaii 70] Goal: allow machines

### Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

### Dirichlet Processes A gentle tutorial

Dirichlet Processes A gentle tutorial SELECT Lab Meeting October 14, 2008 Khalid El-Arini Motivation We are given a data set, and are told that it was generated from a mixture of Gaussian distributions.

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### Markov Chains, Stochastic Processes, and Advanced Matrix Decomposition

Markov Chains, Stochastic Processes, and Advanced Matrix Decomposition Jack Gilbert Copyright (c) 2014 Jack Gilbert. Permission is granted to copy, distribute and/or modify this document under the terms

### Example. Two fair dice are tossed and the two outcomes recorded. As is familiar, we have

Lectures 9-10 jacques@ucsd.edu 5.1 Random Variables Let (Ω, F, P ) be a probability space. The Borel sets in R are the sets in the smallest σ- field on R that contains all countable unions and complements

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 12 04/08/2008. Sven Zenker

Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem Lecture 12 04/08/2008 Sven Zenker Assignment no. 8 Correct setup of likelihood function One fixed set of observation

### Combining information from different survey samples - a case study with data collected by world wide web and telephone

Combining information from different survey samples - a case study with data collected by world wide web and telephone Magne Aldrin Norwegian Computing Center P.O. Box 114 Blindern N-0314 Oslo Norway E-mail:

### ABC methods for model choice in Gibbs random fields

ABC methods for model choice in Gibbs random fields Jean-Michel Marin Institut de Mathématiques et Modélisation Université Montpellier 2 Joint work with Aude Grelaud, Christian Robert, François Rodolphe,

### Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.

Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()

### Metropolis Light Transport. Samuel Donow, Mike Flynn, David Yan CS371 Fall 2014, Morgan McGuire

Metropolis Light Transport Samuel Donow, Mike Flynn, David Yan CS371 Fall 2014, Morgan McGuire Overview of Presentation 1. Description of necessary tools (Path Space, Monte Carlo Integration, Rendering

### Perron vector Optimization applied to search engines

Perron vector Optimization applied to search engines Olivier Fercoq INRIA Saclay and CMAP Ecole Polytechnique May 18, 2011 Web page ranking The core of search engines Semantic rankings (keywords) Hyperlink

### Inference on Phase-type Models via MCMC

Inference on Phase-type Models via MCMC with application to networks of repairable redundant systems Louis JM Aslett and Simon P Wilson Trinity College Dublin 28 th June 202 Toy Example : Redundant Repairable

### Polarization codes and the rate of polarization

Polarization codes and the rate of polarization Erdal Arıkan, Emre Telatar Bilkent U., EPFL Sept 10, 2008 Channel Polarization Given a binary input DMC W, i.i.d. uniformly distributed inputs (X 1,...,

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### Optimising and Adapting the Metropolis Algorithm

Optimising and Adapting the Metropolis Algorithm by Jeffrey S. Rosenthal 1 (February 2013; revised March 2013 and May 2013 and June 2013) 1 Introduction Many modern scientific questions involve high dimensional

### Hypothesis Testing. 1 Introduction. 2 Hypotheses. 2.1 Null and Alternative Hypotheses. 2.2 Simple vs. Composite. 2.3 One-Sided and Two-Sided Tests

Hypothesis Testing 1 Introduction This document is a simple tutorial on hypothesis testing. It presents the basic concepts and definitions as well as some frequently asked questions associated with hypothesis

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Math Homework 7 Solutions

Math 450 - Homework 7 Solutions 1 Find the invariant distribution of the transition matrix: 0 1 0 P = 2 1 0 3 3 p 1 p 0 The equation πp = π, and the normalization condition π 1 + π 2 + π 3 = 1 give the

### Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu)

Paper Author (s) Chenfeng Xiong (corresponding), University of Maryland, College Park (cxiong@umd.edu) Lei Zhang, University of Maryland, College Park (lei@umd.edu) Paper Title & Number Dynamic Travel

### Set theory, and set operations

1 Set theory, and set operations Sayan Mukherjee Motivation It goes without saying that a Bayesian statistician should know probability theory in depth to model. Measure theory is not needed unless we

### On the mathematical theory of splitting and Russian roulette

On the mathematical theory of splitting and Russian roulette techniques St.Petersburg State University, Russia 1. Introduction Splitting is an universal and potentially very powerful technique for increasing

### 15 Markov Chains: Limiting Probabilities

MARKOV CHAINS: LIMITING PROBABILITIES 67 Markov Chains: Limiting Probabilities Example Assume that the transition matrix is given by 7 2 P = 6 Recall that the n-step transition probabilities are given

### Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091)

Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I February

### Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data

Sampling via Moment Sharing: A New Framework for Distributed Bayesian Inference for Big Data (Oxford) in collaboration with: Minjie Xu, Jun Zhu, Bo Zhang (Tsinghua) Balaji Lakshminarayanan (Gatsby) Bayesian

### Markov Chain Monte Carlo and Numerical Differential Equations

Markov Chain Monte Carlo and Numerical Differential Equations J.M. Sanz-Serna 1 Introduction This contribution presents a hopefully readable introduction to Markov Chain Monte Carlo methods with particular

### M/M/1 and M/M/m Queueing Systems

M/M/ and M/M/m Queueing Systems M. Veeraraghavan; March 20, 2004. Preliminaries. Kendall s notation: G/G/n/k queue G: General - can be any distribution. First letter: Arrival process; M: memoryless - exponential

### Parallelization Strategies for Multicore Data Analysis

Parallelization Strategies for Multicore Data Analysis Wei-Chen Chen 1 Russell Zaretzki 2 1 University of Tennessee, Dept of EEB 2 University of Tennessee, Dept. Statistics, Operations, and Management

### Local Search. 22c:145 Artificial Intelligence. Local Search: General Principle. Hill-climbing search. Local Search. Intro example: N-queens

So far: methods that systematically explore the search space, possibly using principled pruning (e.g., A*) c:145 Artificial Intelligence Local Search Readings Textbook: Chapter 4:1 and 6:4 Current best

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### B. Maddah ENMG 501 Engineering Management I 03/17/07

B. Maddah ENMG 501 Engineering Management I 03/17/07 Discrete Time Markov Chains (3) Long-run Properties of MC (Stationary Solution) Consider the two-state MC of the weather condition in Example 4. P 0.5749

### Introduction to Stationary Distributions

13 Introduction to Stationary Distributions We first briefly review the classification of states in a Markov chain with a quick example and then begin the discussion of the important notion of stationary

### Bonus-malus systems and Markov chains

Bonus-malus systems and Markov chains Dutch car insurance bonus-malus system class % increase new class after # claims 0 1 2 >3 14 30 14 9 5 1 13 32.5 14 8 4 1 12 35 13 8 4 1 11 37.5 12 7 3 1 10 40 11

### Markov Chain Monte Carlo Simulation Made Simple

Markov Chain Monte Carlo Simulation Made Simple Alastair Smith Department of Politics New York University April2,2003 1 Markov Chain Monte Carlo (MCMC) simualtion is a powerful technique to perform numerical

### 9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1

9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 Seffi Naor Computer Science Dept. Technion Haifa, Israel Introduction

### Estimating the evidence for statistical models

Estimating the evidence for statistical models Nial Friel University College Dublin nial.friel@ucd.ie March, 2011 Introduction Bayesian model choice Given data y and competing models: m 1,..., m l, each

### 11. Time series and dynamic linear models

11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd

### Bayesian Statistics in One Hour. Patrick Lam

Bayesian Statistics in One Hour Patrick Lam Outline Introduction Bayesian Models Applications Missing Data Hierarchical Models Outline Introduction Bayesian Models Applications Missing Data Hierarchical

### Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization

Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization Archis Ghate a and Robert L. Smith b a Industrial Engineering, University of Washington, Box 352650, Seattle, Washington,

### Model-based Synthesis. Tony O Hagan

Model-based Synthesis Tony O Hagan Stochastic models Synthesising evidence through a statistical model 2 Evidence Synthesis (Session 3), Helsinki, 28/10/11 Graphical modelling The kinds of models that

### Monte Carlo Simulation technique. S. B. Santra Department of Physics Indian Institute of Technology Guwahati

Monte Carlo Simulation technique S. B. Santra Department of Physics Indian Institute of Technology Guwahati What is Monte Carlo (MC)? Introduction Monte Carlo method is a common name for a wide variety

### Lecture 6: The Group Inverse

Lecture 6: The Group Inverse The matrix index Let A C n n, k positive integer. Then R(A k+1 ) R(A k ). The index of A, denoted IndA, is the smallest integer k such that R(A k ) = R(A k+1 ), or equivalently,

Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

### General theory of stochastic processes

CHAPTER 1 General theory of stochastic processes 1.1. Definition of stochastic process First let us recall the definition of a random variable. A random variable is a random number appearing as a result

### Monte Carlo Methods in Finance

Author: Yiyang Yang Advisor: Pr. Xiaolin Li, Pr. Zari Rachev Department of Applied Mathematics and Statistics State University of New York at Stony Brook October 2, 2012 Outline Introduction 1 Introduction

### The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

### Lecture 13: Martingales

Lecture 13: Martingales 1. Definition of a Martingale 1.1 Filtrations 1.2 Definition of a martingale and its basic properties 1.3 Sums of independent random variables and related models 1.4 Products of

### Math 526: Brownian Motion Notes

Math 526: Brownian Motion Notes Definition. Mike Ludkovski, 27, all rights reserved. A stochastic process (X t ) is called Brownian motion if:. The map t X t (ω) is continuous for every ω. 2. (X t X t

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### Welcome to the course Algorithm Design

HEINZ NIXDORF INSTITUTE University of Paderborn Algorithms and Complexity Welcome to the course Algorithm Design Summer Term 2011 Friedhelm Meyer auf der Heide Lecture 6, 20.5.2011 Friedhelm Meyer auf

### Reinforcement Learning with Factored States and Actions

Journal of Machine Learning Research 5 (2004) 1063 1088 Submitted 3/02; Revised 1/04; Published 8/04 Reinforcement Learning with Factored States and Actions Brian Sallans Austrian Research Institute for

### Message-passing sequential detection of multiple change points in networks

Message-passing sequential detection of multiple change points in networks Long Nguyen, Arash Amini Ram Rajagopal University of Michigan Stanford University ISIT, Boston, July 2012 Nguyen/Amini/Rajagopal

### Monte Carlo Simulation

1 Monte Carlo Simulation Stefan Weber Leibniz Universität Hannover email: sweber@stochastik.uni-hannover.de web: www.stochastik.uni-hannover.de/ sweber Monte Carlo Simulation 2 Quantifying and Hedging

### Bounding the Mean Cumulated Reward up to Absorption

Bounding the Mean Cumulated Reward up to Absorption A. P. Couto and G. Rubino COPPE & INRIA/IRISA MAM 2006 A. P. Couto and G. Rubino (COPPE & INRIA/IRISA) Bounding the ECRA MAM 2006 1 / 26 Outline 1/ Introduction

### Hydrodynamic Limits of Randomized Load Balancing Networks

Hydrodynamic Limits of Randomized Load Balancing Networks Kavita Ramanan and Mohammadreza Aghajani Brown University Stochastic Networks and Stochastic Geometry a conference in honour of François Baccelli

### Monte Carlo-based statistical methods (MASM11/FMS091)

Monte Carlo-based statistical methods (MASM11/FMS091) Jimmy Olsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I February 5, 2013 J. Olsson Monte Carlo-based

### The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables

Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables 1 The Monte Carlo Framework Suppose we wish

### 4. Joint Distributions

Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose

### Calibration of Dynamic Traffic Assignment Models

Calibration of Dynamic Traffic Assignment Models m.hazelton@massey.ac.nz Presented at DADDY 09, Salerno Italy, 2-4 December 2009 Outline 1 Statistical Inference as Part of the Modelling Process Model Calibration

### Introduction to Probability Theory for Graduate Economics. Fall 2008

Introduction to Probability Theory for Graduate Economics Fall 2008 Yiğit Sağlam December 1, 2008 CHAPTER 5 - STOCHASTIC PROCESSES 1 Stochastic Processes A stochastic process, or sometimes a random process,

### MASTER S THESIS for stud.techn. Kristian Stormark. Faculty of Information Technology, Mathematics and Electrical Engineering NTNU

NTNU Norwegian University of Science and Technology Faculty of Information Technology, Mathematics and Electrical Engineering MASTER S THESIS for stud.techn. Kristian Stormark Faculty of Information Technology,

### Monte Carlo Simulation

Chapter 4 Monte Carlo Simulation Due to the complexity of the problem, treating irregular spin systems with competing interactions in thermal equilibrium and nonequilibrium, we are forced to use Monte

### Incorporating cost in Bayesian Variable Selection, with application to cost-effective measurement of quality of health care.

Incorporating cost in Bayesian Variable Selection, with application to cost-effective measurement of quality of health care University of Florida 10th Annual Winter Workshop: Bayesian Model Selection and

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### IEOR 6711: Stochastic Models, I Fall 2012, Professor Whitt, Final Exam SOLUTIONS

IEOR 6711: Stochastic Models, I Fall 2012, Professor Whitt, Final Exam SOLUTIONS There are four questions, each with several parts. 1. Customers Coming to an Automatic Teller Machine (ATM) (30 points)

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### Multi Skilled Staff Scheduling with Simulated Annealing

Delft University of Technology MSc Thesis Multi Skilled Staff Scheduling with Simulated Annealing Author: Willem Hendrik Hendriks Supervisor: dr. ir. Frank van der Meulen April 3, 2013 In cooperation with:

### Echantillonnage exact de distributions de Gibbs d énergies sous-modulaires. Exact sampling of Gibbs distributions with submodular energies

Echantillonnage exact de distributions de Gibbs d énergies sous-modulaires Exact sampling of Gibbs distributions with submodular energies Marc Sigelle 1 and Jérôme Darbon 2 1 Institut TELECOM TELECOM ParisTech

### Model Calibration with Open Source Software: R and Friends. Dr. Heiko Frings Mathematical Risk Consulting

Model with Open Source Software: and Friends Dr. Heiko Frings Mathematical isk Consulting Bern, 01.09.2011 Agenda in a Friends Model with & Friends o o o Overview First instance: An Extreme Value Example

### Lecture 4: Random Variables

Lecture 4: Random Variables 1. Definition of Random variables 1.1 Measurable functions and random variables 1.2 Reduction of the measurability condition 1.3 Transformation of random variables 1.4 σ-algebra

### An Introduction to Basic Statistics and Probability

An Introduction to Basic Statistics and Probability Shenek Heyward NCSU An Introduction to Basic Statistics and Probability p. 1/4 Outline Basic probability concepts Conditional probability Discrete Random

### Monte Carlo-based statistical methods (MASM11/FMS091)

Monte Carlo-based statistical methods (MASM11/FMS091) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 6 Sequential Monte Carlo methods II February 7, 2014 M. Wiktorsson

### Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.5 The t Test 9.6

### COS 511: Foundations of Machine Learning. Rob Schapire Lecture #14 Scribe: Qian Xi March 30, 2006

COS 511: Foundations of Machine Learning Rob Schapire Lecture #14 Scribe: Qian Xi March 30, 006 In the previous lecture, we introduced a new learning model, the Online Learning Model. After seeing a concrete

### MCMC-Based Assessment of the Error Characteristics of a Surface-Based Combined Radar - Passive Microwave Cloud Property Retrieval

MCMC-Based Assessment of the Error Characteristics of a Surface-Based Combined Radar - Passive Microwave Cloud Property Retrieval Derek J. Posselt University of Michigan Jay G. Mace University of Utah

### Computational Statistics for Big Data

Lancaster University Computational Statistics for Big Data Author: 1 Supervisors: Paul Fearnhead 1 Emily Fox 2 1 Lancaster University 2 The University of Washington September 1, 2015 Abstract The amount

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### Scheduling Shop Scheduling. Tim Nieberg

Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations

### What is Statistics? Lecture 1. Introduction and probability review. Idea of parametric inference

0. 1. Introduction and probability review 1.1. What is Statistics? What is Statistics? Lecture 1. Introduction and probability review There are many definitions: I will use A set of principle and procedures

### Microclustering: When the Cluster Sizes Grow Sublinearly with the Size of the Data Set

Microclustering: When the Cluster Sizes Grow Sublinearly with the Size of the Data Set Jeffrey W. Miller Brenda Betancourt Abbas Zaidi Hanna Wallach Rebecca C. Steorts Abstract Most generative models for

### Master s Theory Exam Spring 2006

Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

### Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

### Christfried Webers. Canberra February June 2015

c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

### Probabilistic Graphical Models

Probabilistic Graphical Models Raquel Urtasun and Tamir Hazan TTI Chicago April 4, 2011 Raquel Urtasun and Tamir Hazan (TTI-C) Graphical Models April 4, 2011 1 / 22 Bayesian Networks and independences

### Lecture 7: Continuous Random Variables

Lecture 7: Continuous Random Variables 21 September 2005 1 Our First Continuous Random Variable The back of the lecture hall is roughly 10 meters across. Suppose it were exactly 10 meters, and consider