Youssef Marzouk Massachusetts Institute of Technology. joint work with Xun Huan (MIT), Habib Najm (Sandia), Dongbin Xiu (Purdue)

Size: px
Start display at page:

Download "Youssef Marzouk Massachusetts Institute of Technology. joint work with Xun Huan (MIT), Habib Najm (Sandia), Dongbin Xiu (Purdue)"

Transcription

1 Youssef Marzouk Massachusetts Institute of Technology joint work with Xun Huan (MIT), Habib Najm (Sandia), Dongbin Xiu (Purdue)

2 Questions How to quantify confidence in computational predictions? Use these predictions in design and decision-making. Uncertainties abound: parameters, inputs, boundary conditions, model structure How to build models? Learning from noisy, indirect experimental observations. How to choose observations? What best to measure? Need tractable methodologies for complex physical models!

3 A few applications reacting flow, chemical kinetics chemical models in reacting flow atmospheric transport and dispersion groundwater flow, contaminant remediation, carbon sequestration

4 Statistical inference Building and refining models a Bayesian approach Model parameters (or structure) m are random variables Apply Bayes rule: posterior density p( m d) = ( ) p m ( ) p m p d m p d m π m likelihood function L(m) ( ) ( )dm ( ) ( ) p m d prior density The posterior density is the full Bayesian solution to the inference problem Not just a single value for m, but a probability density A complete description of uncertainty An input to future simulations for prediction with uncertainty

5 Computational challenge Extracting information from the posterior Means, variances, higher moments; marginal distributions; realizations; posterior predictions But Posterior evaluations are expensive (forward model = PDE) Posteriors are often high-dimensional Key step: Use stochastic spectral methods in Bayesian inference Enable statistical inference with more realistic physical models (via multiple order-of-magnitude speedups)

6 Things on my poster 1 Accelerating Bayesian inference with stochastic spectral methods 2 Spatially distributed parameters; inference in high dimensions 3 Optimal experimental design

7 Stochastic spectral methods Tool for efficient uncertainty propagation and assessment Polynomial chaos [Wiener 1938, Kakutani 1961, Ghanem 1991, ] Random variable X : Ω R with finite variance PCe: { ( )} i=1 { ( )} ( ) X(ω) = x k Ψ k ξ i1,ξ i2, k =0 ξ i ω are i.i.d. random variables (e.g., Gaussian) Ψ k ξ are orthogonal multivariate polynomials (e.g., Hermite) Polynomials of random variables; numerous extensions (piecewise polynomials, wavelets, etc)

8 Accelerating Bayesian inference Propagate prior uncertainty through the forward problem forward ( ) G j ( ξ ) = g jk Ψ k ( ξ ) m ξ model Galerkin or collocation (many methods here ) Result: a stochastic spectral representation of forward model predictions (compute once) Use this as a surrogate for the forward model in the likelihood function. No repeated forward problem solutions! P k= 0 [Marzouk et al JCP 2007]

9 Convergence result Simplest surrogate posterior density (ξ m) Convergence of the forward approximation implies convergence of the posterior distribution: Assume observational error is i.i.d. Gaussian, is prior density on ξ If π N ξ n d ( ) = p η ( d i G N,i (ξ)) p ξ ( ξ ) i=1 G i ( ξ ) G N,i ( ξ ) L 2 p ξ D( π N π ) N α CN α, 1 i n d, α > 0 then for sufficiently large N. p ξ [Marzouk & Xiu, CiCP 2009]

10 Speedup Total computational time vs number of posterior samples Per-sample cost reduced by 3 4 orders of magnitude!

11 Example: kinetic parameters Example: estimate kinetic parameters in a genetic toggle switch DAE model from [Gardner et al 2000] Real experimental data: steady-state expression levels of one gene (v) du = α 1 dt 1 + v u β dv = α 2 dt 1 + w v γ u w = ( 1 + [IPTG] K ) η

12 Example: kinetic parameters 1-parameter and 2-parameter marginal posterior densities gpc-based pseudospectral approach (dim = 6, N = 4, 6-level sparse grid)

13 Distributed parameters Characterize spatially distributed parameters and states from sparse and indirect data Typical/realistic problems are ill-posed and high-dimensional Example: flow in porous media M(x) = log ν(x) is an unknown log-permeability; infer from a few sources/sensors in the domain D = [0,1] Bayesian approach: u t = ( ν(x) u( x,t) ) + s( x,t) M(x,ω) is a stochastic process Inference and prediction using p(m d) M(x) sources sensors

14 Bayesian inverse problems Big picture of the computational approach: Endow the field M(x,ω) with an appropriate prior and hyperpriors Spectral methods propagate prior uncertainty through the forward model Efficient MCMC simulation from the surrogate posterior M(x,ω) forward model P G j ( ξ) = g jk Ψ k ξ k =0 ( ) Key steps: reduce dimensionality and exploit ill-posedness

15 Gaussian process priors Examples of stationary Gaussian random fields: C( x 1, x 2 ) = θ exp( x 1 x 2 L) C x 1, x 2 (need many KL modes) ( 2L 2 ) ( ) = θ exp x 1 x 2 2 (fewer KL modes) M(x,ω) = µ(x) + K i=1 λ i c i (ω) φ i (x)

16 Bayesian inversion Results: posterior distribution of log-permeability profile Dimensionality reduction (KL expansion) based on prior information MCMC sampling from the surrogate posterior (200K samples) mean, stdev, realizations median, quantiles, true profile [Marzouk & Najm 2009]

17 Bayesian inversion Results: posterior distribution of log-permeability profile Dimensionality reduction (KL expansion) based on prior information MCMC sampling from the surrogate posterior (200K samples) grid-based, direct forward evals s Karhunen-Loève basis + poly chaos 248 s

18 Structure of the posterior 1-D & 2-D posterior marginals of the KL mode strengths, p(c i d): Rougher modes revert to the prior distribution! ( ) p( c i θ) p c m,c m+1, d,θ i m

19 Structure of the posterior Posterior distributions p(c i d,θ) with coarser data: c 6, 13 sensors c 6, 2 sensors ( ) p( c i θ ) p c m,c m +1, d,θ i m at smaller m Additional opportunities for dimensionality reduction

20 Dimensionality reduction How big/complex is an inference problem? The dimensionality of the inference problem should also reflect the data resolution and the forward physics A compromise between information in the prior and information in the likelihood function Example: linear forward model, Gaussian prior + noise y = Gx + η η ~ N ( 0,Γ obs ), x ~ N ( 0,Γ ) pr H G T Γ obs 1 G x y N ( µ post,γ ) post is the Hessian of log-likelihood

21 Dimensionality reduction Linear example (continued): Consider generalized Rayleigh quotient Information about w from the likelihood How much does the prior constrain w? w T Hw w T Γ 1 pr w Hw = λγ pr 1 w Generalized eigenproblem yields low-rank approximation of the change to posterior covariance: Γ post Γ pr W k Σ k W k T, with Σ k = Approximate the forward model only in directions where the posterior is most different from the prior (i.e., range of W k ) Equivalently: seek conditional independence ( Λ 1 k + I ) 1 k

22 Things on my poster 1 Accelerating Bayesian inference with stochastic spectral methods 2 Spatially distributed parameters; inference in high dimensions 3 Optimal experimental design

23 Experimental design We can infer parameters from indirect and imperfect data, and we can propagate the resulting uncertainties to outputs of interest. What about optimal experimental design? What to measure? When/where to do it? What experimental conditions yield the most information? Forward models are nonlinear; need simulation-based approaches Broad applications: structural health monitoring; well placement; sensor steering; testing/characterization of alternative fuels, electrochemical devices, catalytic surfaces

24 Experimental design Combustion example: ignition with detailed chemistry (H 2 -O 2 ) Infer kinetic parameters (pre-exponentials, activation energies) from measurements of ignition delay, peak heat release rate, peak radical concentrations. What initial mixture conditions (T, φ, p, dilution) should one choose? T = 1000 K, φ = 1.0 T = 1000 K, φ = 1.0 T = 2000 K, φ = 5.0

25 Experimental design Combustion example: ignition with detailed chemistry (H 2 -O 2 ) Infer kinetic parameters (pre-exponentials, activation energies) from measurements of ignition delay, peak heat release rate, peak radical concentrations. What initial mixture conditions (T, φ, p, dilution) should one choose? T = 1000 K, φ = 1.0 T = 1000 K, φ = 1.0 T = 900 K, φ = 0.1

26 Experimental design Choose experimental conditions via Bayesian optimal design Expected utility of a design η: U ( η) = U ( θ,η, D) p( D,θ η) dθ dd data D parameters θ Maximize this with respect to η (need to simultaneously integrate and maximize) Good choice for parameter inference: expected information gain (entropy of posterior relative to prior) ( ) = log p θ D,η p( θ) U η ( ) p( D,θ η) dθ dd gpc surrogates and efficient numerical integration make simulation-based optimal design feasible

27 Experimental design Contours of expected information gain in (A, E), with two design variables (T, φ)

28 Summary & thanks Bayesian inference and prediction Stochastic spectral methods to accelerate Bayesian inference From forward UQ to inverse problems Point parameters, spatially distributed parameters (dimensionality) Optimal Bayesian experimental design Support from DOE Office of Advanced Scientific Computing Research (ASCR), Sandia National Laboratories, and KAUST

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Numerical solution of the space-time tempered fractional diffusion equation. finite differences

Numerical solution of the space-time tempered fractional diffusion equation. finite differences Numerical solution of the space-time tempered fractional diffusion equation: Alternatives to finite differences Université catholique de Louvain Belgium emmanuel.hanert@uclouvain.be Which numerical method

More information

Measuring the tracking error of exchange traded funds: an unobserved components approach

Measuring the tracking error of exchange traded funds: an unobserved components approach Measuring the tracking error of exchange traded funds: an unobserved components approach Giuliano De Rossi Quantitative analyst +44 20 7568 3072 UBS Investment Research June 2012 Analyst Certification

More information

L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

More information

Pooling and Meta-analysis. Tony O Hagan

Pooling and Meta-analysis. Tony O Hagan Pooling and Meta-analysis Tony O Hagan Pooling Synthesising prior information from several experts 2 Multiple experts The case of multiple experts is important When elicitation is used to provide expert

More information

Bayesian modeling of inseparable space-time variation in disease risk

Bayesian modeling of inseparable space-time variation in disease risk Bayesian modeling of inseparable space-time variation in disease risk Leonhard Knorr-Held Laina Mercer Department of Statistics UW May 23, 2013 Motivation Area and time-specific disease rates Area and

More information

Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Models vs. Patterns Models A model is a high level, global description of a

More information

Supplement to Regional climate model assessment. using statistical upscaling and downscaling techniques

Supplement to Regional climate model assessment. using statistical upscaling and downscaling techniques Supplement to Regional climate model assessment using statistical upscaling and downscaling techniques Veronica J. Berrocal 1,2, Peter F. Craigmile 3, Peter Guttorp 4,5 1 Department of Biostatistics, University

More information

Model-based Synthesis. Tony O Hagan

Model-based Synthesis. Tony O Hagan Model-based Synthesis Tony O Hagan Stochastic models Synthesising evidence through a statistical model 2 Evidence Synthesis (Session 3), Helsinki, 28/10/11 Graphical modelling The kinds of models that

More information

The Conquest of U.S. Inflation: Learning and Robustness to Model Uncertainty

The Conquest of U.S. Inflation: Learning and Robustness to Model Uncertainty The Conquest of U.S. Inflation: Learning and Robustness to Model Uncertainty Timothy Cogley and Thomas Sargent University of California, Davis and New York University and Hoover Institution Conquest of

More information

Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering

Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Department of Industrial Engineering and Management Sciences Northwestern University September 15th, 2014

More information

Mälardalen University

Mälardalen University Mälardalen University http://www.mdh.se 1/38 Value at Risk and its estimation Anatoliy A. Malyarenko Department of Mathematics & Physics Mälardalen University SE-72 123 Västerås,Sweden email: amo@mdh.se

More information

Christfried Webers. Canberra February June 2015

Christfried Webers. Canberra February June 2015 c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

More information

Treatment and analysis of data Applied statistics Lecture 3: Sampling and descriptive statistics

Treatment and analysis of data Applied statistics Lecture 3: Sampling and descriptive statistics Treatment and analysis of data Applied statistics Lecture 3: Sampling and descriptive statistics Topics covered: Parameters and statistics Sample mean and sample standard deviation Order statistics and

More information

Speeding Up Monte Carlo Simulation Using An Emulator

Speeding Up Monte Carlo Simulation Using An Emulator Speeding Up Monte Carlo Simulation Using An Emulator A.K. Tyagi, W.H.A. Schilders, X. Jonsson, T.G.J. Beelen Center for Analysis, Scientific Computing and Applications (CASA), Mentor Graphics R, Grenoble

More information

The Delta Method and Applications

The Delta Method and Applications Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and

More information

CONVERGE Features, Capabilities and Applications

CONVERGE Features, Capabilities and Applications CONVERGE Features, Capabilities and Applications CONVERGE CONVERGE The industry leading CFD code for complex geometries with moving boundaries. Start using CONVERGE and never make a CFD mesh again. CONVERGE

More information

Sufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng

Sufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng Math 541: Statistical Theory II Lecturer: Songfeng Zheng Sufficient Statistics and Exponential Family 1 Statistics and Sufficient Statistics Suppose we have a random sample X 1,, X n taken from a distribution

More information

Statistics Graduate Courses

Statistics Graduate Courses Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

More information

t-h ey are.. r.. v-e.. an t.. bo th for th e.. s-t ru c.. tur alan d.. t he.. quait.. ta ive un de \ centerline {. Inrodu \quad c t i o na r s

t-h ey are.. r.. v-e.. an t.. bo th for th e.. s-t ru c.. tur alan d.. t he.. quait.. ta ive un de \ centerline {. Inrodu \quad c t i o na r s - - - I - ˆ - - - q I q I ˆ I q R R q I q q I R R R R - - - ˆ @ & q k 7 q k O q k 8 & q & k P S q k q k ˆ q k 3 q k ˆ A & [ 7 O [8 P & S & [ [ 3 ˆ A @ q ˆ U q - : U [ U φ : U D φ φ Dφ A ψ A : I SS N :

More information

Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability

Chapter 1. Probability, Random Variables and Expectations. 1.1 Axiomatic Probability Chapter 1 Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (1999. Other treatments of probability theory include Gallant (1997, Casella & Berger

More information

Lecture 8: Random Walk vs. Brownian Motion, Binomial Model vs. Log-Normal Distribution

Lecture 8: Random Walk vs. Brownian Motion, Binomial Model vs. Log-Normal Distribution Lecture 8: Random Walk vs. Brownian Motion, Binomial Model vs. Log-ormal Distribution October 4, 200 Limiting Distribution of the Scaled Random Walk Recall that we defined a scaled simple random walk last

More information

Gaussian Classifiers CS498

Gaussian Classifiers CS498 Gaussian Classifiers CS498 Today s lecture The Gaussian Gaussian classifiers A slightly more sophisticated classifier Nearest Neighbors We can classify with nearest neighbors x m 1 m 2 Decision boundary

More information

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

More information

A crash course in probability and Naïve Bayes classification

A crash course in probability and Naïve Bayes classification Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

More information

Option pricing in a quadratic variance swap model

Option pricing in a quadratic variance swap model Elise Gourier School of Economics and Finance Queen Mary University of London Joint work with Damir Filipović and Loriano Mancini Swiss Finance Institute and EPFL Workshop "Mathematical Finance beyond

More information

Tutorial on Markov Chain Monte Carlo

Tutorial on Markov Chain Monte Carlo Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,

More information

Introduction to ARMA Models

Introduction to ARMA Models Statistics 910, #8 1 Overview Introduction to ARMA Models 1. Modeling paradigm 2. Review stationary linear processes 3. ARMA processes 4. Stationarity of ARMA processes 5. Identifiability of ARMA processes

More information

Mathematical Background

Mathematical Background Appendix A Mathematical Background A.1 Joint, Marginal and Conditional Probability Let the n (discrete or continuous) random variables y 1,..., y n have a joint joint probability probability p(y 1,...,

More information

Basics of Point-Referenced Data Models

Basics of Point-Referenced Data Models Basics of Point-Referenced Data Models Basic tool is that of a spatial process, {Y (s),s D}, where D R r Note that time series follows this approach with r = 1; we will usually have r = 2 or 3 We begin

More information

BayesX - Software for Bayesian Inference in Structured Additive Regression

BayesX - Software for Bayesian Inference in Structured Additive Regression BayesX - Software for Bayesian Inference in Structured Additive Regression Thomas Kneib Faculty of Mathematics and Economics, University of Ulm Department of Statistics, Ludwig-Maximilians-University Munich

More information

Gaussian Processes in Machine Learning

Gaussian Processes in Machine Learning Gaussian Processes in Machine Learning Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany carl@tuebingen.mpg.de WWW home page: http://www.tuebingen.mpg.de/ carl

More information

Bayesian probability: P. State of the World: X. P(X your information I)

Bayesian probability: P. State of the World: X. P(X your information I) Bayesian probability: P State of the World: X P(X your information I) 1 First example: bag of balls Every probability is conditional to your background knowledge I : P(A I) What is the (your) probability

More information

Advanced Signal Processing and Digital Noise Reduction

Advanced Signal Processing and Digital Noise Reduction Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New

More information

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10 Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay

Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay Lecture 1: Univariate Time Series B41910: Autumn Quarter, 2008, by Mr. Ruey S. Tsay 1 Some Basic Concepts 1. Time Series: A sequence of random variables measuring certain quantity of interest over time.

More information

> plot(exp.btgpllm, main = "treed GP LLM,", proj = c(1)) > plot(exp.btgpllm, main = "treed GP LLM,", proj = c(2)) quantile diff (error)

> plot(exp.btgpllm, main = treed GP LLM,, proj = c(1)) > plot(exp.btgpllm, main = treed GP LLM,, proj = c(2)) quantile diff (error) > plot(exp.btgpllm, main = "treed GP LLM,", proj = c(1)) > plot(exp.btgpllm, main = "treed GP LLM,", proj = c(2)) 0.4 0.2 0.0 0.2 0.4 treed GP LLM, mean treed GP LLM, 0.00 0.05 0.10 0.15 0.20 x1 x1 0.4

More information

Introduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011

Introduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 Introduction to Machine Learning Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 1 Outline 1. What is machine learning? 2. The basic of machine learning 3. Principles and effects of machine learning

More information

Petro-elastic and lithology-fluid inversion from seismic data state-of-the-art and new opportunities Summary

Petro-elastic and lithology-fluid inversion from seismic data state-of-the-art and new opportunities Summary Petro-elastic and lithology-fluid inversion from seismic data state-of-the-art and new opportunities Per Avseth (NTNU/Odin Petroleum) and Kjartan Rimstad (NTNU) Summary During the last couple of decades,

More information

MUSICAL INSTRUMENT FAMILY CLASSIFICATION

MUSICAL INSTRUMENT FAMILY CLASSIFICATION MUSICAL INSTRUMENT FAMILY CLASSIFICATION Ricardo A. Garcia Media Lab, Massachusetts Institute of Technology 0 Ames Street Room E5-40, Cambridge, MA 039 USA PH: 67-53-0 FAX: 67-58-664 e-mail: rago @ media.

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

Statistics - Written Examination MEC Students - BOVISA

Statistics - Written Examination MEC Students - BOVISA Statistics - Written Examination MEC Students - BOVISA Prof.ssa A. Guglielmi 26.0.2 All rights reserved. Legal action will be taken against infringement. Reproduction is prohibited without prior consent.

More information

Using Big Data and Efficient Methods to Capture Stochasticity for Calibration of Macroscopic Traffic Simulation Models

Using Big Data and Efficient Methods to Capture Stochasticity for Calibration of Macroscopic Traffic Simulation Models CIVIL AND ENVIRONMENTAL ENGINEERING Using Big Data and Efficient Methods to Capture Stochasticity for Calibration of Macroscopic Traffic Simulation Models Sandeep Mudigonda 1, Kaan Ozbay 2 1 Department

More information

INDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition)

INDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition) INDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition) Abstract Indirect inference is a simulation-based method for estimating the parameters of economic models. Its

More information

TRANSFORMATIONS OF RANDOM VARIABLES

TRANSFORMATIONS OF RANDOM VARIABLES TRANSFORMATIONS OF RANDOM VARIABLES 1. INTRODUCTION 1.1. Definition. We are often interested in the probability distributions or densities of functions of one or more random variables. Suppose we have

More information

Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

More information

Statistical Models in Data Mining

Statistical Models in Data Mining Statistical Models in Data Mining Sargur N. Srihari University at Buffalo The State University of New York Department of Computer Science and Engineering Department of Biostatistics 1 Srihari Flood of

More information

1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates

1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates The maximum-likelihood method Volker Blobel University of Hamburg March 2005 1. The maximum likelihood principle 2. Properties of maximum-likelihood estimates Keys during display: enter = next page; =

More information

Systematic risk of of mortality on on an an annuity plan

Systematic risk of of mortality on on an an annuity plan www.winter-associes.fr Systematic risk of of mortality on on an an annuity plan Version Version 1.0 1.0 Frédéric PLANCHET / Marc JUILLARD / Laurent FAUCILLON ISFA WINTER & Associés July 2006 Page 1 Motivations

More information

Logistic Regression (1/24/13)

Logistic Regression (1/24/13) STA63/CBB540: Statistical methods in computational biology Logistic Regression (/24/3) Lecturer: Barbara Engelhardt Scribe: Dinesh Manandhar Introduction Logistic regression is model for regression used

More information

Linear Classification. Volker Tresp Summer 2015

Linear Classification. Volker Tresp Summer 2015 Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

More information

Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

More information

CS540 Machine learning Lecture 14 Mixtures, EM, Non-parametric models

CS540 Machine learning Lecture 14 Mixtures, EM, Non-parametric models CS540 Machine learning Lecture 14 Mixtures, EM, Non-parametric models Outline Mixture models EM for mixture models K means clustering Conditional mixtures Kernel density estimation Kernel regression GMM

More information

Section 5. Stan for Big Data. Bob Carpenter. Columbia University

Section 5. Stan for Big Data. Bob Carpenter. Columbia University Section 5. Stan for Big Data Bob Carpenter Columbia University Part I Overview Scaling and Evaluation data size (bytes) 1e18 1e15 1e12 1e9 1e6 Big Model and Big Data approach state of the art big model

More information

Bead moving along a thin, rigid, wire.

Bead moving along a thin, rigid, wire. Bead moving along a thin, rigid, wire. odolfo. osales, Department of Mathematics, Massachusetts Inst. of Technology, Cambridge, Massachusetts, MA 02139 October 17, 2004 Abstract An equation describing

More information

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

Dakota Software Training

Dakota Software Training SAND2015-6866 TR Dakota Software Training Surrogate Models http://dakota.sandia.gov Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned

More information

Paid-Incurred Chain Claims Reserving Method

Paid-Incurred Chain Claims Reserving Method Paid-Incurred Chain Claims Reserving Method Michael Merz Mario V. Wüthrich Abstract We present a novel stochastic model for claims reserving that allows to combine claims payments and incurred losses information.

More information

INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

More information

Cointegration. Basic Ideas and Key results. Egon Zakrajšek Division of Monetary Affairs Federal Reserve Board

Cointegration. Basic Ideas and Key results. Egon Zakrajšek Division of Monetary Affairs Federal Reserve Board Cointegration Basic Ideas and Key results Egon Zakrajšek Division of Monetary Affairs Federal Reserve Board Summer School in Financial Mathematics Faculty of Mathematics & Physics University of Ljubljana

More information

6.4 The Infinitely Many Alleles Model

6.4 The Infinitely Many Alleles Model 6.4. THE INFINITELY MANY ALLELES MODEL 93 NE µ [F (X N 1 ) F (µ)] = + 1 i

More information

APPLIED MATHEMATICS ADVANCED LEVEL

APPLIED MATHEMATICS ADVANCED LEVEL APPLIED MATHEMATICS ADVANCED LEVEL INTRODUCTION This syllabus serves to examine candidates knowledge and skills in introductory mathematical and statistical methods, and their applications. For applications

More information

DYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson

DYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson c 1999 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or

More information

Missing Data Approach to Handle Delayed Outcomes in Early

Missing Data Approach to Handle Delayed Outcomes in Early Missing Data Approach to Handle Delayed Outcomes in Early Phase Clinical Trials Department of Biostatistics The University of Texas, MD Anderson Cancer Center Outline Introduction Method Conclusion Background

More information

Stat260: Bayesian Modeling and Inference Lecture Date: February 1, Lecture 3

Stat260: Bayesian Modeling and Inference Lecture Date: February 1, Lecture 3 Stat26: Bayesian Modeling and Inference Lecture Date: February 1, 21 Lecture 3 Lecturer: Michael I. Jordan Scribe: Joshua G. Schraiber 1 Decision theory Recall that decision theory provides a quantification

More information

Probability, Random Variables and Expectations

Probability, Random Variables and Expectations Chapter Probability, Random Variables and Expectations Note: The primary reference for these notes is Mittelhammer (999). Other treatments of probability theory include Gallant (997), Casella & Berger

More information

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

More information

Lecture Note of Bus 41202, Spring 2012: Stochastic Diffusion & Option Pricing

Lecture Note of Bus 41202, Spring 2012: Stochastic Diffusion & Option Pricing Lecture Note of Bus 41202, Spring 2012: Stochastic Diffusion & Option Pricing Key concept: Ito s lemma Stock Options: A contract giving its holder the right, but not obligation, to trade shares of a common

More information

Enhancing Parallel Pattern Search Optimization with a Gaussian Process Oracle

Enhancing Parallel Pattern Search Optimization with a Gaussian Process Oracle Enhancing Parallel Pattern Search Optimization with a Gaussian Process Oracle Genetha A. Gray, Monica Martinez-Canales Computational Sciences & Mathematics Research Department, Sandia National Laboratories,

More information

Extreme Value Modeling for Detection and Attribution of Climate Extremes

Extreme Value Modeling for Detection and Attribution of Climate Extremes Extreme Value Modeling for Detection and Attribution of Climate Extremes Jun Yan, Yujing Jiang Joint work with Zhuo Wang, Xuebin Zhang Department of Statistics, University of Connecticut February 2, 2016

More information

Methods of Data Analysis Working with probability distributions

Methods of Data Analysis Working with probability distributions Methods of Data Analysis Working with probability distributions Week 4 1 Motivation One of the key problems in non-parametric data analysis is to create a good model of a generating probability distribution,

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 10 Boundary Value Problems for Ordinary Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

More information

Introduction to Monte-Carlo Methods

Introduction to Monte-Carlo Methods Introduction to Monte-Carlo Methods Bernard Lapeyre Halmstad January 2007 Monte-Carlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities

More information

Linear regression methods for large n and streaming data

Linear regression methods for large n and streaming data Linear regression methods for large n and streaming data Large n and small or moderate p is a fairly simple problem. The sufficient statistic for β in OLS (and ridge) is: The concept of sufficiency is

More information

38. STATISTICS. 38. Statistics 1. 38.1. Fundamental concepts

38. STATISTICS. 38. Statistics 1. 38.1. Fundamental concepts 38. Statistics 1 38. STATISTICS Revised September 2013 by G. Cowan (RHUL). This chapter gives an overview of statistical methods used in high-energy physics. In statistics, we are interested in using a

More information

Maximum likelihood estimation of mean reverting processes

Maximum likelihood estimation of mean reverting processes Maximum likelihood estimation of mean reverting processes José Carlos García Franco Onward, Inc. jcpollo@onwardinc.com Abstract Mean reverting processes are frequently used models in real options. For

More information

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2015 CS 551, Fall 2015

More information

Numerical Methods For Image Restoration

Numerical Methods For Image Restoration Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:

More information

AN EXPANSION FORMULA FOR FRACTIONAL DERIVATIVES AND ITS APPLICATION. Abstract

AN EXPANSION FORMULA FOR FRACTIONAL DERIVATIVES AND ITS APPLICATION. Abstract AN EXPANSION FORMULA FOR FRACTIONAL DERIVATIVES AND ITS APPLICATION T. M. Atanackovic 1 and B. Stankovic 2 Dedicated to Professor Ivan H. Dimovski on the occasion of his 7th birthday Abstract An expansion

More information

Passive Optical Resonators

Passive Optical Resonators Passive Optical Resonators Optical Cavities and Feedback Back mirror I 0 I 1 Laser medium with gain, G Output mirror I 3 R = 100% R < 100% I 2 Cavities are essential components of the lasers. They provide

More information

Parametric survival models

Parametric survival models Parametric survival models ST3242: Introduction to Survival Analysis Alex Cook October 2008 ST3242 : Parametric survival models 1/17 Last time in survival analysis Reintroduced parametric models Distinguished

More information

TESTING THE ONE-PART FRACTIONAL RESPONSE MODEL AGAINST AN ALTERNATIVE TWO-PART MODEL

TESTING THE ONE-PART FRACTIONAL RESPONSE MODEL AGAINST AN ALTERNATIVE TWO-PART MODEL TESTING THE ONE-PART FRACTIONAL RESPONSE MODEL AGAINST AN ALTERNATIVE TWO-PART MODEL HARALD OBERHOFER AND MICHAEL PFAFFERMAYR WORKING PAPER NO. 2011-01 Testing the One-Part Fractional Response Model against

More information

Simulation-based optimization methods for urban transportation problems. Carolina Osorio

Simulation-based optimization methods for urban transportation problems. Carolina Osorio Simulation-based optimization methods for urban transportation problems Carolina Osorio Civil and Environmental Engineering Department Massachusetts Institute of Technology (MIT) Joint work with: Prof.

More information

Introduction. Independent Component Analysis

Introduction. Independent Component Analysis Independent Component Analysis 1 Introduction Independent component analysis (ICA) is a method for finding underlying factors or components from multivariate (multidimensional) statistical data. What distinguishes

More information

From CFD to computational finance (and back again?)

From CFD to computational finance (and back again?) computational finance p. 1/21 From CFD to computational finance (and back again?) Mike Giles mike.giles@maths.ox.ac.uk Oxford University Mathematical Institute Oxford-Man Institute of Quantitative Finance

More information

The Exponential Family

The Exponential Family The Exponential Family David M. Blei Columbia University November 3, 2015 Definition A probability density in the exponential family has this form where p.x j / D h.x/ expf > t.x/ a./g; (1) is the natural

More information

Learning Gaussian process models from big data. Alan Qi Purdue University Joint work with Z. Xu, F. Yan, B. Dai, and Y. Zhu

Learning Gaussian process models from big data. Alan Qi Purdue University Joint work with Z. Xu, F. Yan, B. Dai, and Y. Zhu Learning Gaussian process models from big data Alan Qi Purdue University Joint work with Z. Xu, F. Yan, B. Dai, and Y. Zhu Machine learning seminar at University of Cambridge, July 4 2012 Data A lot of

More information

4. Introduction to Statistics

4. Introduction to Statistics Statistics for Engineers 4-1 4. Introduction to Statistics Descriptive Statistics Types of data A variate or random variable is a quantity or attribute whose value may vary from one unit of investigation

More information

Common probability distributionsi Math 217/218 Probability and Statistics Prof. D. Joyce, 2016

Common probability distributionsi Math 217/218 Probability and Statistics Prof. D. Joyce, 2016 Introduction. ommon probability distributionsi Math 7/8 Probability and Statistics Prof. D. Joyce, 06 I summarize here some of the more common distributions used in probability and statistics. Some are

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

Probability and Random Variables. Generation of random variables (r.v.)

Probability and Random Variables. Generation of random variables (r.v.) Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly

More information

Statistiek (WISB361)

Statistiek (WISB361) Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17

More information

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data

More information

Examination 110 Probability and Statistics Examination

Examination 110 Probability and Statistics Examination Examination 0 Probability and Statistics Examination Sample Examination Questions The Probability and Statistics Examination consists of 5 multiple-choice test questions. The test is a three-hour examination

More information

Bayesian Functional Data Analysis for Computer Model Validation

Bayesian Functional Data Analysis for Computer Model Validation Bayesian Functional Data Analysis for Computer Model Validation Fei Liu Institute of Statistics and Decision Sciences Duke University Transition Workshop on Development, Assessment and Utilization of Complex

More information

Estimation and Inference in Cointegration Models Economics 582

Estimation and Inference in Cointegration Models Economics 582 Estimation and Inference in Cointegration Models Economics 582 Eric Zivot May 17, 2012 Tests for Cointegration Let the ( 1) vector Y be (1). Recall, Y is cointegrated with 0 cointegrating vectors if there

More information

Spatial Accuracy Assessment of Digital Surface Models: A Probabilistic Approach

Spatial Accuracy Assessment of Digital Surface Models: A Probabilistic Approach Spatial Accuracy Assessment of Digital Surface Models: A Probabilistic Approach André Jalobeanu Centro de Geofisica de Evora, Portugal part of the SPACEFUSION Project (ANR, France) Outline Goals and objectives

More information

33. STATISTICS. 33. Statistics 1

33. STATISTICS. 33. Statistics 1 33. STATISTICS 33. Statistics 1 Revised September 2011 by G. Cowan (RHUL). This chapter gives an overview of statistical methods used in high-energy physics. In statistics, we are interested in using a

More information

More details on the inputs, functionality, and output can be found below.

More details on the inputs, functionality, and output can be found below. Overview: The SMEEACT (Software for More Efficient, Ethical, and Affordable Clinical Trials) web interface (http://research.mdacc.tmc.edu/smeeactweb) implements a single analysis of a two-armed trial comparing

More information