Generalized BIC for Singular Models Factoring through Regular Models

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Generalized BIC for Singular Models Factoring through Regular Models"

Transcription

1 Generalized BIC for Singular Models Factoring through Regular Models Shaowei Lin shaowei/ Department of Mathematics, University of California, Berkeley PhD student (Advisor: Bernd Sturmfels)

2 Abstract The Bayesian Information Criterion (BIC) is an important tool for model selection, but its use is limited only to regular models. Recently, Watanabe [5] generalized the BIC to singular models using ideas from algebraic geometry. In this paper, we compute this generalized BIC for singular models factoring through regular models, relating it to the real log canonical threshold of polynomial fiber ideals.

3 Bayesian Information Criterion The BIC is a score used for model selection. BIC = log L 0 + d 2 log N L 0 : maximum likelihood, d: dimension, N: sample size. However, it applies only for regular models.

4 Regular and Singular Models A model M(U) is regular if it is identifiable p(x u, M) = p(x u, M) x u = u and the Fisher information matrix I(u) is positive definite u. I jk (u) = log p(x u) u j log p(x u) p(x u)dx u k Otherwise, the model is singular. e.g. most exponential families are regular, while most hidden variable models are singular.

5 Factored Models M 2 (Ω) factors through M 1 (U) if for some function u(ω), p(x ω, M 2 ) = p(x u(ω), M 1 ). e.g. parametrized multivariate Gaussian models N(0, Σ(ω)).

6 Generalized BIC In 2001, Watanabe showed that for singular models M(Ω), the BIC generalizes to log L 0 + λ log N (θ 1) log log N, where λ is the smallest pole of the zeta function ζ(z) = K(ω) z ϕ(ω)dω, z C Ω and θ its multiplicity. Here, ϕ(ω) is a prior on Ω, and K(ω) the Kullback-Leibler distance to the true distribution. We compute (λ, θ) by monomializing K(ω). (a.k.a. resolution of singularities)

7 Key Idea: Fiber Ideals Even if M(Ω) is parametrized by simple polynomials, the Kullback function K(ω) is non-polynomial and difficult to monomialize. We show that if M(Ω) factors through a regular model, we can define a polynomial fiber ideal and compute (λ, θ) by monomializing this ideal.

8 Real Log Canonical Thresholds Given an ideal I = f 1,...,f r generated by polynomials, define the real log canonical threshold (λ,θ) to be the smallest pole λ and multiplicity θ of the zeta function ζ(z) = (f 1 (ω) f r (ω) 2 ) z/2 dω, z C. The RLCT is independent of the choice of generators f 1,...,f r, and can be computed by monomializing I. For monomial ideals, we find RLCTs using a geometriccombinatorial tool involving Newton polyhedra.

9 Main Result Let M 2 (Ω) factor through a regular model M 1 (U) via u(ω). Given N i.i.d. samples, let their M.L.E. in M 1 be û U. Let Z(N) be the marginal likelihood of the data given M 2. Theorem (L.) If û u(ω), then asymptotically, log Z(N) log L 0 + λ log N (θ 1) log log N where (2λ,θ) is the RLCT of the fiber ideal I = u 1 (ω) û 1,...,u d (ω) û d.

10 Example: Classical BIC Let M(U) be a regular model and û the M.L.E. of the data. Then, (2λ,θ) is the RLCT of the fiber ideal which is just (d, 1). I = u 1 û 1,...,u d û d

11 Example: Discrete Models Let M(Ω) be a model on discrete space {1, 2,...,k} with polynomial state probabilities p 1 (ω),...,p k (ω). Let q 1,...,q k be relative frequencies of the data. If q p(ω), then (2λ,θ) is the RLCT of the fiber ideal I = p 1 (ω) q 1,...,p k (ω) q k.

12 Example: Gaussian Models Let N(µ(Ω), Σ(Ω)) be a multivariate Gaussian model with sample mean ˆµ and covariance matrix ˆΣ. If ˆµ µ(ω) and ˆΣ Σ(Ω), then (2λ,θ) is the RLCT of the fiber ideal I = µ(ω) ˆµ, Σ(ω) ˆΣ.

13 Acknowledgements Many thanks go to Mathias Drton and Sumio Watanabe for enlightening discussions and key ideas.

14 References 1. V. I. Arnol d, S. M. Guseĭn-Zade and A. N. Varchenko: Singularities of Differentiable Maps, Vol. II, Birkhäuser, Boston, H. Hironaka: Resolution of singularities of an algebraic variety over a field of characteristic zero I, II, Ann. of Math. (2) 79 (1964) S. Lin: Asymptotic Approximation of Marginal Likelihood Integrals, preprint arxiv: (2010). 4. S. Watanabe: Algebraic analysis for nonidentifiable learning machines, Neural Computation 13 (2001) S. Watanabe: Algebraic Geometry and Statistical Learning Theory, Cambridge Monographs on Applied and Computational Mathematics 25, Cambridge University Press, Cambridge, G. Schwarz: Estimating the Dimension of a Model, Annals of Statistics 6 (1978)

Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

More information

Linear Classification. Volker Tresp Summer 2015

Linear Classification. Volker Tresp Summer 2015 Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

More information

A COUNTEREXAMPLE FOR SUBADDITIVITY OF MULTIPLIER IDEALS ON TORIC VARIETIES

A COUNTEREXAMPLE FOR SUBADDITIVITY OF MULTIPLIER IDEALS ON TORIC VARIETIES Communications in Algebra, 40: 1618 1624, 2012 Copyright Taylor & Francis Group, LLC ISSN: 0092-7872 print/1532-4125 online DOI: 10.1080/00927872.2011.552084 A COUNTEREXAMPLE FOR SUBADDITIVITY OF MULTIPLIER

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

1. Students will demonstrate an understanding of the real number system as evidenced by classroom activities and objective tests

1. Students will demonstrate an understanding of the real number system as evidenced by classroom activities and objective tests MATH 102/102L Inter-Algebra/Lab Properties of the real number system, factoring, linear and quadratic equations polynomial and rational expressions, inequalities, systems of equations, exponents, radicals,

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Package MixGHD. June 26, 2015

Package MixGHD. June 26, 2015 Type Package Package MixGHD June 26, 2015 Title Model Based Clustering, Classification and Discriminant Analysis Using the Mixture of Generalized Hyperbolic Distributions Version 1.7 Date 2015-6-15 Author

More information

Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma

Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma Shinto Eguchi a, and John Copas b a Institute of Statistical Mathematics and Graduate University of Advanced Studies, Minami-azabu

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Gaussian Mixture Models Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique

More information

MATH 590: Meshfree Methods

MATH 590: Meshfree Methods MATH 590: Meshfree Methods Chapter 7: Conditionally Positive Definite Functions Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 fasshauer@iit.edu MATH 590 Chapter

More information

Tropical Mathematics. An Interesting and Useful Variant of Ordinary Arithmetic. John Polking. Rice University.

Tropical Mathematics. An Interesting and Useful Variant of Ordinary Arithmetic. John Polking. Rice University. Tropical Mathematics An Interesting and Useful Variant of Ordinary Arithmetic John Polking polking@rice.edu Rice University Tropical Mathematics A new mathematics Tropical Mathematics A new mathematics

More information

CPM High Schools California Standards Test (CST) Results for 2004-2010

CPM High Schools California Standards Test (CST) Results for 2004-2010 CPM High California Standards Test (CST) Results for 2004-2010 The tables below show a comparison between CPM high schools and all high schools in California based on the percentage of students who scored

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

More information

MATHEMATICS. Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Degree Requirements

MATHEMATICS. Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Degree Requirements MATHEMATICS Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Paul Feit, PhD Dr. Paul Feit is Professor of Mathematics and Coordinator for Mathematics.

More information

Algebraic Computation Models. Algebraic Computation Models

Algebraic Computation Models. Algebraic Computation Models Algebraic Computation Models Ζυγομήτρος Ευάγγελος ΜΠΛΑ 201118 Φεβρουάριος, 2013 Reasons for using algebraic models of computation The Turing machine model captures computations on bits. Many algorithms

More information

Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering

Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Department of Industrial Engineering and Management Sciences Northwestern University September 15th, 2014

More information

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2015 CS 551, Fall 2015

More information

L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

More information

Clustering - example. Given some data x i X Find a partitioning of the data into k disjunctive clusters Example: k-means clustering

Clustering - example. Given some data x i X Find a partitioning of the data into k disjunctive clusters Example: k-means clustering Clustering - example Graph Mining and Graph Kernels Given some data x i X Find a partitioning of the data into k disjunctive clusters Example: k-means clustering x!!!!8!!! 8 x 1 1 Clustering - example

More information

Singularities and Newton Polygons

Singularities and Newton Polygons Publ. RIMS, Kyoto Univ. 12 Suppl. (1977), 123-129. Singularities and Newton Polygons by Jean-Michel KANTOR* It was conjectured by V. I. Arnold that it should be possible to express all "reasonable" invariants

More information

Section 5. Stan for Big Data. Bob Carpenter. Columbia University

Section 5. Stan for Big Data. Bob Carpenter. Columbia University Section 5. Stan for Big Data Bob Carpenter Columbia University Part I Overview Scaling and Evaluation data size (bytes) 1e18 1e15 1e12 1e9 1e6 Big Model and Big Data approach state of the art big model

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

THE MULTIVARIATE ANALYSIS RESEARCH GROUP. Carles M Cuadras Departament d Estadística Facultat de Biologia Universitat de Barcelona

THE MULTIVARIATE ANALYSIS RESEARCH GROUP. Carles M Cuadras Departament d Estadística Facultat de Biologia Universitat de Barcelona THE MULTIVARIATE ANALYSIS RESEARCH GROUP Carles M Cuadras Departament d Estadística Facultat de Biologia Universitat de Barcelona The set of statistical methods known as Multivariate Analysis covers a

More information

Generating Gaussian Mixture Models by Model Selection For Speech Recognition

Generating Gaussian Mixture Models by Model Selection For Speech Recognition Generating Gaussian Mixture Models by Model Selection For Speech Recognition Kai Yu F06 10-701 Final Project Report kaiy@andrew.cmu.edu Abstract While all modern speech recognition systems use Gaussian

More information

The Exponential Family

The Exponential Family The Exponential Family David M. Blei Columbia University November 3, 2015 Definition A probability density in the exponential family has this form where p.x j / D h.x/ expf > t.x/ a./g; (1) is the natural

More information

Parametric or nonparametric: the FIC approach for stationary time series

Parametric or nonparametric: the FIC approach for stationary time series Parametric or nonparametric: the FIC approach for stationary time series Gudmund Horn Hermansen* Department of Mathematics, University of Oslo, Oslo, Norway - gudmunhh@math.uio.no Nils Lid Hjort Department

More information

STUDENT LEARNING OUTCOMES FOR THE SANTIAGO CANYON COLLEGE MATHEMATICS DEPARTMENT (Last Revised 8/20/14)

STUDENT LEARNING OUTCOMES FOR THE SANTIAGO CANYON COLLEGE MATHEMATICS DEPARTMENT (Last Revised 8/20/14) STUDENT LEARNING OUTCOMES FOR THE SANTIAGO CANYON COLLEGE MATHEMATICS DEPARTMENT (Last Revised 8/20/14) Department SLOs: Upon completion of any course in Mathematics the student will be able to: 1. Create

More information

Message-passing sequential detection of multiple change points in networks

Message-passing sequential detection of multiple change points in networks Message-passing sequential detection of multiple change points in networks Long Nguyen, Arash Amini Ram Rajagopal University of Michigan Stanford University ISIT, Boston, July 2012 Nguyen/Amini/Rajagopal

More information

Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.

Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features

More information

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued). MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

More information

Bayesian Classifier for a Gaussian Distribution, Decision Surface Equation, with Application

Bayesian Classifier for a Gaussian Distribution, Decision Surface Equation, with Application Iraqi Journal of Statistical Science (18) 2010 p.p. [35-58] Bayesian Classifier for a Gaussian Distribution, Decision Surface Equation, with Application ABSTRACT Nawzad. M. Ahmad * Bayesian decision theory

More information

ON THE DEGREES OF FREEDOM IN RICHLY PARAMETERISED MODELS

ON THE DEGREES OF FREEDOM IN RICHLY PARAMETERISED MODELS COMPSTAT 2004 Symposium c Physica-Verlag/Springer 2004 ON THE DEGREES OF FREEDOM IN RICHLY PARAMETERISED MODELS Salvatore Ingrassia and Isabella Morlini Key words: Richly parameterised models, small data

More information

Statistics Graduate Courses

Statistics Graduate Courses Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

More information

COURSE SYLLABUS Pre-Calculus A/B Last Modified: April 2015

COURSE SYLLABUS Pre-Calculus A/B Last Modified: April 2015 COURSE SYLLABUS Pre-Calculus A/B Last Modified: April 2015 Course Description: In this year-long Pre-Calculus course, students will cover topics over a two semester period (as designated by A and B sections).

More information

Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure

Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure Belyaev Mikhail 1,2,3, Burnaev Evgeny 1,2,3, Kapushev Yermek 1,2 1 Institute for Information Transmission

More information

C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)}

C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)} C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)} 1. EES 800: Econometrics I Simple linear regression and correlation analysis. Specification and estimation of a regression model. Interpretation of regression

More information

Bayesian Classification

Bayesian Classification CS 650: Computer Vision Bryan S. Morse BYU Computer Science Statistical Basis Training: Class-Conditional Probabilities Suppose that we measure features for a large training set taken from class ω i. Each

More information

Factoring Cubic Polynomials

Factoring Cubic Polynomials Factoring Cubic Polynomials Robert G. Underwood 1. Introduction There are at least two ways in which using the famous Cardano formulas (1545) to factor cubic polynomials present more difficulties than

More information

CONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Relations Between Time Domain and Frequency Domain Prediction Error Methods - Tomas McKelvey

CONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Relations Between Time Domain and Frequency Domain Prediction Error Methods - Tomas McKelvey COTROL SYSTEMS, ROBOTICS, AD AUTOMATIO - Vol. V - Relations Between Time Domain and Frequency Domain RELATIOS BETWEE TIME DOMAI AD FREQUECY DOMAI PREDICTIO ERROR METHODS Tomas McKelvey Signal Processing,

More information

M5MS09. Graphical Modelling

M5MS09. Graphical Modelling Course: MMS09 Setter: Walden Checker: Ginzberg Editor: Calderhead External: Wood Date: April, 0 MSc EXAMINATIONS (STATISTICS) May-June 0 MMS09 Graphical Modelling Setter s signature Checker s signature

More information

Primary Decomposition

Primary Decomposition Primary Decomposition 1 Prime and Primary Ideals solutions defined by prime ideals are irreducible 2 Finite Dimensional Systems modeling of gene regulatory networks 3 Primary Decomposition definition and

More information

ISU Department of Mathematics. Graduate Examination Policies and Procedures

ISU Department of Mathematics. Graduate Examination Policies and Procedures ISU Department of Mathematics Graduate Examination Policies and Procedures There are four primary criteria to be used in evaluating competence on written or oral exams. 1. Knowledge Has the student demonstrated

More information

A HYBRID GENETIC ALGORITHM FOR THE MAXIMUM LIKELIHOOD ESTIMATION OF MODELS WITH MULTIPLE EQUILIBRIA: A FIRST REPORT

A HYBRID GENETIC ALGORITHM FOR THE MAXIMUM LIKELIHOOD ESTIMATION OF MODELS WITH MULTIPLE EQUILIBRIA: A FIRST REPORT New Mathematics and Natural Computation Vol. 1, No. 2 (2005) 295 303 c World Scientific Publishing Company A HYBRID GENETIC ALGORITHM FOR THE MAXIMUM LIKELIHOOD ESTIMATION OF MODELS WITH MULTIPLE EQUILIBRIA:

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK zoubin@gatsby.ucl.ac.uk http://www.gatsby.ucl.ac.uk/~zoubin September 16, 2004 Abstract We give

More information

Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8

Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8 Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Computational algebraic geometry

Computational algebraic geometry Computational algebraic geometry Learning coefficients via symbolic and numerical methods Anton Leykin Georgia Tech AIM, Palo Alto, December 2011 Let k be a field (R or C). Ideals varieties Ideal in R

More information

Probabilistic Graphical Models Final Exam

Probabilistic Graphical Models Final Exam Introduction to Probabilistic Graphical Models Final Exam, March 2007 1 Probabilistic Graphical Models Final Exam Please fill your name and I.D.: Name:... I.D.:... Duration: 3 hours. Guidelines: 1. The

More information

Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classifica6on

Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classifica6on Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classifica6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output

More information

Factorial experimental designs and generalized linear models

Factorial experimental designs and generalized linear models Statistics & Operations Research Transactions SORT 29 (2) July-December 2005, 249-268 ISSN: 1696-2281 www.idescat.net/sort Statistics & Operations Research c Institut d Estadística de Transactions Catalunya

More information

Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

More information

Item selection by latent class-based methods: an application to nursing homes evaluation

Item selection by latent class-based methods: an application to nursing homes evaluation Item selection by latent class-based methods: an application to nursing homes evaluation Francesco Bartolucci, Giorgio E. Montanari, Silvia Pandolfi 1 Department of Economics, Finance and Statistics University

More information

i=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by

i=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a p-dimensional parameter

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

More information

Natural Conjugate Gradient in Variational Inference

Natural Conjugate Gradient in Variational Inference Natural Conjugate Gradient in Variational Inference Antti Honkela, Matti Tornio, Tapani Raiko, and Juha Karhunen Adaptive Informatics Research Centre, Helsinki University of Technology P.O. Box 5400, FI-02015

More information

Anomaly detection for Big Data, networks and cyber-security

Anomaly detection for Big Data, networks and cyber-security Anomaly detection for Big Data, networks and cyber-security Patrick Rubin-Delanchy University of Bristol & Heilbronn Institute for Mathematical Research Joint work with Nick Heard (Imperial College London),

More information

Generalization of Chisquare Goodness-of-Fit Test for Binned Data Using Saturated Models, with Application to Histograms

Generalization of Chisquare Goodness-of-Fit Test for Binned Data Using Saturated Models, with Application to Histograms Generalization of Chisquare Goodness-of-Fit Test for Binned Data Using Saturated Models, with Application to Histograms Robert D. Cousins Dept. of Physics and Astronomy University of California, Los Angeles,

More information

A hidden Markov model for criminal behaviour classification

A hidden Markov model for criminal behaviour classification RSS2004 p.1/19 A hidden Markov model for criminal behaviour classification Francesco Bartolucci, Institute of economic sciences, Urbino University, Italy. Fulvia Pennoni, Department of Statistics, University

More information

Big Data, Statistics, and the Internet

Big Data, Statistics, and the Internet Big Data, Statistics, and the Internet Steven L. Scott April, 4 Steve Scott (Google) Big Data, Statistics, and the Internet April, 4 / 39 Summary Big data live on more than one machine. Computing takes

More information

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

More information

Generalized Linear Model Theory

Generalized Linear Model Theory Appendix B Generalized Linear Model Theory We describe the generalized linear model as formulated by Nelder and Wedderburn (1972), and discuss estimation of the parameters and tests of hypotheses. B.1

More information

Row Ideals and Fibers of Morphisms

Row Ideals and Fibers of Morphisms Michigan Math. J. 57 (2008) Row Ideals and Fibers of Morphisms David Eisenbud & Bernd Ulrich Affectionately dedicated to Mel Hochster, who has been an inspiration to us for many years, on the occasion

More information

Research Article The General Traveling Wave Solutions of the Fisher Equation with Degree Three

Research Article The General Traveling Wave Solutions of the Fisher Equation with Degree Three Advances in Mathematical Physics Volume 203, Article ID 65798, 5 pages http://dx.doi.org/0.55/203/65798 Research Article The General Traveling Wave Solutions of the Fisher Equation with Degree Three Wenjun

More information

Method of Stationary phase. Reference: Hormander vol I. Steve Zelditch Department of Mathematics Northwestern University

Method of Stationary phase. Reference: Hormander vol I. Steve Zelditch Department of Mathematics Northwestern University Method of Stationary phase Reference: Hormander vol I Steve Zelditch Department of Mathematics Northwestern University 1 Method of Stationary Phase We now describe the method of stationary phase, which

More information

STATISTICA Formula Guide: Logistic Regression. Table of Contents

STATISTICA Formula Guide: Logistic Regression. Table of Contents : Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

More information

A GAUSSIAN COMPOUND DECISION BAKEOFF

A GAUSSIAN COMPOUND DECISION BAKEOFF A GAUSSIAN COMPOUND DECISION BAKEOFF ROGER KOENKER Abstract. A nonparametric mixture model approach to empirical Bayes compound decisions for the Gaussian location model is compared with a parametric empirical

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Mathematics. Course List

Mathematics. Course List Mathematics 1 Mathematics Course List Code Course Title Unit I. Ph.D. Programme (Full-time and Part-time); II. M.Phil. Programme (Full-time and Part-time); and III. M.Sc. Programme (Full-time and Part-time)

More information

Bayesian logistic betting strategy against probability forecasting. Akimichi Takemura, Univ. Tokyo. November 12, 2012

Bayesian logistic betting strategy against probability forecasting. Akimichi Takemura, Univ. Tokyo. November 12, 2012 Bayesian logistic betting strategy against probability forecasting Akimichi Takemura, Univ. Tokyo (joint with Masayuki Kumon, Jing Li and Kei Takeuchi) November 12, 2012 arxiv:1204.3496. To appear in Stochastic

More information

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i ) Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

More information

On Gauss s characterization of the normal distribution

On Gauss s characterization of the normal distribution Bernoulli 13(1), 2007, 169 174 DOI: 10.3150/07-BEJ5166 On Gauss s characterization of the normal distribution ADELCHI AZZALINI 1 and MARC G. GENTON 2 1 Department of Statistical Sciences, University of

More information

Max-Min Representation of Piecewise Linear Functions

Max-Min Representation of Piecewise Linear Functions Beiträge zur Algebra und Geometrie Contributions to Algebra and Geometry Volume 43 (2002), No. 1, 297-302. Max-Min Representation of Piecewise Linear Functions Sergei Ovchinnikov Mathematics Department,

More information

GLM, insurance pricing & big data: paying attention to convergence issues.

GLM, insurance pricing & big data: paying attention to convergence issues. GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK - michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.

More information

Penalized Splines - A statistical Idea with numerous Applications...

Penalized Splines - A statistical Idea with numerous Applications... Penalized Splines - A statistical Idea with numerous Applications... Göran Kauermann Ludwig-Maximilians-University Munich Graz 7. September 2011 1 Penalized Splines - A statistical Idea with numerous Applications...

More information

A crash course in probability and Naïve Bayes classification

A crash course in probability and Naïve Bayes classification Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

More information

Diploma Plus in Certificate in Advanced Engineering

Diploma Plus in Certificate in Advanced Engineering Diploma Plus in Certificate in Advanced Engineering Mathematics New Syllabus from April 2011 Ngee Ann Polytechnic / School of Interdisciplinary Studies 1 I. SYNOPSIS APPENDIX A This course of advanced

More information

Main page. Given f ( x, y) = c we differentiate with respect to x so that

Main page. Given f ( x, y) = c we differentiate with respect to x so that Further Calculus Implicit differentiation Parametric differentiation Related rates of change Small variations and linear approximations Stationary points Curve sketching - asymptotes Curve sketching the

More information

RESONANCES AND BALLS IN OBSTACLE SCATTERING WITH NEUMANN BOUNDARY CONDITIONS

RESONANCES AND BALLS IN OBSTACLE SCATTERING WITH NEUMANN BOUNDARY CONDITIONS RESONANCES AND BALLS IN OBSTACLE SCATTERING WITH NEUMANN BOUNDARY CONDITIONS T. J. CHRISTIANSEN Abstract. We consider scattering by an obstacle in R d, d 3 odd. We show that for the Neumann Laplacian if

More information

Gaussian Processes to Speed up Hamiltonian Monte Carlo

Gaussian Processes to Speed up Hamiltonian Monte Carlo Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo

More information

MHF Preprint Series. On systems of algebraic equations with parametric exponents. K. Yokoyama

MHF Preprint Series. On systems of algebraic equations with parametric exponents. K. Yokoyama MHF Preprint Series Kyushu University 21st Century COE Program Development of Dynamic Mathematics with High Functionality On systems of algebraic equations with parametric exponents MHF 2003-4 K. Yokoyama

More information

Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

More information

Prof. Dr. Raymond Hemmecke

Prof. Dr. Raymond Hemmecke Prof. Dr. Raymond Hemmecke Combinatorial Optimization Zentrum Mathematik Technische Universität Munich 85747 Garching Germany Tel.: +49-89-289 16864 Fax: +49-89-289 16859 email: hemmecke@ma.tum.de Curriculum

More information

On The So-Called Huber Sandwich Estimator and Robust Standard Errors by David A. Freedman

On The So-Called Huber Sandwich Estimator and Robust Standard Errors by David A. Freedman On The So-Called Huber Sandwich Estimator and Robust Standard Errors by David A. Freedman Abstract The Huber Sandwich Estimator can be used to estimate the variance of the MLE when the underlying model

More information

Square Root Propagation

Square Root Propagation Square Root Propagation Andrew G. Howard Department of Computer Science Columbia University New York, NY 10027 ahoward@cs.columbia.edu Tony Jebara Department of Computer Science Columbia University New

More information

Math 143 - College Algebra (Online)

Math 143 - College Algebra (Online) Math 143 - College Algebra (Online) 3 Credits Term: Instructor: Spring 2010 Ken Floyd (208) 732-6583 Office Location: Office Hours: Shields 206E 1-2 PM, M - F E-mail: kfloyd@csi.edu 10 AM - Noon Saturday

More information

Exam C, Fall 2006 PRELIMINARY ANSWER KEY

Exam C, Fall 2006 PRELIMINARY ANSWER KEY Exam C, Fall 2006 PRELIMINARY ANSWER KEY Question # Answer Question # Answer 1 E 19 B 2 D 20 D 3 B 21 A 4 C 22 A 5 A 23 E 6 D 24 E 7 B 25 D 8 C 26 A 9 E 27 C 10 D 28 C 11 E 29 C 12 B 30 B 13 C 31 C 14

More information

Bayesian Decision Theory. Prof. Richard Zanibbi

Bayesian Decision Theory. Prof. Richard Zanibbi Bayesian Decision Theory Prof. Richard Zanibbi Bayesian Decision Theory The Basic Idea To minimize errors, choose the least risky class, i.e. the class for which the expected loss is smallest Assumptions

More information

The CUSUM algorithm a small review. Pierre Granjon

The CUSUM algorithm a small review. Pierre Granjon The CUSUM algorithm a small review Pierre Granjon June, 1 Contents 1 The CUSUM algorithm 1.1 Algorithm............................... 1.1.1 The problem......................... 1.1. The different steps......................

More information

Acknowledgments. Data Mining with Regression. Data Mining Context. Overview. Colleagues

Acknowledgments. Data Mining with Regression. Data Mining Context. Overview. Colleagues Data Mining with Regression Teaching an old dog some new tricks Acknowledgments Colleagues Dean Foster in Statistics Lyle Ungar in Computer Science Bob Stine Department of Statistics The School of the

More information

Large Sample Covariance Matrices and High-Dimensional Data Analysis

Large Sample Covariance Matrices and High-Dimensional Data Analysis Large Sample Covariance Matrices and High-Dimensional Data Analysis High-dimensional data appear in many fields, and their analysis has become increasingly important in modern statistics. However, it has

More information

Multivariate Normal Distribution

Multivariate Normal Distribution Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

More information

MTH304: Honors Algebra II

MTH304: Honors Algebra II MTH304: Honors Algebra II This course builds upon algebraic concepts covered in Algebra. Students extend their knowledge and understanding by solving open-ended problems and thinking critically. Topics

More information

Moduli of Tropical Plane Curves

Moduli of Tropical Plane Curves Villa 2015 Berlin Mathematical School, TU Berlin; joint work with Michael Joswig (TU Berlin), Ralph Morrison (UC Berkeley), and Bernd Sturmfels (UC Berkeley) November 17, 2015 Berlin Mathematical School,

More information

Clovis Community College Core Competencies Assessment 2014 2015 Area II: Mathematics Algebra

Clovis Community College Core Competencies Assessment 2014 2015 Area II: Mathematics Algebra Core Assessment 2014 2015 Area II: Mathematics Algebra Class: Math 110 College Algebra Faculty: Erin Akhtar (Learning Outcomes Being Measured) 1. Students will construct and analyze graphs and/or data

More information

UNDERSTANDING INFORMATION CRITERIA FOR SELECTION AMONG CAPTURE-RECAPTURE OR RING RECOVERY MODELS

UNDERSTANDING INFORMATION CRITERIA FOR SELECTION AMONG CAPTURE-RECAPTURE OR RING RECOVERY MODELS UNDERSTANDING INFORMATION CRITERIA FOR SELECTION AMONG CAPTURE-RECAPTURE OR RING RECOVERY MODELS David R. Anderson and Kenneth P. Burnham Colorado Cooperative Fish and Wildlife Research Unit Room 201 Wagar

More information

A REVIEW OF CONSISTENCY AND CONVERGENCE OF POSTERIOR DISTRIBUTION

A REVIEW OF CONSISTENCY AND CONVERGENCE OF POSTERIOR DISTRIBUTION A REVIEW OF CONSISTENCY AND CONVERGENCE OF POSTERIOR DISTRIBUTION By Subhashis Ghosal Indian Statistical Institute, Calcutta Abstract In this article, we review two important issues, namely consistency

More information