# Generalized BIC for Singular Models Factoring through Regular Models

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Generalized BIC for Singular Models Factoring through Regular Models Shaowei Lin shaowei/ Department of Mathematics, University of California, Berkeley PhD student (Advisor: Bernd Sturmfels)

2 Abstract The Bayesian Information Criterion (BIC) is an important tool for model selection, but its use is limited only to regular models. Recently, Watanabe [5] generalized the BIC to singular models using ideas from algebraic geometry. In this paper, we compute this generalized BIC for singular models factoring through regular models, relating it to the real log canonical threshold of polynomial fiber ideals.

3 Bayesian Information Criterion The BIC is a score used for model selection. BIC = log L 0 + d 2 log N L 0 : maximum likelihood, d: dimension, N: sample size. However, it applies only for regular models.

4 Regular and Singular Models A model M(U) is regular if it is identifiable p(x u, M) = p(x u, M) x u = u and the Fisher information matrix I(u) is positive definite u. I jk (u) = log p(x u) u j log p(x u) p(x u)dx u k Otherwise, the model is singular. e.g. most exponential families are regular, while most hidden variable models are singular.

5 Factored Models M 2 (Ω) factors through M 1 (U) if for some function u(ω), p(x ω, M 2 ) = p(x u(ω), M 1 ). e.g. parametrized multivariate Gaussian models N(0, Σ(ω)).

6 Generalized BIC In 2001, Watanabe showed that for singular models M(Ω), the BIC generalizes to log L 0 + λ log N (θ 1) log log N, where λ is the smallest pole of the zeta function ζ(z) = K(ω) z ϕ(ω)dω, z C Ω and θ its multiplicity. Here, ϕ(ω) is a prior on Ω, and K(ω) the Kullback-Leibler distance to the true distribution. We compute (λ, θ) by monomializing K(ω). (a.k.a. resolution of singularities)

7 Key Idea: Fiber Ideals Even if M(Ω) is parametrized by simple polynomials, the Kullback function K(ω) is non-polynomial and difficult to monomialize. We show that if M(Ω) factors through a regular model, we can define a polynomial fiber ideal and compute (λ, θ) by monomializing this ideal.

8 Real Log Canonical Thresholds Given an ideal I = f 1,...,f r generated by polynomials, define the real log canonical threshold (λ,θ) to be the smallest pole λ and multiplicity θ of the zeta function ζ(z) = (f 1 (ω) f r (ω) 2 ) z/2 dω, z C. The RLCT is independent of the choice of generators f 1,...,f r, and can be computed by monomializing I. For monomial ideals, we find RLCTs using a geometriccombinatorial tool involving Newton polyhedra.

9 Main Result Let M 2 (Ω) factor through a regular model M 1 (U) via u(ω). Given N i.i.d. samples, let their M.L.E. in M 1 be û U. Let Z(N) be the marginal likelihood of the data given M 2. Theorem (L.) If û u(ω), then asymptotically, log Z(N) log L 0 + λ log N (θ 1) log log N where (2λ,θ) is the RLCT of the fiber ideal I = u 1 (ω) û 1,...,u d (ω) û d.

10 Example: Classical BIC Let M(U) be a regular model and û the M.L.E. of the data. Then, (2λ,θ) is the RLCT of the fiber ideal which is just (d, 1). I = u 1 û 1,...,u d û d

11 Example: Discrete Models Let M(Ω) be a model on discrete space {1, 2,...,k} with polynomial state probabilities p 1 (ω),...,p k (ω). Let q 1,...,q k be relative frequencies of the data. If q p(ω), then (2λ,θ) is the RLCT of the fiber ideal I = p 1 (ω) q 1,...,p k (ω) q k.

12 Example: Gaussian Models Let N(µ(Ω), Σ(Ω)) be a multivariate Gaussian model with sample mean ˆµ and covariance matrix ˆΣ. If ˆµ µ(ω) and ˆΣ Σ(Ω), then (2λ,θ) is the RLCT of the fiber ideal I = µ(ω) ˆµ, Σ(ω) ˆΣ.

13 Acknowledgements Many thanks go to Mathias Drton and Sumio Watanabe for enlightening discussions and key ideas.

14 References 1. V. I. Arnol d, S. M. Guseĭn-Zade and A. N. Varchenko: Singularities of Differentiable Maps, Vol. II, Birkhäuser, Boston, H. Hironaka: Resolution of singularities of an algebraic variety over a field of characteristic zero I, II, Ann. of Math. (2) 79 (1964) S. Lin: Asymptotic Approximation of Marginal Likelihood Integrals, preprint arxiv: (2010). 4. S. Watanabe: Algebraic analysis for nonidentifiable learning machines, Neural Computation 13 (2001) S. Watanabe: Algebraic Geometry and Statistical Learning Theory, Cambridge Monographs on Applied and Computational Mathematics 25, Cambridge University Press, Cambridge, G. Schwarz: Estimating the Dimension of a Model, Annals of Statistics 6 (1978)

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### Linear Classification. Volker Tresp Summer 2015

Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

### A COUNTEREXAMPLE FOR SUBADDITIVITY OF MULTIPLIER IDEALS ON TORIC VARIETIES

Communications in Algebra, 40: 1618 1624, 2012 Copyright Taylor & Francis Group, LLC ISSN: 0092-7872 print/1532-4125 online DOI: 10.1080/00927872.2011.552084 A COUNTEREXAMPLE FOR SUBADDITIVITY OF MULTIPLIER

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### 1. Students will demonstrate an understanding of the real number system as evidenced by classroom activities and objective tests

MATH 102/102L Inter-Algebra/Lab Properties of the real number system, factoring, linear and quadratic equations polynomial and rational expressions, inequalities, systems of equations, exponents, radicals,

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### Package MixGHD. June 26, 2015

Type Package Package MixGHD June 26, 2015 Title Model Based Clustering, Classification and Discriminant Analysis Using the Mixture of Generalized Hyperbolic Distributions Version 1.7 Date 2015-6-15 Author

### Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma

Interpreting Kullback-Leibler Divergence with the Neyman-Pearson Lemma Shinto Eguchi a, and John Copas b a Institute of Statistical Mathematics and Graduate University of Advanced Studies, Minami-azabu

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Statistical Machine Learning from Data

Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Gaussian Mixture Models Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique

### MATH 590: Meshfree Methods

MATH 590: Meshfree Methods Chapter 7: Conditionally Positive Definite Functions Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 fasshauer@iit.edu MATH 590 Chapter

### Tropical Mathematics. An Interesting and Useful Variant of Ordinary Arithmetic. John Polking. Rice University.

Tropical Mathematics An Interesting and Useful Variant of Ordinary Arithmetic John Polking polking@rice.edu Rice University Tropical Mathematics A new mathematics Tropical Mathematics A new mathematics

### CPM High Schools California Standards Test (CST) Results for 2004-2010

CPM High California Standards Test (CST) Results for 2004-2010 The tables below show a comparison between CPM high schools and all high schools in California based on the percentage of students who scored

### Maximum Likelihood Estimation

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

### MATHEMATICS. Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Degree Requirements

MATHEMATICS Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Paul Feit, PhD Dr. Paul Feit is Professor of Mathematics and Coordinator for Mathematics.

### Algebraic Computation Models. Algebraic Computation Models

Algebraic Computation Models Ζυγομήτρος Ευάγγελος ΜΠΛΑ 201118 Φεβρουάριος, 2013 Reasons for using algebraic models of computation The Turing machine model captures computations on bits. Many algorithms

### Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering

Two Topics in Parametric Integration Applied to Stochastic Simulation in Industrial Engineering Department of Industrial Engineering and Management Sciences Northwestern University September 15th, 2014

### CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

### Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2015 CS 551, Fall 2015

### L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

### Clustering - example. Given some data x i X Find a partitioning of the data into k disjunctive clusters Example: k-means clustering

Clustering - example Graph Mining and Graph Kernels Given some data x i X Find a partitioning of the data into k disjunctive clusters Example: k-means clustering x!!!!8!!! 8 x 1 1 Clustering - example

### Singularities and Newton Polygons

Publ. RIMS, Kyoto Univ. 12 Suppl. (1977), 123-129. Singularities and Newton Polygons by Jean-Michel KANTOR* It was conjectured by V. I. Arnold that it should be possible to express all "reasonable" invariants

### Section 5. Stan for Big Data. Bob Carpenter. Columbia University

Section 5. Stan for Big Data Bob Carpenter Columbia University Part I Overview Scaling and Evaluation data size (bytes) 1e18 1e15 1e12 1e9 1e6 Big Model and Big Data approach state of the art big model

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### THE MULTIVARIATE ANALYSIS RESEARCH GROUP. Carles M Cuadras Departament d Estadística Facultat de Biologia Universitat de Barcelona

THE MULTIVARIATE ANALYSIS RESEARCH GROUP Carles M Cuadras Departament d Estadística Facultat de Biologia Universitat de Barcelona The set of statistical methods known as Multivariate Analysis covers a

### Generating Gaussian Mixture Models by Model Selection For Speech Recognition

Generating Gaussian Mixture Models by Model Selection For Speech Recognition Kai Yu F06 10-701 Final Project Report kaiy@andrew.cmu.edu Abstract While all modern speech recognition systems use Gaussian

### The Exponential Family

The Exponential Family David M. Blei Columbia University November 3, 2015 Definition A probability density in the exponential family has this form where p.x j / D h.x/ expf > t.x/ a./g; (1) is the natural

### Parametric or nonparametric: the FIC approach for stationary time series

Parametric or nonparametric: the FIC approach for stationary time series Gudmund Horn Hermansen* Department of Mathematics, University of Oslo, Oslo, Norway - gudmunhh@math.uio.no Nils Lid Hjort Department

### STUDENT LEARNING OUTCOMES FOR THE SANTIAGO CANYON COLLEGE MATHEMATICS DEPARTMENT (Last Revised 8/20/14)

STUDENT LEARNING OUTCOMES FOR THE SANTIAGO CANYON COLLEGE MATHEMATICS DEPARTMENT (Last Revised 8/20/14) Department SLOs: Upon completion of any course in Mathematics the student will be able to: 1. Create

### Message-passing sequential detection of multiple change points in networks

Message-passing sequential detection of multiple change points in networks Long Nguyen, Arash Amini Ram Rajagopal University of Michigan Stanford University ISIT, Boston, July 2012 Nguyen/Amini/Rajagopal

### Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.

Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### Bayesian Classifier for a Gaussian Distribution, Decision Surface Equation, with Application

Iraqi Journal of Statistical Science (18) 2010 p.p. [35-58] Bayesian Classifier for a Gaussian Distribution, Decision Surface Equation, with Application ABSTRACT Nawzad. M. Ahmad * Bayesian decision theory

### ON THE DEGREES OF FREEDOM IN RICHLY PARAMETERISED MODELS

COMPSTAT 2004 Symposium c Physica-Verlag/Springer 2004 ON THE DEGREES OF FREEDOM IN RICHLY PARAMETERISED MODELS Salvatore Ingrassia and Isabella Morlini Key words: Richly parameterised models, small data

Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

COURSE SYLLABUS Pre-Calculus A/B Last Modified: April 2015 Course Description: In this year-long Pre-Calculus course, students will cover topics over a two semester period (as designated by A and B sections).

### Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure

Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure Belyaev Mikhail 1,2,3, Burnaev Evgeny 1,2,3, Kapushev Yermek 1,2 1 Institute for Information Transmission

### C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)}

C: LEVEL 800 {MASTERS OF ECONOMICS( ECONOMETRICS)} 1. EES 800: Econometrics I Simple linear regression and correlation analysis. Specification and estimation of a regression model. Interpretation of regression

### Bayesian Classification

CS 650: Computer Vision Bryan S. Morse BYU Computer Science Statistical Basis Training: Class-Conditional Probabilities Suppose that we measure features for a large training set taken from class ω i. Each

### Factoring Cubic Polynomials

Factoring Cubic Polynomials Robert G. Underwood 1. Introduction There are at least two ways in which using the famous Cardano formulas (1545) to factor cubic polynomials present more difficulties than

### CONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Relations Between Time Domain and Frequency Domain Prediction Error Methods - Tomas McKelvey

COTROL SYSTEMS, ROBOTICS, AD AUTOMATIO - Vol. V - Relations Between Time Domain and Frequency Domain RELATIOS BETWEE TIME DOMAI AD FREQUECY DOMAI PREDICTIO ERROR METHODS Tomas McKelvey Signal Processing,

### M5MS09. Graphical Modelling

Course: MMS09 Setter: Walden Checker: Ginzberg Editor: Calderhead External: Wood Date: April, 0 MSc EXAMINATIONS (STATISTICS) May-June 0 MMS09 Graphical Modelling Setter s signature Checker s signature

### Primary Decomposition

Primary Decomposition 1 Prime and Primary Ideals solutions defined by prime ideals are irreducible 2 Finite Dimensional Systems modeling of gene regulatory networks 3 Primary Decomposition definition and

### ISU Department of Mathematics. Graduate Examination Policies and Procedures

ISU Department of Mathematics Graduate Examination Policies and Procedures There are four primary criteria to be used in evaluating competence on written or oral exams. 1. Knowledge Has the student demonstrated

### A HYBRID GENETIC ALGORITHM FOR THE MAXIMUM LIKELIHOOD ESTIMATION OF MODELS WITH MULTIPLE EQUILIBRIA: A FIRST REPORT

New Mathematics and Natural Computation Vol. 1, No. 2 (2005) 295 303 c World Scientific Publishing Company A HYBRID GENETIC ALGORITHM FOR THE MAXIMUM LIKELIHOOD ESTIMATION OF MODELS WITH MULTIPLE EQUILIBRIA:

### Unsupervised Learning

Unsupervised Learning Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK zoubin@gatsby.ucl.ac.uk http://www.gatsby.ucl.ac.uk/~zoubin September 16, 2004 Abstract We give

### Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8

Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Computational algebraic geometry

Computational algebraic geometry Learning coefficients via symbolic and numerical methods Anton Leykin Georgia Tech AIM, Palo Alto, December 2011 Let k be a field (R or C). Ideals varieties Ideal in R

### Probabilistic Graphical Models Final Exam

Introduction to Probabilistic Graphical Models Final Exam, March 2007 1 Probabilistic Graphical Models Final Exam Please fill your name and I.D.: Name:... I.D.:... Duration: 3 hours. Guidelines: 1. The

### Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classiﬁca6on

Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classiﬁca6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output

### Factorial experimental designs and generalized linear models

Statistics & Operations Research Transactions SORT 29 (2) July-December 2005, 249-268 ISSN: 1696-2281 www.idescat.net/sort Statistics & Operations Research c Institut d Estadística de Transactions Catalunya

### Linear algebra and the geometry of quadratic equations. Similarity transformations and orthogonal matrices

MATH 30 Differential Equations Spring 006 Linear algebra and the geometry of quadratic equations Similarity transformations and orthogonal matrices First, some things to recall from linear algebra Two

### Item selection by latent class-based methods: an application to nursing homes evaluation

Item selection by latent class-based methods: an application to nursing homes evaluation Francesco Bartolucci, Giorgio E. Montanari, Silvia Pandolfi 1 Department of Economics, Finance and Statistics University

### i=1 In practice, the natural logarithm of the likelihood function, called the log-likelihood function and denoted by

Statistics 580 Maximum Likelihood Estimation Introduction Let y (y 1, y 2,..., y n be a vector of iid, random variables from one of a family of distributions on R n and indexed by a p-dimensional parameter

### Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

### Natural Conjugate Gradient in Variational Inference

Natural Conjugate Gradient in Variational Inference Antti Honkela, Matti Tornio, Tapani Raiko, and Juha Karhunen Adaptive Informatics Research Centre, Helsinki University of Technology P.O. Box 5400, FI-02015

### Anomaly detection for Big Data, networks and cyber-security

Anomaly detection for Big Data, networks and cyber-security Patrick Rubin-Delanchy University of Bristol & Heilbronn Institute for Mathematical Research Joint work with Nick Heard (Imperial College London),

### Generalization of Chisquare Goodness-of-Fit Test for Binned Data Using Saturated Models, with Application to Histograms

Generalization of Chisquare Goodness-of-Fit Test for Binned Data Using Saturated Models, with Application to Histograms Robert D. Cousins Dept. of Physics and Astronomy University of California, Los Angeles,

### A hidden Markov model for criminal behaviour classification

RSS2004 p.1/19 A hidden Markov model for criminal behaviour classification Francesco Bartolucci, Institute of economic sciences, Urbino University, Italy. Fulvia Pennoni, Department of Statistics, University

### Big Data, Statistics, and the Internet

Big Data, Statistics, and the Internet Steven L. Scott April, 4 Steve Scott (Google) Big Data, Statistics, and the Internet April, 4 / 39 Summary Big data live on more than one machine. Computing takes

### These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

### Generalized Linear Model Theory

Appendix B Generalized Linear Model Theory We describe the generalized linear model as formulated by Nelder and Wedderburn (1972), and discuss estimation of the parameters and tests of hypotheses. B.1

### Row Ideals and Fibers of Morphisms

Michigan Math. J. 57 (2008) Row Ideals and Fibers of Morphisms David Eisenbud & Bernd Ulrich Affectionately dedicated to Mel Hochster, who has been an inspiration to us for many years, on the occasion

### Research Article The General Traveling Wave Solutions of the Fisher Equation with Degree Three

Advances in Mathematical Physics Volume 203, Article ID 65798, 5 pages http://dx.doi.org/0.55/203/65798 Research Article The General Traveling Wave Solutions of the Fisher Equation with Degree Three Wenjun

### Method of Stationary phase. Reference: Hormander vol I. Steve Zelditch Department of Mathematics Northwestern University

Method of Stationary phase Reference: Hormander vol I Steve Zelditch Department of Mathematics Northwestern University 1 Method of Stationary Phase We now describe the method of stationary phase, which

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### A GAUSSIAN COMPOUND DECISION BAKEOFF

A GAUSSIAN COMPOUND DECISION BAKEOFF ROGER KOENKER Abstract. A nonparametric mixture model approach to empirical Bayes compound decisions for the Gaussian location model is compared with a parametric empirical

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### Mathematics. Course List

Mathematics 1 Mathematics Course List Code Course Title Unit I. Ph.D. Programme (Full-time and Part-time); II. M.Phil. Programme (Full-time and Part-time); and III. M.Sc. Programme (Full-time and Part-time)

### Bayesian logistic betting strategy against probability forecasting. Akimichi Takemura, Univ. Tokyo. November 12, 2012

Bayesian logistic betting strategy against probability forecasting Akimichi Takemura, Univ. Tokyo (joint with Masayuki Kumon, Jing Li and Kei Takeuchi) November 12, 2012 arxiv:1204.3496. To appear in Stochastic

### P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

### On Gauss s characterization of the normal distribution

Bernoulli 13(1), 2007, 169 174 DOI: 10.3150/07-BEJ5166 On Gauss s characterization of the normal distribution ADELCHI AZZALINI 1 and MARC G. GENTON 2 1 Department of Statistical Sciences, University of

### Max-Min Representation of Piecewise Linear Functions

Beiträge zur Algebra und Geometrie Contributions to Algebra and Geometry Volume 43 (2002), No. 1, 297-302. Max-Min Representation of Piecewise Linear Functions Sergei Ovchinnikov Mathematics Department,

### GLM, insurance pricing & big data: paying attention to convergence issues.

GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK - michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.

### Penalized Splines - A statistical Idea with numerous Applications...

Penalized Splines - A statistical Idea with numerous Applications... Göran Kauermann Ludwig-Maximilians-University Munich Graz 7. September 2011 1 Penalized Splines - A statistical Idea with numerous Applications...

### A crash course in probability and Naïve Bayes classification

Probability theory A crash course in probability and Naïve Bayes classification Chapter 9 Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s

### Diploma Plus in Certificate in Advanced Engineering

Diploma Plus in Certificate in Advanced Engineering Mathematics New Syllabus from April 2011 Ngee Ann Polytechnic / School of Interdisciplinary Studies 1 I. SYNOPSIS APPENDIX A This course of advanced

### Main page. Given f ( x, y) = c we differentiate with respect to x so that

Further Calculus Implicit differentiation Parametric differentiation Related rates of change Small variations and linear approximations Stationary points Curve sketching - asymptotes Curve sketching the

### RESONANCES AND BALLS IN OBSTACLE SCATTERING WITH NEUMANN BOUNDARY CONDITIONS

RESONANCES AND BALLS IN OBSTACLE SCATTERING WITH NEUMANN BOUNDARY CONDITIONS T. J. CHRISTIANSEN Abstract. We consider scattering by an obstacle in R d, d 3 odd. We show that for the Neumann Laplacian if

### Gaussian Processes to Speed up Hamiltonian Monte Carlo

Gaussian Processes to Speed up Hamiltonian Monte Carlo Matthieu Lê Murray, Iain http://videolectures.net/mlss09uk_murray_mcmc/ Rasmussen, Carl Edward. "Gaussian processes to speed up hybrid Monte Carlo

### MHF Preprint Series. On systems of algebraic equations with parametric exponents. K. Yokoyama

MHF Preprint Series Kyushu University 21st Century COE Program Development of Dynamic Mathematics with High Functionality On systems of algebraic equations with parametric exponents MHF 2003-4 K. Yokoyama

### Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

### Prof. Dr. Raymond Hemmecke

Prof. Dr. Raymond Hemmecke Combinatorial Optimization Zentrum Mathematik Technische Universität Munich 85747 Garching Germany Tel.: +49-89-289 16864 Fax: +49-89-289 16859 email: hemmecke@ma.tum.de Curriculum

### On The So-Called Huber Sandwich Estimator and Robust Standard Errors by David A. Freedman

On The So-Called Huber Sandwich Estimator and Robust Standard Errors by David A. Freedman Abstract The Huber Sandwich Estimator can be used to estimate the variance of the MLE when the underlying model

### Square Root Propagation

Square Root Propagation Andrew G. Howard Department of Computer Science Columbia University New York, NY 10027 ahoward@cs.columbia.edu Tony Jebara Department of Computer Science Columbia University New

### Math 143 - College Algebra (Online)

Math 143 - College Algebra (Online) 3 Credits Term: Instructor: Spring 2010 Ken Floyd (208) 732-6583 Office Location: Office Hours: Shields 206E 1-2 PM, M - F E-mail: kfloyd@csi.edu 10 AM - Noon Saturday

### Exam C, Fall 2006 PRELIMINARY ANSWER KEY

Exam C, Fall 2006 PRELIMINARY ANSWER KEY Question # Answer Question # Answer 1 E 19 B 2 D 20 D 3 B 21 A 4 C 22 A 5 A 23 E 6 D 24 E 7 B 25 D 8 C 26 A 9 E 27 C 10 D 28 C 11 E 29 C 12 B 30 B 13 C 31 C 14

### Bayesian Decision Theory. Prof. Richard Zanibbi

Bayesian Decision Theory Prof. Richard Zanibbi Bayesian Decision Theory The Basic Idea To minimize errors, choose the least risky class, i.e. the class for which the expected loss is smallest Assumptions

### The CUSUM algorithm a small review. Pierre Granjon

The CUSUM algorithm a small review Pierre Granjon June, 1 Contents 1 The CUSUM algorithm 1.1 Algorithm............................... 1.1.1 The problem......................... 1.1. The different steps......................

### Acknowledgments. Data Mining with Regression. Data Mining Context. Overview. Colleagues

Data Mining with Regression Teaching an old dog some new tricks Acknowledgments Colleagues Dean Foster in Statistics Lyle Ungar in Computer Science Bob Stine Department of Statistics The School of the

### Large Sample Covariance Matrices and High-Dimensional Data Analysis

Large Sample Covariance Matrices and High-Dimensional Data Analysis High-dimensional data appear in many fields, and their analysis has become increasingly important in modern statistics. However, it has

### Multivariate Normal Distribution

Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

### MTH304: Honors Algebra II

MTH304: Honors Algebra II This course builds upon algebraic concepts covered in Algebra. Students extend their knowledge and understanding by solving open-ended problems and thinking critically. Topics

### Moduli of Tropical Plane Curves

Villa 2015 Berlin Mathematical School, TU Berlin; joint work with Michael Joswig (TU Berlin), Ralph Morrison (UC Berkeley), and Bernd Sturmfels (UC Berkeley) November 17, 2015 Berlin Mathematical School,

### Clovis Community College Core Competencies Assessment 2014 2015 Area II: Mathematics Algebra

Core Assessment 2014 2015 Area II: Mathematics Algebra Class: Math 110 College Algebra Faculty: Erin Akhtar (Learning Outcomes Being Measured) 1. Students will construct and analyze graphs and/or data