# Detection of changes in variance using binary segmentation and optimal partitioning

Save this PDF as:

Size: px
Start display at page:

Download "Detection of changes in variance using binary segmentation and optimal partitioning"

## Transcription

1 Detection of changes in variance using binary segmentation and optimal partitioning Christian Rohrbeck Abstract This work explores the performance of binary segmentation and optimal partitioning in the context of detecting changes in variance for time-series. Both, binary segmentation and optimal partitioning, are based on cost functions that penalise a high amount of changepoints in order to avoid overfitting. Analysis is performed on simulated time-series; first on Normal data with constant but unknown mean and changing variance and second on Exponential data with changing parameter. Results suggest a good performance of both approaches. 1 Introduction In a wide range of sciences, it is a great issue to investigate the consequences of the climate change. For example, hydrologists study the relation between the climate change and significant wave heights. Similarly, meteorologists survey the coherency between the increasing average temperature and the number and intensity of storms. In both sciences, analysis is often based on data collected other the past decades. The appearance and intensity of a storm at sea and the variability of the sea level in form of waves have a high positive correlation. Consequently, detecting changes in the variability of the sea level allows conclusions on the number and intensity of storms over the considered period at sea. In medicine, particularly in hospitals, parameters of intensive care patients are measured and analysed continuously by a medical monitor. If one parameters drops below a specific value or the frequency of the heart changes, the medical staff are alerted. In order to offer the best chance of survival, it is necessary to detect changes in mean and variability of the parameters immediately. Therefore, the speed of detection is of high interest in this context, in contrast to the first example. Further areas for which the detection of change points is of interest are bioinformatics (Lio and Vanucci, 2000) and econometrics (Zhou et al., 2010). Given a time-series {y t : t 1,... n}, a changepoint occurs at time τ if the distributions of {y 1,..., y τ } and {y τ+1,..., y n } differ with respect to at least one criterion such as mean, variance or regression structure. For example: 1. Change in mean: y t has mean { µ 1, t τ µ t = µ n, t > τ, where µ 1 µ n. 2. Change in variance: y t has variance { σ 1, t τ σ t = σ n, t > τ, where σ 1 σ n. In this work, we are interested in detecting changes in variance of time-series. The task is to decide for time-series whether changepoints exist and if so, to detect their location. In Section 2, we explore the methods of binary segmentation (Scott and Knott, 1974) and optimal partitioning (Jackson et al., 2005) for this purpose. In Section 3, both methods are compared and discussed by evaluating the performance of the approaches on simulated Normal and Exponential data with multiple changepoints. For the Normal data, we assume that the mean is constant but unknown and only the variance changes. 1

2 2 Detecting changes in variance The detection of changes in variance has been well studied in the past years and several methods exist, including cumulative sums of square (Inclan and Tiao, 1994), penalised likelihood (Yao, 1988) and Bayesian posterior odds (Fearnhead, 2006). Formally speaking, for an ordered sequence of data (y 1,..., y n ), we aim to determine an unknown number, m, of changepoints τ 1,..., τ m, where each changepoint is an integer value between 2 and n 1. Let us define that the sequence of changepoints is ordered, such that τ i < τ j if, and only if, i < j. Further, we denote τ 0 = 1 and τ m+1 = n. Consequently, the data is split into m + 1 segments with the ith segment containing y (τi 1 +1):τ i. One approach, mentioned in several publications, is to identify multiple changepoints by minimising m+1 i=1 [ C(y(τi 1+1):τ i ) ] + βf(m), (1) with respect to m and τ 1,..., τ m. Here, C is a cost function for a segment and βf(m) a penalty term in order to avoid overfitting. The choice of C as twice the negative log-likelihood is commonly used in the changepoint literature (see for example Chen and Gupta (2000)). However, Inclan and Tiao (1994), for example, propose a different cost function. In the following, we consider the penalty term as to be linear in m, i.e. βf(m) = βm. The naive approach of testing all possible changepoint locations is hardly practicable for large n as the number of possible partitions is 2 n 1. Therefore, computationally effective algorithms are required. In the following binary segmentation by Scott and Knott (1974) and optimal partitioning by Jackson et al. (2005) are explained as approaches to this problem. The notation in Section 2.2 is based on Killick et al. (2012). 2.1 Binary segmentation Binary segmentation is the standard method in changepoint literature (Killick et al., 2012). Basically, it iteratively applies a single changepoint method to different subsets of the sequence y 1,..., y n in order to detect multiple changepoints. It starts by applying the single changepoint method to the entire sequence, i.e. it tests if a split of the sequence exists such that the cost function over the two sub sequences plus the penalty term is smaller than the cost function on the entire sequence. Formally, in the context of (1), the single changepoint method tests whether there exist an integer τ {1,..., n 1} that satisfies C(y 1:τ ) + C(y (τ+1):n ) + β < C(y 1:n ). (2) If such an τ does not exist, no changepoint is detected and the algorithm stops. Otherwise, the corresponding value of τ is identified as a changepoint and the sequence is split up into two subsequences y 1:τ and y (τ+1):n, i.e. the sequences before and after the changepoint. Then the single changepoint method is applied to each of these subsequences. The procedure continues until no further changepoint is detected. Binary segmentation can be seen as an approach to minimise (1) by iteratively deciding whether a changepoint should be added or not. Binary segmentation is computationally efficient, with O(n log n) calculations. However, binary segmentation does not automatically lead to the global minimum of equation (1) and is thus only approximative. 2.2 Optimal partitioning Optimal partitioning by Jackson et al. (2005) is in contrast to binary segmentation an exact method to solve the minimisation problem described by (1) with linear penalty term, i.e. f(m) = m. The method uses dynamic programming which can be used because the principle of optimality holds which states that any subpartition of an optimal partition is optimal; for details and proof see Jackson et al. (2005). Dynamic programming is a recursive method which allows to solve several kinds of combinatorial optimisation problems, here, the minimisation problem (1). Starting with the first observation, the algorithm iteratively determines the optimal partition of the first t + 1 data points by using the optimal partition of the first t. At each iteration step t + 1, the algorithm considers all possibilities j {0,..., t} for the last changepoint of the optimal partition. Consequently, due to the principle of optimality, the cost of the partition is given by the cost of the optimal partition prior to j and the cost of the sequence from the last changepoint to the end of the data, i.e. C(y j:(t+1) ). The cost of the optimal partition prior to j was calculated in previous iteration steps. At the end of each iteration step, the partition which 2

3 minimises the cost is stored. The algorithm runs until t + 1 = n. More formally, we denote the set of possible changepoints for the first t data points by P t = {τ : 0 < τ 1 < < τ m < t}. Further, we define by F (t) the cost of the optimal partition of the data until time point t and set F (0) = β. Based on equation (1), F(t) is determined as { m+1 [ F (t) = min C(y(τi 1+1):τ τ P i ) + β ] } β. (3) t i=1 Using the principle of optimality, we can write F (t) in (3) as F (t) = min [ F (τ ) + C(y (τ +1):t) + β ] (4) τ which implies that in iteration step t only t 1 calculations are necessary. Consequently, the number of calculations for a sequence of n observations is O(n 2 ) and hence it is not as computationally efficient as binary segmentation. However, the approach of optimal partitioning leads to the calculation of the global minimum of the minimsation problem (1); see Theorem 2 of Jackson et al. (2005) for details. Therefore, optimal partitioning is more accurate than binary segmentation. Steps for implementing the optimal partitioning approach are given in Algorithm 1. Algorithm 1 Optimal Partitioning by Killick et al. (2012) based on Jackson et al. (2005) Require: Set of data (y 1,..., y n ) where y i R Require: Cost function C( ) Require: Penalty constant β 1: Set n= length of data. 2: Set F (0) = β and t = 1. 3: Set cp(0) = NULL and F = F (0). 4: while t < n do 5: Get F (t) = min τ [ F (τ ) + C(y (τ +1):t) + β ]. 6: Get τ = argmin τ [ F (τ ) + C(y (τ +1):t) + β ]. 7: Set cp(t) = (cp(τ ), τ ). 8: Set F = (F, F (t)). 9: t:= t+1 10: end while 11: return Changepoints recorded in cp(n). 12: return Optimal Costs recorded in F. Line 6 in Algorithm 1 enables us to get the the global minimum of (1) by backtracking, as all necessary information is recorded in cp(n). The optimal costs for each time point are then recorded in F. 3 Simulation study 3.1 Application to Normal data In the following simulation study, the approaches of binary segmentation and optimal partitioning are applied to a sequence {y i } n i=1 of independent and normally distributed random variables with unknown parameters {µ, σ i } n i=1. Such samples occur in a lot of applications like oceanology (Killick et al., 2010) and finance (Chen and Gupta, 1997). We consider the penalised likelihood approach by Yao (1988) (Killick et al., 2010). Therefore, the cost function for a sequence {y i } t, 1 s t n, is chosen as twice the negative log likelihood. Formally, C(y s:t ) = 2l(µ, σ s,..., σ t y s,..., y t ) [ (yi µ) 2 = + log ( 2πσi 2 ) ]. σ 2 i (5) As the exact parameters µ and {σ i } t are unknown, they are replaced by their maximum likelihood estimates. Because the mean is constant over the entire sequence, the estimate on the mean, ˆµ, simplifies to the average over all observations. For the penalty term β, we consider the Schwarz Information Criterion (SIC), proposed by Yao (1988) and set β = log n. Further, we select f(m) as to be linear in m, i.e. f(m) = m. After defining all parameters and functions for (1), binary segmentation and optimal partitioning are performed. As mentioned in Section 2.2, binary segmentation iteratively performs a single changepoint method on a subsequence {y i } t, 1 s t n. For the considered case of normally distributed data, the single changepoint method can be viewed as an approach to test the hypothesis H 0 of no changepoint (Gupta and Tang, 1987) H 0 : σ 2 s = σ 2 s+1 = = σ 2 t versus the alternative of an unknown changepoint τ {s,..., t 1} H 1 : σ 2 s = = σ 2 τ σ 2 τ+1 = σ 2 t. Following the test formulation, the cost function (5) for a sequence {y i } t determines to C(y s:t ) = [ (yi µ) 2 + log ( 2πˆσ s:t) ] 2, (6) ˆσ 2 s:t 3

4 Figure 1: Two simulated time series with Normal data (top and bottom) and detected changepoints by binary segmentation(left) and optimal partitioning (right). where ˆσ s:t is the maximum likelihood estimate of the variance based on the data {y i } t. The decision whether to reject H 0 or not is decided by equation (2). This approach is also considered by Killick et al. (2010) and Chen and Gupta (1997). Thus, we first determine the integer value τ which minimises the left-hand side in (2). More formally, τ [ = argmin C(ys:τ ) + C(y (τ+1):t ) + log(n) ]. (7) τ Second, we check whether C(y s:τ ) + C(y (τ +1):t) + log(n) < C(y s:t ). (8) If so, the algorithm repeats the procedure considered above for the subsequences {y i } τ and {y i } t i=τ +1. If (8) is not fulfilled, we conclude that the variance is constant between the time points s and t and can be estimated by maximum likelihood. For the method of optimal partitioning, we set the variance constant in each subsequence and estimate it by maximum likelihood, as for the binary segmentation, and perform Algorithm 1. For the simulation study, we generate two time series with n = 266 data points and perform the two approaches considered above. The variance changes for each time-series four times at the same time points. In particular, we select τ 1 = 81, τ 2 = 130, τ 3 = 162 and τ 4 = 226 as change points and the variances for the first time-series are σ (1) = (1.3, 0.3, 0.8, 0.4, 1.1) and for the second σ (2) = (0.4, 0.7, 0.5, 0.3, 0.8). The estimated changepoints for the two time-series and the two methods are illustrated in Figure 1. For the first time-series, the results are identical for the two methods and get the real changepoints. For the second time-series, the results are different and each method does not detect one changepoint. Further, optimal partitioning detects one changepoint, at t = 6, at which the true variance does not change. 3.2 Application to Exponential data In the second simulation study, we consider a sequence {y i } n i=1 of n independent and exponentially distributed random variables with unknown parameters {λ i } n i=1. Such cases occur in queuing modelling, where we have to decide whether the arrival rates are constant over a time period or not. In contrast to Normal data, the mean is determined by the variance and thus a change of the variance results in a change of the mean. Nevertheless, as both functionals only depend on one variable λ i, binary segmentation and optimal partition are applicable. As for Normal data, we set the cost function C(y s:t ) equal to twice the negative log-likelihood of the data {y i } t, 1 s t n, β = log n and 4

5 Figure 2: Two simulated time series with Exponential data (top and bottom) and detected changepoints by binary segmentation(left) and optimal partitioning (right). f(m) = m. Consequently, the cost function for the sequence {y i } t determines to C(y s:t ) = 2 [λ i y i log λ i ] (9) and as the true parameter values are unknown, they are set to their maximum likelihood estimates. The approach of formulating the single changepoint method as a test, as done for Normal data, is also applicable for Exponential data. Hence, we test for a sequence {y i } t the hypothesis H 0 : λ s = = λ t = λ versus the alternative of an unknown changepoint τ H 1 : λ s = = λ τ λ τ+1 = = λ t. Consequently, this results in a cost function of C(y s:t ) = 2 [ˆλs:t y i log ˆλ s:t ], (10) where ˆλ s:t is the maximum likelihood estimate of the parameter λ s based on the data {y i } t. The algorithms of binary segmentation and optimal partitioning are than applied as for the previous case of Normal data under consideration of equation (10). We generate two time-series with multiple changepoints in order to evaluate the performance of binary segmentation and optimal partitioning. The set of changepoints is the same in both timeseries, τ 1 = 81, τ 2 = 130, τ 3 = 162 and τ 4 = 226. For the different values of the parameter, we select λ (1) = (1.4, 0.3, 0.1, 1.9, 0.1) for the first and = (0.3, 1.9, 0.4, 1.2, 0.7) for the second timeseries. The results presented in Figure 2 illustrate the detected changepoints for the two time-series by binary segmentation and optimal partitioning. The results for the first time-series are quite good as all changepoints are detected by both methods. However, each method detects one changepoint λ (2) where the parameter stays constant. For the second time-series binary segmentation detects the exact number of changepoints, but it detects a wrong changepoint at the end and not τ 4. Further, optimal partitioning detects all true changepoints and two changepoints at which the true parameter value stays constant. Nevertheless, in all cases at least three changepoints are correctly detected. 4 Discussion This work aimed to present methods to detect changepoints in variance for time-series. Both methods considered, binary segmentation and optimal 5

6 partitioning, are based on a minimisation problem that includes a penalty term to avoid overfitting. Optimal partitioning solves the related minimisation problem exactly. In contrast, binary segmentation only approximates the exact solution. Another example for an exact method is segment neighbourhoods by Auger and Lawrence (1989); however it has a lower computational efficiency than optimal partitioning (Killick et al., 2012). In contrast, binary segmentation is more computationally efficient than optimal partitioning. Both approaches fit the changepoints in simulated time-series quite well but often missed at least one changepoint. In the context of applications to detect changes in parameters of intensive care patients, both approaches seem applicable as the number of computations is at maximum quadratic in the number of observations. If we keep all observations, optimal partitioning only needs to perform as many computations as we have observations to detect whether the second to last observation is a changepoint. As the computational efficiency for binary segmentation is O(n log(n)), optimal partition is even faster than binary segmentation for continuously incoming new observations because for binary segmentation all calculations have to be done again. For the case of changes over an already observed time period, like for the GOMOS hindcast time-series considered by Killick et al. (2010), binary segmentation is more computationally efficient. Recent literature explores methods to improve the computational efficiency of the exact solution of the minimisation problem for a linear penalty term. Killick et al. (2012) introduce the Pruned Exact Linear Time (PELT) method which allows to compute the global minimum of the optimisation problem (1) with a linear computational cost by pruning. In this approach, the computationally efficiency of optimal partitioning is improved by removing values which can never be minima at each iteration step (see Section 3 of Killick et al. (2012) for details). References Auger, I. and Lawrence, C. (1989). Algorithms for the optimal detection of segment neighborhoods. Bulletin of Mathematical Biology, 51: Chen, J. and Gupta, A. (1997). Testing and locating variance changepoints with application to stock prices. Journal of the American Statistical Association, 92(438): Chen, J. and Gupta, A. (2000). Parametric statistical change point analysis. Birkhauser. Fearnhead, P. (2006). Exact and efficient bayesian inference for multiple changepoint problems. Statistical Computing, 16: Gupta, A. and Tang, J. (1987). On testing homogeneity of variance for gaussian models. Journal of Statistical Computation and Simulation, 27: Inclan, C. and Tiao, G. (1994). Use of cumulative sum of squares for retrospective detection of changes of variance. Journal of the American Statistical Association, 89: Jackson, B., Scargle, J., Barnes, D., Arabhi, S., Alt, A., Gioumousis, P., Gwin, E., Sangtrakulcharoen, P., Tan, L., and Tsai, T. (2005). An algorithm for optimal partitioning of data on an interval. IEEE, Signal Processing Letters, 12: Killick, R., Eckley, I., Ewans, K., and Jonathan, P. (2010). Detection of changes in variance of oceanographic time-series using changepoint analysis. Ocean Engineering, 37: Killick, R., Fearnhead, P., and Eckley, I. (2012). Optimal detection of changepoints with a linear computational cost. Journal of the American Statistical Association, 107: Lio, P. and Vanucci, M. (2000). Wavelet changepoint prediction of transmembrane proteins. Bioinformatics, 16: Scott, A. and Knott, M. (1974). A cluster analysis method for grouping means in the analysis of variance. Biometrics, 30: Yao, Y. (1988). Estimating the number of changepoints via schwarz criterion. Statistics and Probability Letters, 6: Zhou, Y., Wan, A., Xie, S., and Wang, X. (2010). Wavelet analysis of change-points in a nonparametric regression with heteroscedastic variance. Journal of Econometrics, 259:

### Package changepoint. R topics documented: November 9, 2015. Type Package Title Methods for Changepoint Detection Version 2.

Type Package Title Methods for Changepoint Detection Version 2.2 Date 2015-10-23 Package changepoint November 9, 2015 Maintainer Rebecca Killick Implements various mainstream and

### Probabilistic Methods for Time-Series Analysis

Probabilistic Methods for Time-Series Analysis 2 Contents 1 Analysis of Changepoint Models 1 1.1 Introduction................................ 1 1.1.1 Model and Notation....................... 2 1.1.2 Example:

### Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Least Squares Estimation

Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David

Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

### Nonparametric adaptive age replacement with a one-cycle criterion

Nonparametric adaptive age replacement with a one-cycle criterion P. Coolen-Schrijner, F.P.A. Coolen Department of Mathematical Sciences University of Durham, Durham, DH1 3LE, UK e-mail: Pauline.Schrijner@durham.ac.uk

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### Data Mining and Data Warehousing. Henryk Maciejewski. Data Mining Predictive modelling: regression

Data Mining and Data Warehousing Henryk Maciejewski Data Mining Predictive modelling: regression Algorithms for Predictive Modelling Contents Regression Classification Auxiliary topics: Estimation of prediction

### NPTEL STRUCTURAL RELIABILITY

NPTEL Course On STRUCTURAL RELIABILITY Module # 02 Lecture 6 Course Format: Web Instructor: Dr. Arunasis Chakraborty Department of Civil Engineering Indian Institute of Technology Guwahati 6. Lecture 06:

### Penalized regression: Introduction

Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihood

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### Intro to Data Analysis, Economic Statistics and Econometrics

Intro to Data Analysis, Economic Statistics and Econometrics Statistics deals with the techniques for collecting and analyzing data that arise in many different contexts. Econometrics involves the development

### Regression Modeling Strategies

Frank E. Harrell, Jr. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis With 141 Figures Springer Contents Preface Typographical Conventions

### Likelihood Approaches for Trial Designs in Early Phase Oncology

Likelihood Approaches for Trial Designs in Early Phase Oncology Clinical Trials Elizabeth Garrett-Mayer, PhD Cody Chiuzan, PhD Hollings Cancer Center Department of Public Health Sciences Medical University

### 5. Multiple regression

5. Multiple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/5 QBUS6840 Predictive Analytics 5. Multiple regression 2/39 Outline Introduction to multiple linear regression Some useful

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### 9-3.4 Likelihood ratio test. Neyman-Pearson lemma

9-3.4 Likelihood ratio test Neyman-Pearson lemma 9-1 Hypothesis Testing 9-1.1 Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental

### Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.

Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x

### Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Christfried Webers. Canberra February June 2015

c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

### Linear Classification. Volker Tresp Summer 2015

Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

### Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach

Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach Refik Soyer * Department of Management Science The George Washington University M. Murat Tarimcilar Department of Management Science

### STRATEGIC CAPACITY PLANNING USING STOCK CONTROL MODEL

Session 6. Applications of Mathematical Methods to Logistics and Business Proceedings of the 9th International Conference Reliability and Statistics in Transportation and Communication (RelStat 09), 21

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

### 11. Analysis of Case-control Studies Logistic Regression

Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

### Designing a learning system

Lecture Designing a learning system Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x4-8845 http://.cs.pitt.edu/~milos/courses/cs750/ Design of a learning system (first vie) Application or Testing

### SPARE PARTS INVENTORY SYSTEMS UNDER AN INCREASING FAILURE RATE DEMAND INTERVAL DISTRIBUTION

SPARE PARS INVENORY SYSEMS UNDER AN INCREASING FAILURE RAE DEMAND INERVAL DISRIBUION Safa Saidane 1, M. Zied Babai 2, M. Salah Aguir 3, Ouajdi Korbaa 4 1 National School of Computer Sciences (unisia),

### Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

### Nonparametric Methods for Online Changepoint Detection

Lancaster University STOR601: Research Topic II Nonparametric Methods for Online Changepoint Detection Author: Paul Sharkey Supervisor: Rebecca Killick May 18, 2014 Contents 1 Introduction 1 1.1 Changepoint

### Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví. Pavel Kříž. Seminář z aktuárských věd MFF 4.

Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví Pavel Kříž Seminář z aktuárských věd MFF 4. dubna 2014 Summary 1. Application areas of Insurance Analytics 2. Insurance Analytics

### Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Module No. #01 Lecture No. #15 Special Distributions-VI Today, I am going to introduce

### Univariate Regression

Univariate Regression Correlation and Regression The regression line summarizes the linear relationship between 2 variables Correlation coefficient, r, measures strength of relationship: the closer r is

### A LOGNORMAL MODEL FOR INSURANCE CLAIMS DATA

REVSTAT Statistical Journal Volume 4, Number 2, June 2006, 131 142 A LOGNORMAL MODEL FOR INSURANCE CLAIMS DATA Authors: Daiane Aparecida Zuanetti Departamento de Estatística, Universidade Federal de São

### LOGISTIC REGRESSION. Nitin R Patel. where the dependent variable, y, is binary (for convenience we often code these values as

LOGISTIC REGRESSION Nitin R Patel Logistic regression extends the ideas of multiple linear regression to the situation where the dependent variable, y, is binary (for convenience we often code these values

### Interpretation of Somers D under four simple models

Interpretation of Somers D under four simple models Roger B. Newson 03 September, 04 Introduction Somers D is an ordinal measure of association introduced by Somers (96)[9]. It can be defined in terms

### Web-based Supplementary Materials for. Modeling of Hormone Secretion-Generating. Mechanisms With Splines: A Pseudo-Likelihood.

Web-based Supplementary Materials for Modeling of Hormone Secretion-Generating Mechanisms With Splines: A Pseudo-Likelihood Approach by Anna Liu and Yuedong Wang Web Appendix A This appendix computes mean

### Chapter 1 Hypothesis Testing

Chapter 1 Hypothesis Testing Principles of Hypothesis Testing tests for one sample case 1 Statistical Hypotheses They are defined as assertion or conjecture about the parameter or parameters of a population,

### Journal of Statistical Software

JSS Journal of Statistical Software August 2015, Volume 66, Issue 3. http://www.jstatsoft.org/ Parametric and Nonparametric Sequential Change Detection in R: The cpm Package Gordon J. Ross University College

### Elements of statistics (MATH0487-1)

Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium December 10, 2012 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis -

### Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

### SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

### Package cpm. July 28, 2015

Package cpm July 28, 2015 Title Sequential and Batch Change Detection Using Parametric and Nonparametric Methods Version 2.2 Date 2015-07-09 Depends R (>= 2.15.0), methods Author Gordon J. Ross Maintainer

### Course: Model, Learning, and Inference: Lecture 5

Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.

### P(a X b) = f X (x)dx. A p.d.f. must integrate to one: f X (x)dx = 1. Z b

Continuous Random Variables The probability that a continuous random variable, X, has a value between a and b is computed by integrating its probability density function (p.d.f.) over the interval [a,b]:

### Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Name: Section: I pledge my honor that I have not violated the Honor Code Signature: This exam has 34 pages. You have 3 hours to complete this

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

### Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

### Basic Statistics and Data Analysis for Health Researchers from Foreign Countries

Basic Statistics and Data Analysis for Health Researchers from Foreign Countries Volkert Siersma siersma@sund.ku.dk The Research Unit for General Practice in Copenhagen Dias 1 Content Quantifying association

### Statistics 104: Section 6!

Page 1 Statistics 104: Section 6! TF: Deirdre (say: Dear-dra) Bloome Email: dbloome@fas.harvard.edu Section Times Thursday 2pm-3pm in SC 109, Thursday 5pm-6pm in SC 705 Office Hours: Thursday 6pm-7pm SC

### Principle of Data Reduction

Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then

### Statistical Machine Learning from Data

Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Gaussian Mixture Models Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique

### Tests for Two Survival Curves Using Cox s Proportional Hazards Model

Chapter 730 Tests for Two Survival Curves Using Cox s Proportional Hazards Model Introduction A clinical trial is often employed to test the equality of survival distributions of two treatment groups.

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### HYPOTHESIS TESTING: POWER OF THE TEST

HYPOTHESIS TESTING: POWER OF THE TEST The first 6 steps of the 9-step test of hypothesis are called "the test". These steps are not dependent on the observed data values. When planning a research project,

### Yiming Peng, Department of Statistics. February 12, 2013

Regression Analysis Using JMP Yiming Peng, Department of Statistics February 12, 2013 2 Presentation and Data http://www.lisa.stat.vt.edu Short Courses Regression Analysis Using JMP Download Data to Desktop

### MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

### Poisson Models for Count Data

Chapter 4 Poisson Models for Count Data In this chapter we study log-linear models for count data under the assumption of a Poisson error structure. These models have many applications, not only to the

### A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS

A SURVEY ON CONTINUOUS ELLIPTICAL VECTOR DISTRIBUTIONS Eusebio GÓMEZ, Miguel A. GÓMEZ-VILLEGAS and J. Miguel MARÍN Abstract In this paper it is taken up a revision and characterization of the class of

### Mathematical Induction

Mathematical Induction Victor Adamchik Fall of 2005 Lecture 2 (out of three) Plan 1. Strong Induction 2. Faulty Inductions 3. Induction and the Least Element Principal Strong Induction Fibonacci Numbers

### Chris Slaughter, DrPH. GI Research Conference June 19, 2008

Chris Slaughter, DrPH Assistant Professor, Department of Biostatistics Vanderbilt University School of Medicine GI Research Conference June 19, 2008 Outline 1 2 3 Factors that Impact Power 4 5 6 Conclusions

### Model-Based Recursive Partitioning for Detecting Interaction Effects in Subgroups

Model-Based Recursive Partitioning for Detecting Interaction Effects in Subgroups Achim Zeileis, Torsten Hothorn, Kurt Hornik http://eeecon.uibk.ac.at/~zeileis/ Overview Motivation: Trees, leaves, and

### Generalized Linear Models. Today: definition of GLM, maximum likelihood estimation. Involves choice of a link function (systematic component)

Generalized Linear Models Last time: definition of exponential family, derivation of mean and variance (memorize) Today: definition of GLM, maximum likelihood estimation Include predictors x i through

### Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities

Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities Oscar Kipersztok Mathematics and Computing Technology Phantom Works, The Boeing Company P.O.Box 3707, MC: 7L-44 Seattle, WA 98124

### Monte Carlo testing with Big Data

Monte Carlo testing with Big Data Patrick Rubin-Delanchy University of Bristol & Heilbronn Institute for Mathematical Research Joint work with: Axel Gandy (Imperial College London) with contributions from:

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### DATA INTERPRETATION AND STATISTICS

PholC60 September 001 DATA INTERPRETATION AND STATISTICS Books A easy and systematic introductory text is Essentials of Medical Statistics by Betty Kirkwood, published by Blackwell at about 14. DESCRIPTIVE

### Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

### Chapter 7 Notes - Inference for Single Samples. You know already for a large sample, you can invoke the CLT so:

Chapter 7 Notes - Inference for Single Samples You know already for a large sample, you can invoke the CLT so: X N(µ, ). Also for a large sample, you can replace an unknown σ by s. You know how to do a

### Projects Involving Statistics (& SPSS)

Projects Involving Statistics (& SPSS) Academic Skills Advice Starting a project which involves using statistics can feel confusing as there seems to be many different things you can do (charts, graphs,

### A New Interpretation of Information Rate

A New Interpretation of Information Rate reproduced with permission of AT&T By J. L. Kelly, jr. (Manuscript received March 2, 956) If the input symbols to a communication channel represent the outcomes

### Parametric and Nonparametric Sequential Change Detection in R: The cpm Package

Parametric and Nonparametric Sequential Change Detection in R: The cpm Package Gordon J. Ross Department of Statistical Science University College London gordon@gordonjross.co.uk Abstract The change point

### Summary of Probability

Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

### A Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails

12th International Congress on Insurance: Mathematics and Economics July 16-18, 2008 A Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails XUEMIAO HAO (Based on a joint

### Ridge Regression. Patrick Breheny. September 1. Ridge regression Selection of λ Ridge regression in R/SAS

Ridge Regression Patrick Breheny September 1 Patrick Breheny BST 764: Applied Statistical Modeling 1/22 Ridge regression: Definition Definition and solution Properties As mentioned in the previous lecture,

### Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

### Introduction to Convex Optimization for Machine Learning

Introduction to Convex Optimization for Machine Learning John Duchi University of California, Berkeley Practical Machine Learning, Fall 2009 Duchi (UC Berkeley) Convex Optimization for Machine Learning

### Reject Inference in Credit Scoring. Jie-Men Mok

Reject Inference in Credit Scoring Jie-Men Mok BMI paper January 2009 ii Preface In the Master programme of Business Mathematics and Informatics (BMI), it is required to perform research on a business

### CONTENTS OF DAY 2. II. Why Random Sampling is Important 9 A myth, an urban legend, and the real reason NOTES FOR SUMMER STATISTICS INSTITUTE COURSE

1 2 CONTENTS OF DAY 2 I. More Precise Definition of Simple Random Sample 3 Connection with independent random variables 3 Problems with small populations 8 II. Why Random Sampling is Important 9 A myth,

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### Part 2: One-parameter models

Part 2: One-parameter models Bernoilli/binomial models Return to iid Y 1,...,Y n Bin(1, θ). The sampling model/likelihood is p(y 1,...,y n θ) =θ P y i (1 θ) n P y i When combined with a prior p(θ), Bayes

### A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data

A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data Faming Liang University of Florida August 9, 2015 Abstract MCMC methods have proven to be a very powerful tool for analyzing

### Java Modules for Time Series Analysis

Java Modules for Time Series Analysis Agenda Clustering Non-normal distributions Multifactor modeling Implied ratings Time series prediction 1. Clustering + Cluster 1 Synthetic Clustering + Time series

### CHAPTER 6: Continuous Uniform Distribution: 6.1. Definition: The density function of the continuous random variable X on the interval [A, B] is.

Some Continuous Probability Distributions CHAPTER 6: Continuous Uniform Distribution: 6. Definition: The density function of the continuous random variable X on the interval [A, B] is B A A x B f(x; A,

### Nominal and ordinal logistic regression

Nominal and ordinal logistic regression April 26 Nominal and ordinal logistic regression Our goal for today is to briefly go over ways to extend the logistic regression model to the case where the outcome

### Forecasting in supply chains

1 Forecasting in supply chains Role of demand forecasting Effective transportation system or supply chain design is predicated on the availability of accurate inputs to the modeling process. One of the

### Chapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 )

Chapter 13 Introduction to Nonlinear Regression( 非 線 性 迴 歸 ) and Neural Networks( 類 神 經 網 路 ) 許 湘 伶 Applied Linear Regression Models (Kutner, Nachtsheim, Neter, Li) hsuhl (NUK) LR Chap 10 1 / 35 13 Examples

### Hypothesis Testing for Beginners

Hypothesis Testing for Beginners Michele Piffer LSE August, 2011 Michele Piffer (LSE) Hypothesis Testing for Beginners August, 2011 1 / 53 One year ago a friend asked me to put down some easy-to-read notes

### Estimating an ARMA Process

Statistics 910, #12 1 Overview Estimating an ARMA Process 1. Main ideas 2. Fitting autoregressions 3. Fitting with moving average components 4. Standard errors 5. Examples 6. Appendix: Simple estimators

### Non Parametric Inference

Maura Department of Economics and Finance Università Tor Vergata Outline 1 2 3 Inverse distribution function Theorem: Let U be a uniform random variable on (0, 1). Let X be a continuous random variable

### AHAMADA IBRAHIM GREQAM, université de la méditerranée and CERESUR, université de La Réunion. Abstract

A complementary test for the KPSS test with an application to the US Dollar/Euro exchange rate AHAMADA IBRAHIM GREQAM, université de la méditerranée and CERESUR, université de La Réunion Abstract his paper

### Segmentation models and applications with R

Segmentation models and applications with R Franck Picard UMR 5558 UCB CNRS LBBE, Lyon, France franck.picard@univ-lyon1.fr http://pbil.univ-lyon1.fr/members/fpicard/ INED-28/04/11 F. Picard (CNRS-LBBE)

### Stochastic Inventory Control

Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using