# Model-Based Recursive Partitioning for Detecting Interaction Effects in Subgroups

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Model-Based Recursive Partitioning for Detecting Interaction Effects in Subgroups Achim Zeileis, Torsten Hothorn, Kurt Hornik

2 Overview Motivation: Trees, leaves, and branches Model-based recursive partitioning Model estimation Tests for parameter instability Segmentation Pruning Application: Treatment effect for chronic disease Summary

3 Motivation: Trees Breiman (2001, Statistical Science) distinguishes two cultures of statistical modeling. Data models: Stochastic models, typically parametric. Algorithmic models: Flexible models, data-generating process unknown. Example: Recursive partitioning models dependent variable Y by learning a partition w.r.t explanatory variables Z 1,..., Z l. Key features: Predictive power in nonlinear regression relationships with automatic interaction detection. Interpretability (enhanced by visualization), i.e., no black box methods.

4 Motivation: Leaves Typically: Simple models for univariate Y, e.g., mean or proportion. Examples: CART and C4.5 in statistical and machine learning, respectively. Problems: For classical tree algorithms. No concept of significance, possibly biased variable selection. No complex (parametric) models in leaves. Many different tree algorithms for different types of data. Here: Synthesis of parametric data models and algorithmic tree models. Fitting local models by partitioning of the sample space. Based on statistical hypothesis tests for parameter instability.

5 Motivation: Branches Base algorithm: Growth of branches from the roots to the leaves of the tree follows a generic recursive partitioning algorithm. 1 Fit a (possibly very simple) model for the response Y. 2 Assess association of Y and each Z j. 3 Split sample along the Z j with strongest association: Choose breakpoint with highest improvement of the model fit. 4 Repeat steps 1 3 recursively in the subsamples until some stopping criterion is met. Here: Segmentation (3) of parametric models (1) with additive objective function using parameter instability tests (2) and associated statistical significance (4).

6 Model-based recursive partitioning: Estimation Models: M(Y, θ) with (potentially) multivariate observations Y Y and k-dimensional parameter vector θ Θ. Parameter estimation: θ by optimization of objective function Ψ(Y, θ) for n observations Y i (i = 1,..., n): θ = argmin θ Θ n Ψ(Y i, θ). i=1 Special cases: Maximum likelihood (ML), weighted and ordinary least squares (OLS and WLS), quasi-ml, and other M-estimators. Central limit theorem: If there is a true parameter θ 0 and given certain weak regularity conditions, ˆθ is asymptotically normal with mean θ 0 and sandwich-type covariance.

7 Model-based recursive partitioning: Estimation Estimating function: θ can also be defined in terms of n ψ(y i, θ) = 0, i=1 where ψ(y, θ) = Ψ(Y, θ)/ θ. Idea: In many situations, a single global model M(Y, θ) that fits all n observations cannot be found. But it might be possible to find a partition w.r.t. the variables Z = (Z 1,..., Z l ) so that a well-fitting model can be found locally in each cell of the partition. Tool: Assess parameter instability w.r.t to partitioning variables Z j Z j (j = 1,..., l).

8 Model-based recursive partitioning: Tests Generalized M-fluctuation tests capture instabilities in θ for an ordering w.r.t Z j. Basis: Empirical fluctuation process of cumulative deviations w.r.t. to an ordering σ(z ij ). nt W j (t, θ) = V 1/2 n 1/2 ψ(y σ(zij ), θ) (0 t 1) i=1 Functional central limit theorem: Under parameter stability W j (, θ) d W 0 ( ), where W 0 is a k-dimensional Brownian bridge.

9 Model-based recursive partitioning: Tests Test statistics: Scalar functional λ(w j ) that captures deviations from zero. Null distribution: Asymptotic distribution of λ(w 0 ). Special cases: Class of test encompasses many well-known tests for different classes of models. Certain functionals λ are particularly intuitive for numeric and categorical Z j, respectively. Advantage: Model M(Y, θ) just has to be estimated once. Empirical estimating functions ψ(y i, θ) just have to be re-ordered and aggregated for each Z j.

10 Model-based recursive partitioning: Tests Splitting numeric variables: Assess instability using suplm statistics. λ suplm (W j ) = max i=i,...,ı ( i n n i ) 1 ( ) i 2 W j. n n 2 Interpretation: Maximization of single shift LM statistics for all conceivable breakpoints in [i, ı]. Limiting distribution: Supremum of a squared, k-dimensional tied-down Bessel process.

11 Model-based recursive partitioning: Tests Splitting categorical variables: Assess instability using χ 2 statistics. λ χ 2(W j ) = C c=1 n I c ( ) i 2 I c W j n 2 Feature: Invariant for re-ordering of the C categories and the observations within each category. Interpretation: Captures instability for split-up into C categories. Limiting distribution: χ 2 with k (C 1) degrees of freedom.

12 Model-based recursive partitioning: Segmentation Goal: Split model into b = 1,..., B segments along the partitioning variable Z j associated with the highest parameter instability. Local optimization of b i I b Ψ(Y i, θ b ). B = 2: Exhaustive search of order O(n). B > 2: Exhaustive search is of order O(n B 1 ), but can be replaced by dynamic programming of order O(n 2 ). Different methods (e.g., information criteria) can choose B adaptively. Here: Binary partitioning.

13 Model-based recursive partitioning: Pruning Pruning: Avoid overfitting. Pre-pruning: Internal stopping criterion. Stop splitting when there is no significant parameter instability. Post-pruning: Grow large tree and prune splits that do not improve the model fit (e.g., via cross-validation or information criteria). Here: Pre-pruning based on Bonferroni-corrected p values of the fluctuation tests.

14 Application: Treatment effect for chronic disease Task: Identify groups of chronic disease patients with different treatment effects. Source: Anonymized data from consulting project. Model: Logistic regression estimated by maximum likelihood. Response: Improvement (yes/no) of chronic disease for 1354 patients after treatment over several weeks. Regressor: Treatment (active drug/placebo). Partitioning variables: 11 variables that describe disease status of patients. Lower values indicate more severe forms of the disease. Result: Treatment most effective for certain intermediate forms.

15 Application: Treatment effect for chronic disease 1 risk7 p = > 0 3 risk5 p = > Node 2 (n = 689) 1 Node 4 (n = 527) 1 Node 5 (n = 138) yes no yes no yes no placebo drug placebo drug placebo drug

16 Application: Treatment effect for chronic disease Model-based recursive partitioning: Coefficient estimates for regressors. Parameter instability tests for partitioning variables (bold = significant at adjusted 5% level, underlined = smallest p value). Regressors Partitioning variables (const.) treatment risk4 risk5 risk7 risk9 risk

17 Summary Synthesis of parametric data models and algorithmic tree models. Based on modern class of parameter instability tests. Aims to minimize clearly defined objective function by greedy forward search. Can be applied to general class of parametric models: generalized linear models, psychometric models (e.g., Rasch, Bradley-Terry), models for location and scale, etc. Automatic interaction detection of effects in subgroups. Alternative to traditional means of model specification, especially for variables with unknown association. Object-oriented implementation freely available: Extension for new models requires some coding, though.

18 References Zeileis A, Hothorn T, Hornik K (2008). Model-Based Recursive Partitioning. Journal of Computational and Graphical Statistics, 17(2), doi: / x Zeileis A, Hornik K (2007). Generalized M-Fluctuation Tests for Parameter Instability. Statistica Neerlandica, 61(4), doi: /j x Hothorn T, Hornik K, Strobl C, Zeileis A (2013). party: A Laboratory for Recursive Partytioning. R package version

### Testing, Monitoring, and Dating Structural Changes in Exchange Rate Regimes

Testing, Monitoring, and Dating Structural Changes in Exchange Rate Regimes Achim Zeileis http://eeecon.uibk.ac.at/~zeileis/ Overview Motivation Exchange rate regimes Exchange rate regression What is the

### Testing, Monitoring, and Dating Structural Changes in FX Regimes

Overview,, and Dating Structural Changes in FX Regimes Achim Zeileis Motivation What is the new Chinese exchange rate regime? Exchange rate regimes Exchange rate regression Structural change tools Model

### Testing, Monitoring, and Dating Structural Changes in FX Regimes

Testing, Monitoring, and Dating Structural Changes in FX Regimes Achim Zeileis http://statmath.wu.ac.at/~zeileis/ Overview Motivation What is the new Chinese exchange rate regime? Exchange rate regimes

### Implementing a Class of Structural Change Tests: An Econometric Computing Approach

Implementing a Class of Structural Change Tests: An Econometric Computing Approach Achim Zeileis http://www.ci.tuwien.ac.at/~zeileis/ Contents Why should we want to do tests for structural change, econometric

### Model-based Recursive Partitioning

Model-based Recursive Partitioning Achim Zeileis Wirtschaftsuniversität Wien Torsten Hothorn Ludwig-Maximilians-Universität München Kurt Hornik Wirtschaftsuniversität Wien Abstract Recursive partitioning

### A Unified Approach to Structural Change Tests Based on ML Scores, F Statistics, and OLS Residuals

A Unified Approach to Structural Change Tests Based on ML Scores, F Statistics, and OLS Residuals Achim Zeileis Wirtschaftsuniversität Wien Abstract Three classes of structural change tests (or tests for

### partykit: A Toolkit for Recursive Partytioning

partykit: A Toolkit for Recursive Partytioning Achim Zeileis, Torsten Hothorn http://eeecon.uibk.ac.at/~zeileis/ Overview Status quo: R software for tree models New package: partykit Unified infrastructure

### Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Testing, Monitoring, and Dating Structural Changes in Exchange Rate Regimes

Testing, Monitoring, and Dating Structural Changes in Exchange Rate Regimes Achim Zeileis WU Wirtschaftsuniversität Wien Ajay Shah NIPFP, New Delhi Ila Patnaik NIPFP, New Delhi Abstract Linear regression

### Gerry Hobbs, Department of Statistics, West Virginia University

Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

### Data Mining Classification: Decision Trees

Data Mining Classification: Decision Trees Classification Decision Trees: what they are and how they work Hunt s (TDIDT) algorithm How to select the best split How to handle Inconsistent data Continuous

### Monitoring Structural Change in Dynamic Econometric Models

Monitoring Structural Change in Dynamic Econometric Models Achim Zeileis Friedrich Leisch Christian Kleiber Kurt Hornik http://www.ci.tuwien.ac.at/~zeileis/ Contents Model frame Generalized fluctuation

### Machine Learning Methods for Causal Effects. Susan Athey, Stanford University Guido Imbens, Stanford University

Machine Learning Methods for Causal Effects Susan Athey, Stanford University Guido Imbens, Stanford University Introduction Supervised Machine Learning v. Econometrics/Statistics Lit. on Causality Supervised

### Learning Example. Machine learning and our focus. Another Example. An example: data (loan application) The data and the goal

Learning Example Chapter 18: Learning from Examples 22c:145 An emergency room in a hospital measures 17 variables (e.g., blood pressure, age, etc) of newly admitted patients. A decision is needed: whether

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### Data mining techniques: decision trees

Data mining techniques: decision trees 1/39 Agenda Rule systems Building rule systems vs rule systems Quick reference 2/39 1 Agenda Rule systems Building rule systems vs rule systems Quick reference 3/39

### Data Mining Techniques Chapter 6: Decision Trees

Data Mining Techniques Chapter 6: Decision Trees What is a classification decision tree?.......................................... 2 Visualizing decision trees...................................................

### Location matters. 3 techniques to incorporate geo-spatial effects in one's predictive model

Location matters. 3 techniques to incorporate geo-spatial effects in one's predictive model Xavier Conort xavier.conort@gear-analytics.com Motivation Location matters! Observed value at one location is

### Data Mining with R. Decision Trees and Random Forests. Hugh Murrell

Data Mining with R Decision Trees and Random Forests Hugh Murrell reference books These slides are based on a book by Graham Williams: Data Mining with Rattle and R, The Art of Excavating Data for Knowledge

### p Values and Alternative Boundaries

p Values and Alternative Boundaries for CUSUM Tests Achim Zeileis Working Paper No. 78 December 2000 December 2000 SFB Adaptive Information Systems and Modelling in Economics and Management Science Vienna

### EXPLORING & MODELING USING INTERACTIVE DECISION TREES IN SAS ENTERPRISE MINER. Copyr i g ht 2013, SAS Ins titut e Inc. All rights res er ve d.

EXPLORING & MODELING USING INTERACTIVE DECISION TREES IN SAS ENTERPRISE MINER ANALYTICS LIFECYCLE Evaluate & Monitor Model Formulate Problem Data Preparation Deploy Model Data Exploration Validate Models

Risk and Return of Investments in Online Peer-to-Peer Lending (Extended Abstract) Harpreet Singh a, Ram Gopal b, Xinxin Li b a School of Management, University of Texas at Dallas, Richardson, Texas 75083-0688

### Generalized M-Fluctuation Tests for Parameter Instability

Generalized M-Fluctuation Tests for Parameter Instability Achim Zeileis Kurt Hornik Department of Statistics and Mathematics, Wirtschaftsuniversität Wien Abstract A general class of fluctuation tests for

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Data Mining Methods: Applications for Institutional Research

Data Mining Methods: Applications for Institutional Research Nora Galambos, PhD Office of Institutional Research, Planning & Effectiveness Stony Brook University NEAIR Annual Conference Philadelphia 2014

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### Classification and Regression Trees

Classification and Regression Trees Bob Stine Dept of Statistics, School University of Pennsylvania Trees Familiar metaphor Biology Decision tree Medical diagnosis Org chart Properties Recursive, partitioning

### Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

### Benchmarking Open-Source Tree Learners in R/RWeka

Benchmarking Open-Source Tree Learners in R/RWeka Michael Schauerhuber 1, Achim Zeileis 1, David Meyer 2, Kurt Hornik 1 Department of Statistics and Mathematics 1 Institute for Management Information Systems

### Exchange Rate Regime Analysis for the Chinese Yuan

Exchange Rate Regime Analysis for the Chinese Yuan Achim Zeileis Ajay Shah Ila Patnaik Abstract We investigate the Chinese exchange rate regime after China gave up on a fixed exchange rate to the US dollar

### Applied Data Mining Analysis: A Step-by-Step Introduction Using Real-World Data Sets

Applied Data Mining Analysis: A Step-by-Step Introduction Using Real-World Data Sets http://info.salford-systems.com/jsm-2015-ctw August 2015 Salford Systems Course Outline Demonstration of two classification

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### Classification and regression trees

Classification and regression trees Michael Friendly December 7, 215 Recursive partitioning methods provide an alternative to (generalized) linear models for categorical responses, particularly when there

### Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví. Pavel Kříž. Seminář z aktuárských věd MFF 4.

Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví Pavel Kříž Seminář z aktuárských věd MFF 4. dubna 2014 Summary 1. Application areas of Insurance Analytics 2. Insurance Analytics

### TACKLING SIMPSON'S PARADOX IN BIG DATA USING CLASSIFICATION & REGRESSION TREES

TACKLING SIMPSON'S PARADOX IN BIG DATA USING CLASSIFICATION & REGRESSION TREES Research in Progress Shmueli, Galit, Indian School of Business, Hyderabad, India, galit_shmueli@isb.edu Yahav, Inbal, Bar

Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.

### Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

### An Overview and Evaluation of Decision Tree Methodology

An Overview and Evaluation of Decision Tree Methodology ASA Quality and Productivity Conference Terri Moore Motorola Austin, TX terri.moore@motorola.com Carole Jesse Cargill, Inc. Wayzata, MN carole_jesse@cargill.com

### Data Mining - Evaluation of Classifiers

Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010

### Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation. Lecture Notes for Chapter 4. Introduction to Data Mining

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data

### Decision Trees. The predictor vector x can be a mixture of continuous and categorical variables.

Decision Trees 1 Introduction to tree based methods 1.1 The general setting Supervised learning problem. Data: {(y i, x i ) : i = 1,..., n}, where y is the response, x is a p-dimensional input vector,

### Decision-Tree Learning

Decision-Tree Learning Introduction ID3 Attribute selection Entropy, Information, Information Gain Gain Ratio C4.5 Decision Trees TDIDT: Top-Down Induction of Decision Trees Numeric Values Missing Values

### COMP3420: Advanced Databases and Data Mining. Classification and prediction: Introduction and Decision Tree Induction

COMP3420: Advanced Databases and Data Mining Classification and prediction: Introduction and Decision Tree Induction Lecture outline Classification versus prediction Classification A two step process Supervised

### Smart Grid Data Analytics for Decision Support

1 Smart Grid Data Analytics for Decision Support Prakash Ranganathan, Department of Electrical Engineering, University of North Dakota, Grand Forks, ND, USA Prakash.Ranganathan@engr.und.edu, 701-777-4431

### Data Mining Algorithms Part 1. Dejan Sarka

Data Mining Algorithms Part 1 Dejan Sarka Join the conversation on Twitter: @DevWeek #DW2015 Instructor Bio Dejan Sarka (dsarka@solidq.com) 30 years of experience SQL Server MVP, MCT, 13 books 7+ courses

### A THEORETICAL COMPARISON OF DATA MASKING TECHNIQUES FOR NUMERICAL MICRODATA

A THEORETICAL COMPARISON OF DATA MASKING TECHNIQUES FOR NUMERICAL MICRODATA Krish Muralidhar University of Kentucky Rathindra Sarathy Oklahoma State University Agency Internal User Unmasked Result Subjects

### Classification: Basic Concepts, Decision Trees, and Model Evaluation. General Approach for Building Classification Model

10 10 Classification: Basic Concepts, Decision Trees, and Model Evaluation Dr. Hui Xiong Rutgers University Introduction to Data Mining 1//009 1 General Approach for Building Classification Model Tid Attrib1

### STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and

Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Web-based Analytics Table

### Classification and Prediction

Classification and Prediction Slides for Data Mining: Concepts and Techniques Chapter 7 Jiawei Han and Micheline Kamber Intelligent Database Systems Research Lab School of Computing Science Simon Fraser

### LOGISTIC REGRESSION. Nitin R Patel. where the dependent variable, y, is binary (for convenience we often code these values as

LOGISTIC REGRESSION Nitin R Patel Logistic regression extends the ideas of multiple linear regression to the situation where the dependent variable, y, is binary (for convenience we often code these values

### Binary recursive partitioning: Background, methods, and application to psychology. Department of Psychology. Wichita State University

Binary Recursive Partitioning 1 Running head: BINARY RECURSIVE PARTITIONING Binary recursive partitioning: Background, methods, and application to psychology Edgar C. Merkle Victoria A. Shaffer Department

### Data Mining for Knowledge Management. Classification

1 Data Mining for Knowledge Management Classification Themis Palpanas University of Trento http://disi.unitn.eu/~themis Data Mining for Knowledge Management 1 Thanks for slides to: Jiawei Han Eamonn Keogh

### From the help desk: Bootstrapped standard errors

The Stata Journal (2003) 3, Number 1, pp. 71 80 From the help desk: Bootstrapped standard errors Weihua Guan Stata Corporation Abstract. Bootstrapping is a nonparametric approach for evaluating the distribution

### Detection of changes in variance using binary segmentation and optimal partitioning

Detection of changes in variance using binary segmentation and optimal partitioning Christian Rohrbeck Abstract This work explores the performance of binary segmentation and optimal partitioning in the

### Decision Trees. Classification Trees

Decision Tree: Introduction 323 Decision Tree: Introduction A decision tree is a powerful method for classification and prediction and for facilitating decision making in sequential decision problems.

### Penalized Logistic Regression and Classification of Microarray Data

Penalized Logistic Regression and Classification of Microarray Data Milan, May 2003 Anestis Antoniadis Laboratoire IMAG-LMC University Joseph Fourier Grenoble, France Penalized Logistic Regression andclassification

### Section on Statistical Learning and Data Mining JSM 2009

Using Data Mining to Explore Seasonal Differences Between the U.S. Current Employment Statistics Survey and the Quarterly Census of Employment and Wages October 2009 G. Erkens BLS G. Erkens, Bureau of

### Yiming Peng, Department of Statistics. February 12, 2013

Regression Analysis Using JMP Yiming Peng, Department of Statistics February 12, 2013 2 Presentation and Data http://www.lisa.stat.vt.edu Short Courses Regression Analysis Using JMP Download Data to Desktop

### Step 5: Conduct Analysis. The CCA Algorithm

Model Parameterization: Step 5: Conduct Analysis P Dropped species with fewer than 5 occurrences P Log-transformed species abundances P Row-normalized species log abundances (chord distance) P Selected

### Linear Model Selection and Regularization

Linear Model Selection and Regularization Recall the linear model Y = β 0 + β 1 X 1 + + β p X p + ɛ. In the lectures that follow, we consider some approaches for extending the linear model framework. In

### Quantitative Methods for Finance

Quantitative Methods for Finance Module 1: The Time Value of Money 1 Learning how to interpret interest rates as required rates of return, discount rates, or opportunity costs. 2 Learning how to explain

### The Data Mining Process

Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data

### Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

### Ordinary Least Squares and Poisson Regression Models by Luc Anselin University of Illinois Champaign-Urbana, IL

Appendix C Ordinary Least Squares and Poisson Regression Models by Luc Anselin University of Illinois Champaign-Urbana, IL This note provides a brief description of the statistical background, estimators

### Chapter 12 Discovering New Knowledge Data Mining

Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to

### 1 Teaching notes on GMM 1.

Bent E. Sørensen January 23, 2007 1 Teaching notes on GMM 1. Generalized Method of Moment (GMM) estimation is one of two developments in econometrics in the 80ies that revolutionized empirical work in

### Decision Trees (Sections )

Decision Trees (Sections 8.1-8.4) Non-metric methods CART (Classification & regression Trees) Number of splits Query selection & node impurity Multiway splits When to stop splitting? Pruning Assignment

### DATA INTERPRETATION AND STATISTICS

PholC60 September 001 DATA INTERPRETATION AND STATISTICS Books A easy and systematic introductory text is Essentials of Medical Statistics by Betty Kirkwood, published by Blackwell at about 14. DESCRIPTIVE

### Decision Trees from large Databases: SLIQ

Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values

### Supervised Learning (Big Data Analytics)

Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used

### Maximally Selected Rank Statistics in R

Maximally Selected Rank Statistics in R by Torsten Hothorn and Berthold Lausen This document gives some examples on how to use the maxstat package and is basically an extention to Hothorn and Lausen (2002).

### Online Appendices to the Corporate Propensity to Save

Online Appendices to the Corporate Propensity to Save Appendix A: Monte Carlo Experiments In order to allay skepticism of empirical results that have been produced by unusual estimators on fairly small

### Data Mining and Data Warehousing. Henryk Maciejewski. Data Mining Predictive modelling: regression

Data Mining and Data Warehousing Henryk Maciejewski Data Mining Predictive modelling: regression Algorithms for Predictive Modelling Contents Regression Classification Auxiliary topics: Estimation of prediction

### Business Analytics and Credit Scoring

Study Unit 5 Business Analytics and Credit Scoring ANL 309 Business Analytics Applications Introduction Process of credit scoring The role of business analytics in credit scoring Methods of logistic regression

### THE RISE OF THE BIG DATA: WHY SHOULD STATISTICIANS EMBRACE COLLABORATIONS WITH COMPUTER SCIENTISTS XIAO CHENG. (Under the Direction of Jeongyoun Ahn)

THE RISE OF THE BIG DATA: WHY SHOULD STATISTICIANS EMBRACE COLLABORATIONS WITH COMPUTER SCIENTISTS by XIAO CHENG (Under the Direction of Jeongyoun Ahn) ABSTRACT Big Data has been the new trend in businesses.

### !"!!"#\$\$%&'()*+\$(,%!"#\$%\$&'()*""%(+,'-*&./#-\$&'(-&(0*".\$#-\$1"(2&."3\$'45"

!"!!"#\$\$%&'()*+\$(,%!"#\$%\$&'()*""%(+,'-*&./#-\$&'(-&(0*".\$#-\$1"(2&."3\$'45"!"#"\$%&#'()*+',\$\$-.&#',/"-0%.12'32./4'5,5'6/%&)\$).2&'7./&)8'5,5'9/2%.%3%&8':")08';:

### Principles of Data Mining by Hand&Mannila&Smyth

Principles of Data Mining by Hand&Mannila&Smyth Slides for Textbook Ari Visa,, Institute of Signal Processing Tampere University of Technology October 4, 2010 Data Mining: Concepts and Techniques 1 Differences

Data Mining: An Overview David Madigan http://www.stat.columbia.edu/~madigan Overview Brief Introduction to Data Mining Data Mining Algorithms Specific Eamples Algorithms: Disease Clusters Algorithms:

### Efficiency in Software Development Projects

Efficiency in Software Development Projects Aneesh Chinubhai Dharmsinh Desai University aneeshchinubhai@gmail.com Abstract A number of different factors are thought to influence the efficiency of the software

### The Design and Analysis of Benchmark Experiments

The Design and Analysis of Benchmark Experiments Torsten Hothorn Friedrich-Alexander-Universität Erlangen-Nürnberg Friedrich Leisch Technische Universität Wien Kurt Hornik Wirtschaftsuniversität Wien Achim

### Data Mining Practical Machine Learning Tools and Techniques

Ensemble learning Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Combining multiple models Bagging The basic idea

### Penalized regression: Introduction

Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihood

### Data mining on the EPIRARE survey data

Deliverable D1.5 Data mining on the EPIRARE survey data A. Coi 1, M. Santoro 2, M. Lipucci 2, A.M. Bianucci 1, F. Bianchi 2 1 Department of Pharmacy, Unit of Research of Bioinformatic and Computational

### Estimating the probability of positive crossmatch after negative virtual crossmatch

EI 2012-25 Estimating the probability of positive crossmatch after negative virtual crossmatch Kristiaan Glorie Econometric Institute, Erasmus University Rotterdam Abbreviations: PRA (panel reactive antibodies),

### Multivariate Logistic Regression

1 Multivariate Logistic Regression As in univariate logistic regression, let π(x) represent the probability of an event that depends on p covariates or independent variables. Then, using an inv.logit formulation

### D-optimal plans in observational studies

D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational

### Variable Selection and Transformation of Variables in SAS Enterprise Miner

Variable Selection and Transformation of Variables in SAS Enterprise Miner Kattamuri S. Sarma, Ph.D Ecostat Research Corp., White Plains NY kssarma@worldnet.att.net kssarma@ecostat-research.com 2 Issues

### Regression Modeling Strategies

Frank E. Harrell, Jr. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis With 141 Figures Springer Contents Preface Typographical Conventions

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

### Big Data: Big N. V.C. 14.387 Note. December 2, 2014

Big Data: Big N V.C. 14.387 Note December 2, 2014 Examples of Very Big Data Congressional record text, in 100 GBs Nielsen s scanner data, 5TBs Medicare claims data are in 100 TBs Facebook 200,000 TBs See

### Classification and Regression Trees as a Part of Data Mining in Six Sigma Methodology

, October 20-22, 2010, San Francisco, USA Classification and Regression Trees as a Part of Data Mining in Six Sigma Methodology Andrej Trnka, Member, IAENG Abstract The paper deals with implementation

### Exchange Rate Regime Analysis for the Indian Rupee

Exchange Rate Regime Analysis for the Indian Rupee Achim Zeileis Ajay Shah Ila Patnaik Abstract We investigate the Indian exchange rate regime starting from 1993 when trading in the Indian rupee began

### THE HYBRID CART-LOGIT MODEL IN CLASSIFICATION AND DATA MINING. Dan Steinberg and N. Scott Cardell

THE HYBID CAT-LOGIT MODEL IN CLASSIFICATION AND DATA MINING Introduction Dan Steinberg and N. Scott Cardell Most data-mining projects involve classification problems assigning objects to classes whether

### Classification and regression trees

Classification and regression trees December 9 Introduction We ve seen that local methods and splines both operate by partitioning the sample space of the regression variable(s), and then fitting separate/piecewise

### Model Combination. 24 Novembre 2009

Model Combination 24 Novembre 2009 Datamining 1 2009-2010 Plan 1 Principles of model combination 2 Resampling methods Bagging Random Forests Boosting 3 Hybrid methods Stacking Generic algorithm for mulistrategy

### Classification and Prediction

Classification and Prediction 1. Objectives...2 2. Classification vs. Prediction...3 2.1. Definitions...3 2.2. Supervised vs. Unsupervised Learning...3 2.3. Classification and Prediction Related Issues...4

### Model-based boosting in R

Model-based boosting in R Introduction to Gradient Boosting Matthias Schmid Institut für Medizininformatik, Biometrie und Epidemiologie (IMBE) Friedrich-Alexander-Universität Erlangen-Nürnberg Statistical