Regression Modeling Strategies

Transcription

1 Frank E. Harrell, Jr. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis With 141 Figures Springer

2 Contents Preface Typographical Conventions vii xxiii 1 Introduction Hypothesis Testing, Estimation, and Prediction Examples of Uses of Predictive Multivariable Modeling Planning for Modeling Emphasizing Continuous Variables Choice of the Model Further Reading 8 2 General Aspects of Fitting Regression Models Notation for Multivariable Regression Models Model Formulations Interpreting Model Parameters Nominal Predictors Interactions 14

3 xiv Contents Example: Inference for a Simple Model Relaxing Linearity Assumption for Continuous Predictors Simple Nonlinear Terms Splines for Estimating Shape of Regression Function and Determining Predictor Transformations Cubic Spline Functions Restricted Cubic Splines Choosing Number and Position of Knots Nonparametric Regression Advantages of Regression Splines over Other Methods Recursive Partitioning: Tree-Based Models Multiple Degree of Freedom Tests of Association Assessment of Model Fit Regression Assumptions Modeling and Testing Complex Interactions Fitting Ordinal Predictors Distributional Assumptions Further Reading Problems 37 3 Missing Data Types of Missing Data Prelude to Modeling Missing Values for Different Types of Response Variables Problems with Simple Alternatives to Imputation Strategies for Developing Imputation Algorithms Single Conditional Mean Imputation Multiple Imputation Summary and Rough Guidelines Further Reading Problems 51 4 Multivariable Modeling Strategies Prespecification of Predictor Complexity Without Later Simplification 53

4 Contents xv 4.2 Checking Assumptions of Multiple Predictors Simultaneously Variable Selection Overfitting and Limits on Number of Predictors Shrinkage Collinearity Data Reduction Variable Clustering Transformation and Scaling Variables Without Using Y Simultaneous Transformation and Imputation Simple Scoring of Variable Clusters Simplifying Cluster Scores How Much Data Reduction Is Necessary? Overly Influential Observations Comparing Two Models Summary: Possible Modeling Strategies Developing Predictive Models Developing Models for Effect Estimation Developing Models for Hypothesis Testing Further Reading 84 5 Resampling, Validating, Describing, and Simplifying the Model The Bootstrap Model Validation Introduction Which Quantities Should Be Used in Validation? Data-Splitting Improvements on Data-Splitting: Resampling Validation Using the Bootstrap Describing the Fitted Model Simplifying the Final Model by Approximating It Difficulties Using Full Models Approximating the Full Model Further Reading S-Plus Software 105

5 xvi Contents 6.1 The S Modeling Language User-Contributed Functions The Design Library Other Functions Further Reading Case Study in Least Squares Fitting and Interpretation of a Linear Model Descriptive Statistics Spending Degrees of Freedom/Specifying Predictor Complexity Fitting the Model Using Least Squares Checking Distributional Assumptions Checking Goodness of Fit Overly Influential Observations Test Statistics and Partial R Interpreting the Model Problems Case Study in Imputation and Data Reduction Data How Many Parameters Can Be Estimated? Variable Clustering Single Imputation Using Constants or Recursive Partitioning Transformation and Single Imputation Using transcan Data Reduction Using Principal Components Detailed Examination of Individual Transformations Examination of Variable Clusters on Transformed Variables Transformation Using Nonparametric Smoothers Multiple Imputation Further Reading Problems Overview of Maximum Likelihood Estimation General Notions Simple Cases Hypothesis Tests 183

6 Contents xvii Likelihood Ratio Test Wald Test Score Test Normal Distribution One Sample General Case Global Test Statistics Testing a Subset of the Parameters Which Test Statistics to Use When Example: Binomial Comparing Two Proportions Iterative ML Estimation Robust Estimation of the Covariance Matrix Wald, Score, and Likelihood-Based Confidence Intervals Bootstrap Confidence Regions Further Use of the Log Likelihood Rating Two Models, Penalizing for Complexity Testing Whether One Model Is Better than Another Unitless Index of Predictive Ability Unitless Index of Adequacy of a Subset of Predictors Weighted Maximum Likelihood Estimation Penalized Maximum Likelihood Estimation Further Reading Problems Binary Logistic Regression Model Model Assumptions and Interpretation of Parameters Odds Ratio, Risk Ratio, and Risk Difference Detailed Example Design Formulations Estimation Maximum Likelihood Estimates Estimation of Odds Ratios and Probabilities Test Statistics Residuals 230

7 xviii Contents 10.5 Assessment of Model Fit Collinearity Overly Influential Observations Quantifying Predictive Ability Validating the Fitted Model Describing the Fitted Model S-PLUS Functions Further Reading Problems Logistic Model Case Study 1: Predicting Cause of Death Preparation for Modeling Regression on Principal Components, Cluster Scores, and Pretransformations Fit and Diagnostics for a Full Model, and Interpreting Pretransformations Describing Results Using a Reduced Model Approximating the Full Model Using Recursive Partitioning Validating the Reduced Model Logistic Model Case Study 2: Survival of Titanic Passengers Descriptive Statistics Exploring Trends with Nonparametric Regression Binary Logistic Model With Casewise Deletion of Missing Values Examining Missing Data Patterns ' Single Conditional Mean Imputation Multiple Imputation Summarizing the Fitted Model Problems Ordinal Logistic Regression Background Ordinality Assumption Proportional Odds Model Model Assumptions and Interpretation of Parameters 333

8 Contents xix Estimation Residuals Assessment of Model Fit Quantifying Predictive Ability Validating the Fitted Model S-PLUS Functions Continuation Ratio Model Model Assumptions and Interpretation of Parameters Estimation Residuals Assessment of Model Fit Extended CR Model Role of Penalization in Extended CR Model Validating the Fitted Model S-PLUS Functions Further Reading Problems Case Study in Ordinal Regression, Data Reduction, and Penalization Response Variable Variable Clustering Developing Cluster Summary Scores Assessing Ordinality of Y for each X, and Unadjusted Checking of PO and CR Assumptions A Tentative Full Proportional Odds Model Residual Plots Graphical Assessment of Fit of CR Model Extended Continuation Ratio Model Penalized Estimation Using Approximations to Simplify the Model Validating the Model Summary Further Reading 371

9 xx Contents Problems Models Using Nonparametric Transformations of X and Y Background Generalized Additive Models Nonparametric Estimation of ^-Transformation Obtaining Estimates on the Original Scale S-PLUS Functions Case Study Introduction to Survival Analysis Background Censoring, Delayed Entry, and Truncation Notation, Survival, and Hazard Functions Homogeneous Failure Time Distributions Nonparametric Estimation of 5 and A Kaplan-Meier Estimator Altschuler-Nelson Estimator Analysis of Multiple Endpoints Competing Risks Competing Dependent Risks State Transitions and Multiple Types of Nonfatal Events Joint Analysis of Time and Severity of an Event Analysis of Multiple Events S-PLUS Functions Further Reading Problems Parametric Survival Models Homogeneous Models (No Predictors) Specific Models Estimation Assessment of Model Fit Parametric Proportional Hazards Models Model 417

10 Contents xxi Model Assumptions and Interpretation of Parameters Hazard Ratio, Risk Ratio, and Risk Difference Specific Models Estimation Assessment of Model Fit Accelerated Failure Time Models Model Model Assumptions and Interpretation of Parameters Specific Models Estimation Residuals Assessment of Model Fit Validating the Fitted Model Buckley-James Regression Model Design Formulations Test Statistics Quantifying Predictive Ability S-PLUS Functions Further Reading Problems Case Study in Parametric Survival Modeling and Model Approximation Descriptive Statistics Checking Adequacy of Log-Normal Accelerated Failure Time Model Summarizing the Fitted Model Internal Validation of the Fitted Model Using the Bootstrap Approximating the Full Model Problems Cox Proportional Hazards Regression Model Model Preliminaries Model Definition Estimation of/? 466

11 xxii Contents Model Assumptions and Interpretation of Parameters Example Design Formulations Extending the Model by Stratification Estimation of Survival Probability and Secondary Parameters Test Statistics Residuals Assessment of Model Fit Regression Assumptions Proportional Hazards Assumption What to Do When PH Fails Collinearity Overly Influential Observations Quantifying Predictive Ability Validating the Fitted Model Validation of Model Calibration Validation of Discrimination and Other Statistical Indexes Describing the Fitted Model S-PLUS Functions Further Reading Case Study in Cox Regression Choosing the Number of Parameters and Fitting the Model Checking Proportional Hazards Testing Interactions Describing Predictor Effects Validating the Model Presenting the Model Problems 522 Appendix 523 References 527 Index 559