Dealing with Missing Data

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Dealing with Missing Data"

Transcription

1 Dealing with Missing Data Roch Giorgi UMR 912 SESSTIM, Aix Marseille Université / INSERM / IRD, Marseille, France BioSTIC, APHM, Hôpital Timone, Marseille, France January 23, 2014 EPAAC WP9 Satellite Meeting Ispra (Italy)

2 Background (1) Importance of quality control is well known Covariate values may be missing for some subjects Collected routinely: tumor size, lymph node status, metastasis (mainly) Collected for specific studies: estrogen receptor, socioprofessional category, Missing values may concern Dependent variable: Time/Status in survival analysis Independent variable(s): tumor size, Whatever the question (incidence, survival, ) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 2

3 Background (2) Consequences of missing data Loss of irrelevant/non informative information No impact on estimates Loss of relevant/informative information Impact depends on the percentage of missing values Possible bias in both point estimates and standard errors Loss of statistical power Univariate/Multivariate analysis? Multivariate analysis: increase of the total percentage of missing values What can we do? Discard all the data set? Choose an appropriate method to perform analysis? Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 3

4 Objectives Present an overview of The types of missing data Some methods used to deal with missing data Provide outline guidelines Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 4

5 Missing Data Mechanism: Notations Y = ( y ) ij : (n x k) rectangular data set without missing values M = ( m ) ij m ij =1 if y ij is missing m ij =0 if y ij is present Defines the missingness pattern Univariate Y 1 Y 2 Y k 1 2 n? Monotone Y 1 Y 2 Y 3 Y 4 Y k 1 2???? n??? Non-Monotone Y 1 Y 2 Y 3 Y 4 Y k 1? 2???? n?? Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 5

6 Missing Data Mechanism: Classification Characterized by the conditional distribution of M given Y Missing Completely At Random (MCAR) Missingness mechanism independent of the values of the data Y (missing-y mis - or observed-y obs ) Missing At Random (MAR) Missingness mechanism depends only on Y obs, not on Y mis Missing Not At Random (MNAR) Missingness mechanism depends on Y mis Ignorable (MCAR, MAR) / Non-ignorable (MNAR) missing data Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 6

7 Missing Data Mechanism What do we learn with that? MCAR, MAR: handling missing data in an appropriate way do not need to model the missingness process Statistical tests H 0 : MCAR vs MAR? Yes H 0 : ignorable vs non-ignorable? No Classical methods used to handle missing data Provide valid statistical inferences with ignorable missing data Are not valid with non-ignorable missing data Sensitivity analyses under various scenarios of nonreponse when the MNAR hypothesis is suspected (e.g. self-reported characteristics as psychological disorders, quality of life, income, ) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 7

8 Classical Methods Complete cases Indicator variable Multiple imputation and others Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 8

9 Complete Cases Method Based only on the individuals having no missing values on the covariates included in the analysis The preferred method of many statistical softwares! Pos Easy to perform! but not necessarily a good point Unbiased results under MCAR hypothesis Neg Reduction of sample size Loss of statistical power Bias in standard errors Inappropriate variable selection (regression analysis) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 9

10 Indicator Variable Method Creation of a missing data indicator variable Treat missing data as just another category Pos Includes all the observations for the analysis No loss of statistical power May help to interpret results (similarity with another category) Neg Biased estimates (usually) May not help to interpret results (absence of similarity) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 10

11 Multiple Imputation (MI): Principle Step 1 MAR assumption Imputations of the missing values for M completed data sets Step 2 Analyze of each of these completed data sets estimates and standard errors Step 3 Combination to produce a single set of estimates with their standard errors 1...? Imputation model? 2...? Analysis model e 1 e 2 (se 1 )(se 2 ) e (se) M... e M (se M ) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 11

12 MI: Imputation of the Missing Values Goal: to account for the relationships between Y mis and Y obs, while taking into account the uncertainty of the imputation Y * ~ f Y Y ( ) mis obs Imputation model (non exhaustive) Continuous variable (e.g.: age at diagnosis): propensity methods, predictive mean matching Binary data (e.g.: M-stage): logistic regression Categorical data (e.g.: T-stage): polytomous logistic regression, proportional odds Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 12

13 MI: Analyses of the Completed Data Sets Analysis model: classical methods used to estimate Incidence Survival Effect of prognostic factors Independent analyses Each applied on the new completed data sets Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 13

14 MI: Combined Analysis Combination of the M estimates into an overall estimate and variance covariance matrix using Rubin s rules Take into account the uncertainty due to missing data Statistics that can be combined Mean, proportion, regression coefficient, Statistics that may require transformation Odds ratio, hazard ratio, baseline hazard, survival probability, Adapted from: White IR, et coll. Statistics in Medicine 2009 Statistics that cannot be combined P-value, likelihood ratio test statistic, Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 14

15 MI: Issue and Guidance for Practice (1) How many missing at most? Do not think in term of % of missing by covariate, but in term of reduction of % from the original data set when all variables used for the analyses are considered Think about the missingness mechanism Which variables to include in the imputation model? Covariates and outcome from the analysis model In survival model: status, time (t, log(t)) or cumulative baseline hazard function All predictors of the incomplete variable The number of variables in the imputation model may be greater than in the analysis model Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 15

16 MI: Issue and Guidance for Practice (2) Should we pay attention to the form of the imputation model? Yes in theory, hard to do (linearity? Interaction term?...) How many imputations are necessary? M=5-10 usually considered to be adequate Other rule exist based on the fraction of missing data Do we have to perform new imputations for each analysis? The imputed data set may be used for several analysis Need attention on the elaboration of the imputation model (more congenial ) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 16

17 MI: Issue and Guidance for Practice (3) Is there a particular model building strategy? Variable selection can be performed to all imputed data sets, or considering a single data set (after merging) with an appropriate weighting procedure Model checking could be performed on each imputed data set Prediction could be obtained using Rubin s rules How to be confident about the fact that the missingness mechanism is ignorable? Think about your data Perform sensitivity analysis Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 17

18 Thank you Roch Giorgi Challenges in the Estimation of Net SURvival working survival group French National Research Agency (ANR-12-BSV1-0028) Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 18

19 References Eisemann N, Waldmann A, Katalinic A. Imputation of missing values of tumour stage in population-based cancer registration. BMC Medical Research in Methodololgy 2011;11:129. Giorgi R, Belot A, Gaudart J, Launoy G; French Network of Cancer Registries FRANCIM. The performance of multiple imputation for missing covariate data within the context of regression relative survival analysis. Statistics in Medicine 2008;27(30): Howlader N, Noone AM, Yu M, Cronin KA. Use of imputed population-based cancer registry data as a method of accounting for missing information: application to estrogen receptor status for breast cancer. American Journal of Epidemiology 2012;176(4): Little RJA, Rubin DB. Statistical Analysis with Missing Data (2nd edn). Wiley: New York, Nur U, Shack LG, Rachet B, Carpenter JR, Coleman MP. Modelling relative survival in the presence of incomplete data: a tutorial. International Journal of Epidemiology 2010;39(1): Resseguier N, Giorgi R, Paoletti X. Sensitivity analysis when data are missing not-atrandom. Epidemiology 2011;22(2):282. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine 2011;30(4): Roch Giorgi, SESSTIM, Faculty of Medicine, Aix-Marseille University CENSUR working survival group 19

Missing data and net survival analysis Bernard Rachet

Missing data and net survival analysis Bernard Rachet Workshop on Flexible Models for Longitudinal and Survival Data with Applications in Biostatistics Warwick, 27-29 July 2015 Missing data and net survival analysis Bernard Rachet General context Population-based,

More information

Missing Data Dr Eleni Matechou

Missing Data Dr Eleni Matechou 1 Statistical Methods Principles Missing Data Dr Eleni Matechou matechou@stats.ox.ac.uk References: R.J.A. Little and D.B. Rubin 2nd edition Statistical Analysis with Missing Data J.L. Schafer and J.W.

More information

Missing Data in Longitudinal Studies: To Impute or not to Impute? Robert Platt, PhD McGill University

Missing Data in Longitudinal Studies: To Impute or not to Impute? Robert Platt, PhD McGill University Missing Data in Longitudinal Studies: To Impute or not to Impute? Robert Platt, PhD McGill University 1 Outline Missing data definitions Longitudinal data specific issues Methods Simple methods Multiple

More information

Introduction to mixed model and missing data issues in longitudinal studies

Introduction to mixed model and missing data issues in longitudinal studies Introduction to mixed model and missing data issues in longitudinal studies Hélène Jacqmin-Gadda INSERM, U897, Bordeaux, France Inserm workshop, St Raphael Outline of the talk I Introduction Mixed models

More information

Handling missing data in Stata a whirlwind tour

Handling missing data in Stata a whirlwind tour Handling missing data in Stata a whirlwind tour 2012 Italian Stata Users Group Meeting Jonathan Bartlett www.missingdata.org.uk 20th September 2012 1/55 Outline The problem of missing data and a principled

More information

Handling missing data in large data sets. Agostino Di Ciaccio Dept. of Statistics University of Rome La Sapienza

Handling missing data in large data sets. Agostino Di Ciaccio Dept. of Statistics University of Rome La Sapienza Handling missing data in large data sets Agostino Di Ciaccio Dept. of Statistics University of Rome La Sapienza The problem Often in official statistics we have large data sets with many variables and

More information

Multiple Imputation for Missing Data: A Cautionary Tale

Multiple Imputation for Missing Data: A Cautionary Tale Multiple Imputation for Missing Data: A Cautionary Tale Paul D. Allison University of Pennsylvania Address correspondence to Paul D. Allison, Sociology Department, University of Pennsylvania, 3718 Locust

More information

Missing Data: Part 1 What to Do? Carol B. Thompson Johns Hopkins Biostatistics Center SON Brown Bag 3/20/13

Missing Data: Part 1 What to Do? Carol B. Thompson Johns Hopkins Biostatistics Center SON Brown Bag 3/20/13 Missing Data: Part 1 What to Do? Carol B. Thompson Johns Hopkins Biostatistics Center SON Brown Bag 3/20/13 Overview Missingness and impact on statistical analysis Missing data assumptions/mechanisms Conventional

More information

A Basic Introduction to Missing Data

A Basic Introduction to Missing Data John Fox Sociology 740 Winter 2014 Outline Why Missing Data Arise Why Missing Data Arise Global or unit non-response. In a survey, certain respondents may be unreachable or may refuse to participate. Item

More information

Problem of Missing Data

Problem of Missing Data VASA Mission of VA Statisticians Association (VASA) Promote & disseminate statistical methodological research relevant to VA studies; Facilitate communication & collaboration among VA-affiliated statisticians;

More information

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis

Review of the Methods for Handling Missing Data in. Longitudinal Data Analysis Int. Journal of Math. Analysis, Vol. 5, 2011, no. 1, 1-13 Review of the Methods for Handling Missing Data in Longitudinal Data Analysis Michikazu Nakai and Weiming Ke Department of Mathematics and Statistics

More information

Missing Data. A Typology Of Missing Data. Missing At Random Or Not Missing At Random

Missing Data. A Typology Of Missing Data. Missing At Random Or Not Missing At Random [Leeuw, Edith D. de, and Joop Hox. (2008). Missing Data. Encyclopedia of Survey Research Methods. Retrieved from http://sage-ereference.com/survey/article_n298.html] Missing Data An important indicator

More information

Combining Multiple Imputation and Inverse Probability Weighting

Combining Multiple Imputation and Inverse Probability Weighting Combining Multiple Imputation and Inverse Probability Weighting Shaun Seaman 1, Ian White 1, Andrew Copas 2,3, Leah Li 4 1 MRC Biostatistics Unit, Cambridge 2 MRC Clinical Trials Unit, London 3 UCL Research

More information

Dealing with Missing Data

Dealing with Missing Data Res. Lett. Inf. Math. Sci. (2002) 3, 153-160 Available online at http://www.massey.ac.nz/~wwiims/research/letters/ Dealing with Missing Data Judi Scheffer I.I.M.S. Quad A, Massey University, P.O. Box 102904

More information

Challenges in Longitudinal Data Analysis: Baseline Adjustment, Missing Data, and Drop-out

Challenges in Longitudinal Data Analysis: Baseline Adjustment, Missing Data, and Drop-out Challenges in Longitudinal Data Analysis: Baseline Adjustment, Missing Data, and Drop-out Sandra Taylor, Ph.D. IDDRC BBRD Core 23 April 2014 Objectives Baseline Adjustment Introduce approaches Guidance

More information

MISSING DATA IMPUTATION IN CARDIAC DATA SET (SURVIVAL PROGNOSIS)

MISSING DATA IMPUTATION IN CARDIAC DATA SET (SURVIVAL PROGNOSIS) MISSING DATA IMPUTATION IN CARDIAC DATA SET (SURVIVAL PROGNOSIS) R.KAVITHA KUMAR Department of Computer Science and Engineering Pondicherry Engineering College, Pudhucherry, India DR. R.M.CHADRASEKAR Professor,

More information

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could

More information

Tips for surviving the analysis of survival data. Philip Twumasi-Ankrah, PhD

Tips for surviving the analysis of survival data. Philip Twumasi-Ankrah, PhD Tips for surviving the analysis of survival data Philip Twumasi-Ankrah, PhD Big picture In medical research and many other areas of research, we often confront continuous, ordinal or dichotomous outcomes

More information

Missing Data in Survival Analysis and Results from the MESS Trial

Missing Data in Survival Analysis and Results from the MESS Trial Missing Data in Survival Analysis and Results from the MESS Trial J. K. Rogers J. L. Hutton K. Hemming Department of Statistics University of Warwick Research Students Conference, 2008 Outline Background

More information

Re-analysis using Inverse Probability Weighting and Multiple Imputation of Data from the Southampton Women s Survey

Re-analysis using Inverse Probability Weighting and Multiple Imputation of Data from the Southampton Women s Survey Re-analysis using Inverse Probability Weighting and Multiple Imputation of Data from the Southampton Women s Survey MRC Biostatistics Unit Institute of Public Health Forvie Site Robinson Way Cambridge

More information

Sensitivity Analysis in Multiple Imputation for Missing Data

Sensitivity Analysis in Multiple Imputation for Missing Data Paper SAS270-2014 Sensitivity Analysis in Multiple Imputation for Missing Data Yang Yuan, SAS Institute Inc. ABSTRACT Multiple imputation, a popular strategy for dealing with missing values, usually assumes

More information

HANDLING DROPOUT AND WITHDRAWAL IN LONGITUDINAL CLINICAL TRIALS

HANDLING DROPOUT AND WITHDRAWAL IN LONGITUDINAL CLINICAL TRIALS HANDLING DROPOUT AND WITHDRAWAL IN LONGITUDINAL CLINICAL TRIALS Mike Kenward London School of Hygiene and Tropical Medicine Acknowledgements to James Carpenter (LSHTM) Geert Molenberghs (Universities of

More information

2. Making example missing-value datasets: MCAR, MAR, and MNAR

2. Making example missing-value datasets: MCAR, MAR, and MNAR Lecture 20 1. Types of missing values 2. Making example missing-value datasets: MCAR, MAR, and MNAR 3. Common methods for missing data 4. Compare results on example MCAR, MAR, MNAR data 1 Missing Data

More information

CONTINUOUS VARIABLES: TO CATEGORIZE OR TO MODEL?

CONTINUOUS VARIABLES: TO CATEGORIZE OR TO MODEL? CONTINUOUS VARIABLES: TO CATEGORIZE OR TO MODEL? Willi Sauerbrei 1 and Patrick Royston 2 1 IMBI, University Medical Center Freiburg, Germany 2 MRC Central Trials Unit, United Kingdom wfs@imbi.uni-freiburg.de

More information

Adequacy of Biomath. Models. Empirical Modeling Tools. Bayesian Modeling. Model Uncertainty / Selection

Adequacy of Biomath. Models. Empirical Modeling Tools. Bayesian Modeling. Model Uncertainty / Selection Directions in Statistical Methodology for Multivariable Predictive Modeling Frank E Harrell Jr University of Virginia Seattle WA 19May98 Overview of Modeling Process Model selection Regression shape Diagnostics

More information

Programme du parcours Clinical Epidemiology 2014-2015. UMR 1. Methods in therapeutic evaluation A Dechartres/A Flahault

Programme du parcours Clinical Epidemiology 2014-2015. UMR 1. Methods in therapeutic evaluation A Dechartres/A Flahault Programme du parcours Clinical Epidemiology 2014-2015 UR 1. ethods in therapeutic evaluation A /A Date cours Horaires 15/10/2014 14-17h General principal of therapeutic evaluation (1) 22/10/2014 14-17h

More information

Overview. Longitudinal Data Variation and Correlation Different Approaches. Linear Mixed Models Generalized Linear Mixed Models

Overview. Longitudinal Data Variation and Correlation Different Approaches. Linear Mixed Models Generalized Linear Mixed Models Overview 1 Introduction Longitudinal Data Variation and Correlation Different Approaches 2 Mixed Models Linear Mixed Models Generalized Linear Mixed Models 3 Marginal Models Linear Models Generalized Linear

More information

Imputing Missing Data using SAS

Imputing Missing Data using SAS ABSTRACT Paper 3295-2015 Imputing Missing Data using SAS Christopher Yim, California Polytechnic State University, San Luis Obispo Missing data is an unfortunate reality of statistics. However, there are

More information

Dealing with missing data: Key assumptions and methods for applied analysis

Dealing with missing data: Key assumptions and methods for applied analysis Technical Report No. 4 May 6, 2013 Dealing with missing data: Key assumptions and methods for applied analysis Marina Soley-Bori msoley@bu.edu This paper was published in fulfillment of the requirements

More information

Statistical Analysis with Missing Data

Statistical Analysis with Missing Data Statistical Analysis with Missing Data Second Edition RODERICK J. A. LITTLE DONALD B. RUBIN WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents Preface PARTI OVERVIEW AND BASIC APPROACHES

More information

SPSS TRAINING SESSION 3 ADVANCED TOPICS (PASW STATISTICS 17.0) Sun Li Centre for Academic Computing lsun@smu.edu.sg

SPSS TRAINING SESSION 3 ADVANCED TOPICS (PASW STATISTICS 17.0) Sun Li Centre for Academic Computing lsun@smu.edu.sg SPSS TRAINING SESSION 3 ADVANCED TOPICS (PASW STATISTICS 17.0) Sun Li Centre for Academic Computing lsun@smu.edu.sg IN SPSS SESSION 2, WE HAVE LEARNT: Elementary Data Analysis Group Comparison & One-way

More information

Missing data are ubiquitous in clinical research.

Missing data are ubiquitous in clinical research. Advanced Statistics: Missing Data in Clinical Research Part 1: An Introduction and Conceptual Framework Jason S. Haukoos, MD, MS, Craig D. Newgard, MD, MPH Abstract Missing data are commonly encountered

More information

Missing Data & How to Deal: An overview of missing data. Melissa Humphries Population Research Center

Missing Data & How to Deal: An overview of missing data. Melissa Humphries Population Research Center Missing Data & How to Deal: An overview of missing data Melissa Humphries Population Research Center Goals Discuss ways to evaluate and understand missing data Discuss common missing data methods Know

More information

Item Imputation Without Specifying Scale Structure

Item Imputation Without Specifying Scale Structure Original Article Item Imputation Without Specifying Scale Structure Stef van Buuren TNO Quality of Life, Leiden, The Netherlands University of Utrecht, The Netherlands Abstract. Imputation of incomplete

More information

STAT 5817: Logistic Regression and Odds Ratio

STAT 5817: Logistic Regression and Odds Ratio A cohort of people is a group of people whose membership is clearly defined. A prospective study is one in which a cohort of people is followed for the occurrence or nonoccurrence of specified endpoints

More information

Applied Missing Data Analysis in the Health Sciences. Statistics in Practice

Applied Missing Data Analysis in the Health Sciences. Statistics in Practice Brochure More information from http://www.researchandmarkets.com/reports/2741464/ Applied Missing Data Analysis in the Health Sciences. Statistics in Practice Description: A modern and practical guide

More information

Nonrandomly Missing Data in Multiple Regression Analysis: An Empirical Comparison of Ten Missing Data Treatments

Nonrandomly Missing Data in Multiple Regression Analysis: An Empirical Comparison of Ten Missing Data Treatments Brockmeier, Kromrey, & Hogarty Nonrandomly Missing Data in Multiple Regression Analysis: An Empirical Comparison of Ten s Lantry L. Brockmeier Jeffrey D. Kromrey Kristine Y. Hogarty Florida A & M University

More information

Handling attrition and non-response in longitudinal data

Handling attrition and non-response in longitudinal data Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

More information

Missing Data. Paul D. Allison INTRODUCTION

Missing Data. Paul D. Allison INTRODUCTION 4 Missing Data Paul D. Allison INTRODUCTION Missing data are ubiquitous in psychological research. By missing data, I mean data that are missing for some (but not all) variables and for some (but not all)

More information

A Mixed Model Approach for Intent-to-Treat Analysis in Longitudinal Clinical Trials with Missing Values

A Mixed Model Approach for Intent-to-Treat Analysis in Longitudinal Clinical Trials with Missing Values Methods Report A Mixed Model Approach for Intent-to-Treat Analysis in Longitudinal Clinical Trials with Missing Values Hrishikesh Chakraborty and Hong Gu March 9 RTI Press About the Author Hrishikesh Chakraborty,

More information

Epidemiology-Biostatistics Exam Exam 2, 2001 PRINT YOUR LEGAL NAME:

Epidemiology-Biostatistics Exam Exam 2, 2001 PRINT YOUR LEGAL NAME: Epidemiology-Biostatistics Exam Exam 2, 2001 PRINT YOUR LEGAL NAME: Instructions: This exam is 30% of your course grade. The maximum number of points for the course is 1,000; hence this exam is worth 300

More information

Analyzing Structural Equation Models With Missing Data

Analyzing Structural Equation Models With Missing Data Analyzing Structural Equation Models With Missing Data Craig Enders* Arizona State University cenders@asu.edu based on Enders, C. K. (006). Analyzing structural equation models with missing data. In G.

More information

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence Survival Analysis Using SPSS By Hui Bian Office for Faculty Excellence Survival analysis What is survival analysis Event history analysis Time series analysis When use survival analysis Research interest

More information

, then the form of the model is given by: which comprises a deterministic component involving the three regression coefficients (

, then the form of the model is given by: which comprises a deterministic component involving the three regression coefficients ( Multiple regression Introduction Multiple regression is a logical extension of the principles of simple linear regression to situations in which there are several predictor variables. For instance if we

More information

Regression Modeling Strategies

Regression Modeling Strategies Frank E. Harrell, Jr. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis With 141 Figures Springer Contents Preface Typographical Conventions

More information

Checking proportionality for Cox s regression model

Checking proportionality for Cox s regression model Checking proportionality for Cox s regression model by Hui Hong Zhang Thesis for the degree of Master of Science (Master i Modellering og dataanalyse) Department of Mathematics Faculty of Mathematics and

More information

Logistic Regression: Basics

Logistic Regression: Basics Evidence is no evidence if based solely on p value Logistic Regression: Basics Prediction Model: Binary Outcomes Nemours Stats 101 Laurens Holmes, Jr. General Linear Model OUTCOME Continuous Counts Data

More information

Missing values in data analysis: Ignore or Impute?

Missing values in data analysis: Ignore or Impute? ORIGINAL ARTICLE Missing values in data analysis: Ignore or Impute? Ng Chong Guan 1, Muhamad Saiful Bahri Yusoff 2 1 Department of Psychological Medicine, Faculty of Medicine, University Malaya 2 Medical

More information

A REVIEW OF CURRENT SOFTWARE FOR HANDLING MISSING DATA

A REVIEW OF CURRENT SOFTWARE FOR HANDLING MISSING DATA 123 Kwantitatieve Methoden (1999), 62, 123-138. A REVIEW OF CURRENT SOFTWARE FOR HANDLING MISSING DATA Joop J. Hox 1 ABSTRACT. When we deal with a large data set with missing data, we have to undertake

More information

Modern Methods for Missing Data

Modern Methods for Missing Data Modern Methods for Missing Data Paul D. Allison, Ph.D. Statistical Horizons LLC www.statisticalhorizons.com 1 Introduction Missing data problems are nearly universal in statistical practice. Last 25 years

More information

Bayesian Approaches to Handling Missing Data

Bayesian Approaches to Handling Missing Data Bayesian Approaches to Handling Missing Data Nicky Best and Alexina Mason BIAS Short Course, Jan 30, 2012 Lecture 1. Introduction to Missing Data Bayesian Missing Data Course (Lecture 1) Introduction to

More information

Title: Categorical Data Imputation Using Non-Parametric or Semi-Parametric Imputation Methods

Title: Categorical Data Imputation Using Non-Parametric or Semi-Parametric Imputation Methods Masters by Coursework and Research Report Mathematical Statistics School of Statistics and Actuarial Science Title: Categorical Data Imputation Using Non-Parametric or Semi-Parametric Imputation Methods

More information

(Brief rest break when appealing) Please silence your cell phones and pagers. Invite brief questions of clarification.

(Brief rest break when appealing) Please silence your cell phones and pagers. Invite brief questions of clarification. The Ugly, the Bad, and the Good of Missing and Dropout Data in Analysis and Sample Size Selection Keith E. Muller Professor and Director of the Division of Biostatistics, Epidemiology and Health Policy

More information

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not. Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

More information

Prognosis of survival for breast cancer patients

Prognosis of survival for breast cancer patients Prognosis of survival for breast cancer patients Ken Ryder Breast Cancer Unit Data Section Guy s Hospital Patrick Royston, MRC Clinical Trials Unit London Outline Introduce the data and outcomes requested

More information

Dealing with Missing Data for Credit Scoring

Dealing with Missing Data for Credit Scoring ABSTRACT Paper SD-08 Dealing with Missing Data for Credit Scoring Steve Fleming, Clarity Services Inc. How well can statistical adjustments account for missing data in the development of credit scores?

More information

IBM SPSS Missing Values 22

IBM SPSS Missing Values 22 IBM SPSS Missing Values 22 Note Before using this information and the product it supports, read the information in Notices on page 23. Product Information This edition applies to version 22, release 0,

More information

Ordinal Regression. Chapter

Ordinal Regression. Chapter Ordinal Regression Chapter 4 Many variables of interest are ordinal. That is, you can rank the values, but the real distance between categories is unknown. Diseases are graded on scales from least severe

More information

Introduction to Fixed Effects Methods

Introduction to Fixed Effects Methods Introduction to Fixed Effects Methods 1 1.1 The Promise of Fixed Effects for Nonexperimental Research... 1 1.2 The Paired-Comparisons t-test as a Fixed Effects Method... 2 1.3 Costs and Benefits of Fixed

More information

Study Design and Statistical Analysis

Study Design and Statistical Analysis Study Design and Statistical Analysis Anny H Xiang, PhD Department of Preventive Medicine University of Southern California Outline Designing Clinical Research Studies Statistical Data Analysis Designing

More information

Analysis of Longitudinal Data with Missing Values.

Analysis of Longitudinal Data with Missing Values. Analysis of Longitudinal Data with Missing Values. Methods and Applications in Medical Statistics. Ingrid Garli Dragset Master of Science in Physics and Mathematics Submission date: June 2009 Supervisor:

More information

TABLE OF CONTENTS ALLISON 1 1. INTRODUCTION... 3

TABLE OF CONTENTS ALLISON 1 1. INTRODUCTION... 3 ALLISON 1 TABLE OF CONTENTS 1. INTRODUCTION... 3 2. ASSUMPTIONS... 6 MISSING COMPLETELY AT RANDOM (MCAR)... 6 MISSING AT RANDOM (MAR)... 7 IGNORABLE... 8 NONIGNORABLE... 8 3. CONVENTIONAL METHODS... 10

More information

Missing Data. Katyn & Elena

Missing Data. Katyn & Elena Missing Data Katyn & Elena What to do with Missing Data Standard is complete case analysis/listwise dele;on ie. Delete cases with missing data so only complete cases are le> Two other popular op;ons: Mul;ple

More information

PATTERN MIXTURE MODELS FOR MISSING DATA. Mike Kenward. London School of Hygiene and Tropical Medicine. Talk at the University of Turku,

PATTERN MIXTURE MODELS FOR MISSING DATA. Mike Kenward. London School of Hygiene and Tropical Medicine. Talk at the University of Turku, PATTERN MIXTURE MODELS FOR MISSING DATA Mike Kenward London School of Hygiene and Tropical Medicine Talk at the University of Turku, April 10th 2012 1 / 90 CONTENTS 1 Examples 2 Modelling Incomplete Data

More information

Module 14: Missing Data Stata Practical

Module 14: Missing Data Stata Practical Module 14: Missing Data Stata Practical Jonathan Bartlett & James Carpenter London School of Hygiene & Tropical Medicine www.missingdata.org.uk Supported by ESRC grant RES 189-25-0103 and MRC grant G0900724

More information

Statistical modelling with missing data using multiple imputation. Session 4: Sensitivity Analysis after Multiple Imputation

Statistical modelling with missing data using multiple imputation. Session 4: Sensitivity Analysis after Multiple Imputation Statistical modelling with missing data using multiple imputation Session 4: Sensitivity Analysis after Multiple Imputation James Carpenter London School of Hygiene & Tropical Medicine Email: james.carpenter@lshtm.ac.uk

More information

Longitudinal Data Analysis. Wiley Series in Probability and Statistics

Longitudinal Data Analysis. Wiley Series in Probability and Statistics Brochure More information from http://www.researchandmarkets.com/reports/2172736/ Longitudinal Data Analysis. Wiley Series in Probability and Statistics Description: Longitudinal data analysis for biomedical

More information

Methods for Meta-analysis in Medical Research

Methods for Meta-analysis in Medical Research Methods for Meta-analysis in Medical Research Alex J. Sutton University of Leicester, UK Keith R. Abrams University of Leicester, UK David R. Jones University of Leicester, UK Trevor A. Sheldon University

More information

Data Cleaning and Missing Data Analysis

Data Cleaning and Missing Data Analysis Data Cleaning and Missing Data Analysis Dan Merson vagabond@psu.edu India McHale imm120@psu.edu April 13, 2010 Overview Introduction to SACS What do we mean by Data Cleaning and why do we do it? The SACS

More information

Imputation of missing data under missing not at random assumption & sensitivity analysis

Imputation of missing data under missing not at random assumption & sensitivity analysis Imputation of missing data under missing not at random assumption & sensitivity analysis S. Jolani Department of Methodology and Statistics, Utrecht University, the Netherlands Advanced Multiple Imputation,

More information

Personalized Predictive Medicine and Genomic Clinical Trials

Personalized Predictive Medicine and Genomic Clinical Trials Personalized Predictive Medicine and Genomic Clinical Trials Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov brb.nci.nih.gov Powerpoint presentations

More information

Module 3: Multiple Regression Concepts

Module 3: Multiple Regression Concepts Contents Module 3: Multiple Regression Concepts Fiona Steele 1 Centre for Multilevel Modelling...4 What is Multiple Regression?... 4 Motivation... 4 Conditioning... 4 Data for multiple regression analysis...

More information

How to choose an analysis to handle missing data in longitudinal observational studies

How to choose an analysis to handle missing data in longitudinal observational studies How to choose an analysis to handle missing data in longitudinal observational studies ICH, 25 th February 2015 Ian White MRC Biostatistics Unit, Cambridge, UK Plan Why are missing data a problem? Methods:

More information

EBM Cheat Sheet- Measurements Card

EBM Cheat Sheet- Measurements Card EBM Cheat Sheet- Measurements Card Basic terms: Prevalence = Number of existing cases of disease at a point in time / Total population. Notes: Numerator includes old and new cases Prevalence is cross-sectional

More information

Missing data in randomized controlled trials (RCTs) can

Missing data in randomized controlled trials (RCTs) can EVALUATION TECHNICAL ASSISTANCE BRIEF for OAH & ACYF Teenage Pregnancy Prevention Grantees May 2013 Brief 3 Coping with Missing Data in Randomized Controlled Trials Missing data in randomized controlled

More information

Komorbide brystkræftpatienter kan de tåle behandling? Et registerstudie baseret på Danish Breast Cancer Cooperative Group

Komorbide brystkræftpatienter kan de tåle behandling? Et registerstudie baseret på Danish Breast Cancer Cooperative Group Komorbide brystkræftpatienter kan de tåle behandling? Et registerstudie baseret på Danish Breast Cancer Cooperative Group Lotte Holm Land MD, ph.d. Onkologisk Afd. R. OUH Kræft og komorbiditet - alle skal

More information

Coffee and Amyotrophic Lateral Sclerosis (ALS): A Possible Preventive Role

Coffee and Amyotrophic Lateral Sclerosis (ALS): A Possible Preventive Role Coffee and Amyotrophic Lateral Sclerosis (ALS): A Possible Preventive Role Ettore Beghi, Elisabetta Pupillo, Paolo Messina, et al. American Journal of Epidemiology 2011; 174(9): 1002-1008 What is Amyotrophic

More information

Part 3: Methods for Propensity Analysis

Part 3: Methods for Propensity Analysis Part 3: Methods for Propensity Analysis SMDM Short Course on Propensity Charles Thomas Center for Health Care Research & Policy Outline Definition of the Propensity Score Estimating the Propensity Score

More information

Journal Article Reporting Standards (JARS)

Journal Article Reporting Standards (JARS) APPENDIX Journal Article Reporting Standards (JARS), Meta-Analysis Reporting Standards (MARS), and Flow of Participants Through Each Stage of an Experiment or Quasi-Experiment 245 Journal Article Reporting

More information

Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP

Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP ABSTRACT In data mining modelling, data preparation

More information

AVOIDING BIAS AND RANDOM ERROR IN DATA ANALYSIS

AVOIDING BIAS AND RANDOM ERROR IN DATA ANALYSIS AVOIDING BIAS AND RANDOM ERROR IN DATA ANALYSIS Susan Ellenberg, Ph.D. Perelman School of Medicine University of Pennsylvania School of Medicine FDA Clinical Investigator Course White Oak, MD November

More information

Gerry Hobbs, Department of Statistics, West Virginia University

Gerry Hobbs, Department of Statistics, West Virginia University Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

More information

SPSS Multivariable Linear Models and Logistic Regression

SPSS Multivariable Linear Models and Logistic Regression 1 SPSS Multivariable Linear Models and Logistic Regression Multivariable Models Single continuous outcome (dependent variable), one main exposure (independent) variable, and one or more potential confounders

More information

Social inequalities impacts of care management and survival in patients with non-hodgkin lymphomas (ISO-LYMPH)

Social inequalities impacts of care management and survival in patients with non-hodgkin lymphomas (ISO-LYMPH) Session 3 : Epidemiology and public health Social inequalities impacts of care management and survival in patients with non-hodgkin lymphomas (ISO-LYMPH) Le Guyader-Peyrou Sandra Bergonie Institut Context:

More information

Bra wearing not associated with breast cancer risk: a population based case-control study

Bra wearing not associated with breast cancer risk: a population based case-control study Bra wearing not associated with breast cancer risk: a population based case-control study Authors and affiliations: Lu Chen 1 Kathleen E. Malone 1 Christopher I. Li 1 1 Division of Public Health Sciences,

More information

Statistical Rules of Thumb

Statistical Rules of Thumb Statistical Rules of Thumb Second Edition Gerald van Belle University of Washington Department of Biostatistics and Department of Environmental and Occupational Health Sciences Seattle, WA WILEY AJOHN

More information

INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

More information

13. Poisson Regression Analysis

13. Poisson Regression Analysis 136 Poisson Regression Analysis 13. Poisson Regression Analysis We have so far considered situations where the outcome variable is numeric and Normally distributed, or binary. In clinical work one often

More information

Workpackage 11 Imputation and Non-Response. Deliverable 11.2

Workpackage 11 Imputation and Non-Response. Deliverable 11.2 Workpackage 11 Imputation and Non-Response Deliverable 11.2 2004 II List of contributors: Seppo Laaksonen, Statistics Finland; Ueli Oetliker, Swiss Federal Statistical Office; Susanne Rässler, University

More information

APPLIED MISSING DATA ANALYSIS

APPLIED MISSING DATA ANALYSIS APPLIED MISSING DATA ANALYSIS Craig K. Enders Series Editor's Note by Todd D. little THE GUILFORD PRESS New York London Contents 1 An Introduction to Missing Data 1 1.1 Introduction 1 1.2 Chapter Overview

More information

Imputing Attendance Data in a Longitudinal Multilevel Panel Data Set

Imputing Attendance Data in a Longitudinal Multilevel Panel Data Set Imputing Attendance Data in a Longitudinal Multilevel Panel Data Set April 2015 SHORT REPORT Baby FACES 2009 This page is left blank for double-sided printing. Imputing Attendance Data in a Longitudinal

More information

Craig K. Enders Arizona State University Department of Psychology craig.enders@asu.edu

Craig K. Enders Arizona State University Department of Psychology craig.enders@asu.edu Craig K. Enders Arizona State University Department of Psychology craig.enders@asu.edu Topic Page Missing Data Patterns And Missing Data Mechanisms 1 Traditional Missing Data Techniques 7 Maximum Likelihood

More information

An extension of the factoring likelihood approach for non-monotone missing data

An extension of the factoring likelihood approach for non-monotone missing data An extension of the factoring likelihood approach for non-monotone missing data Jae Kwang Kim Dong Wan Shin January 14, 2010 ABSTRACT We address the problem of parameter estimation in multivariate distributions

More information

Lecture 7 Logistic Regression with Random Intercept

Lecture 7 Logistic Regression with Random Intercept Lecture 7 Logistic Regression with Random Intercept Logistic Regression Odds: expected number of successes for each failure P(y log i x i ) = β 1 + β 2 x i 1 P(y i x i ) log{ Od(y i =1 x i =a +1) } log{

More information

SAMPLE SIZE TABLES FOR LOGISTIC REGRESSION

SAMPLE SIZE TABLES FOR LOGISTIC REGRESSION STATISTICS IN MEDICINE, VOL. 8, 795-802 (1989) SAMPLE SIZE TABLES FOR LOGISTIC REGRESSION F. Y. HSIEH* Department of Epidemiology and Social Medicine, Albert Einstein College of Medicine, Bronx, N Y 10461,

More information

Prognosis of Patients With Breast Cancer: Causes of Death and Effects of Time Since Diagnosis, Age, and Tumor Characteristics

Prognosis of Patients With Breast Cancer: Causes of Death and Effects of Time Since Diagnosis, Age, and Tumor Characteristics Prognosis of Patients With Breast Cancer: Causes of Death and Effects of Time Since Diagnosis, Age, and Tumor Characteristics Edoardo Colzani, M.D., M.P.H. Department of Medical Epidemiology and Biostatistics

More information

Early mortality rate (EMR) in Acute Myeloid Leukemia (AML)

Early mortality rate (EMR) in Acute Myeloid Leukemia (AML) Early mortality rate (EMR) in Acute Myeloid Leukemia (AML) George Yaghmour, MD Hematology Oncology Fellow PGY5 UTHSC/West cancer Center, Memphis, TN May,1st,2015 Off-Label Use Disclosure(s) I do not intend

More information

Electronic Theses and Dissertations UC Riverside

Electronic Theses and Dissertations UC Riverside Electronic Theses and Dissertations UC Riverside Peer Reviewed Title: Bayesian and Non-parametric Approaches to Missing Data Analysis Author: Yu, Yao Acceptance Date: 01 Series: UC Riverside Electronic

More information

CHOOSING APPROPRIATE METHODS FOR MISSING DATA IN MEDICAL RESEARCH: A DECISION ALGORITHM ON METHODS FOR MISSING DATA

CHOOSING APPROPRIATE METHODS FOR MISSING DATA IN MEDICAL RESEARCH: A DECISION ALGORITHM ON METHODS FOR MISSING DATA CHOOSING APPROPRIATE METHODS FOR MISSING DATA IN MEDICAL RESEARCH: A DECISION ALGORITHM ON METHODS FOR MISSING DATA Hatice UENAL Institute of Epidemiology and Medical Biometry, Ulm University, Germany

More information

A Guide to Imputing Missing Data with Stata Revision: 1.4

A Guide to Imputing Missing Data with Stata Revision: 1.4 A Guide to Imputing Missing Data with Stata Revision: 1.4 Mark Lunt December 6, 2011 Contents 1 Introduction 3 2 Installing Packages 4 3 How big is the problem? 5 4 First steps in imputation 5 5 Imputation

More information