Individual patient data meta-analysis of continuous diagnostic markers J.B. Reitsma Julius Center for Health Sciences and Primary Care UMC Utrecht / www.juliuscenter.nl
Outline IPD benefits Meta-analytical challenges ROC regression based on standardized marker values of cases Example study: mesothelin for diagnosing pleural mesothelioma
What is IPD? Traditional meta-analysis uses published summary (aggregate) data from primary studies Dataset: one row per included study with effect measure(s), their precision and study features Individual Patient Data meta-analysis combines the original (raw, crude) data from individual patients of different studies Dataset: each row is a individual patient with outcomes, patient and study characteristics
IPD benefits: general & diagnostic General IPD benefits: quality control & use of additional data & improve comparability Examining patient-level subgroups: more flexibility & more statistical power Continuous marker: handling of differences in cutoffs & better estimation of the summary ROC curve Statistical approach not straightforward
IPD Challenges Need to adjust for study when combining curves: Example overall shift in marker values Covariate adjusted ROC curve = weighted average of the study-specific curves Covariates affecting the ROC curve: Covariates have an impact on the overall accuracy: affect the separation between cases and controls) Covariate-specific ROC curves
Meta-analytical Challenges
ROC curves: need for adjustment Distribution of marker values of control groups differs between studies Mesothelin example: different kits by different studies
2 ROC curves: Covariate-specific (red) + Pooled ROC curve (black) Prevalence or P(D=1 Z=0) is 25% Prevalence or P(D=1 Z=1) is 75%
2 ROC curves: Covariate-specific (red) + Pooled ROC curve (black) Prevalence in Z=0 is 50% Prevalence in Z=1 is 50%
Cumulative distribution of Standardized Marker Approach
Standardized case marker approach Based on the approach of Janes & Pepe to adjust ROC curves for covariates in a single study Modified steps of their approach: 1. Estimate the cumulative distribution of the marker among controls as a function of study 2. Calculate the percentile values of the cases based on this distribution conditional on study 3. The cumulative distribution of these percentile values of cases is then the weighted ROC curve Janes & Pepe. Adjusting for covariates in studies of diagnostic, screening, or prognostic markers: an old concept in a new setting. Am J Epidmiol 2008.
Step 1 & 2: Percentile values of cases based on control distribution Specify how study (Z) affects the distribution of Y among controls: = β + β Z + ε Y 0 1 Calculate percentile values of cases using the control distribution standardized on Z as reference: pv DZ = Fˆ ( Y ( ˆ β + β 0 1Z )) Interpretation percentile values (pv): If a case has value Y and 20% of the corresponding controls have a value higher than Y, the pv =0.2 Also known as placement value
Calculate ROC curve using PV of cases ROC curve is the cumulative pv distribution of cases: ROC( f ) = P(1 pv f DZ ) To fit ROC, select a set (p) of discrete False Positive Rate values (f) spanning the interval (0,1) For each case create, n p records where the outcome is either U=1 or 0 A binary regression model with link g, outcome U and covariate g -1 (f) will generate the binormal (g=standard normal CDF) or bilogistic ROC curve Bootstrap to obtain standard errors or CI bands
Case Study: Mesothelin for Mesothelioma
Mesothelin for malignant mesothelioma Mesothelioma is asbestos-related malignancy with a poor prognosis in general Early diagnosis improves outcome, but diagnosis typically occurs in advanced stage due to non-specific complaints & slow onset Biomarkers may lead to an earlier diagnosis Problems identified in traditional reviews: - use of different cut-offs - non-standardized measurements - impact clinical subgroups Hollevoet et al. J Clin Oncol 2012
Hollevoet et al. Journal of Clinical Oncology 2012
Mesothelin example Hollevoet et al. J Clin Oncol 2012
Mesothelin example
Mesothelin example AUC = average of percentile values of cases Hollevoet et al. J Clin Oncol 2012
Discussion & Conclusion IPD data generate much more insight in the distribution of marker values across studies Better estimation of summary ROC curve and comparison of ROC curves between subgroups IPD no cure for poorly designed studies Approach of placement values appears attractive: flexible and use of existing techniques Comparison with alternative approaches needed