Prognosis: diagnosing the future. The temporal dimension of diagnosis. Predictive models Gennaro D Amico Gastroenterology Unit V Cervello Hospital Palermo gedamico@libero.it
Diagnosing the future The link between diagnosis and prognosis Patients mainly benefit from medical testing if it affects future medical decisions This implies that diagnostic tests are clinically relevant if they not only inform on the disease status but also on the future outcome Hence the relevance of diagnostic information is closely related to prognosis: the implication for the future course of the patient s condition
The temporal dimension of diagnosis Cirrhosis: survival probability Ratib S J Hepatol 2014;60:282-289 Diagnosis of cirrhosis Future implication of diagnosis of cirrhosis
The temporal dimension of diagnosis Esophageal varices in cirrhosis 1.00 0.75 No varices Small varices 0.50 0.25 Large varices 0.00 % free of bleeding 0 50 100 Dig Dis Sci 1986; 31: 468-75; APT 2014;39:1180 1193
Investigating the temporal dimension of diagnosis Clinical condition Test + - Follow-up Follow-up outcome outcome Lijmer G.J. J Clin Epidemiol 2009;62:364-373
The temporal dimension of diagnosis is the prediction of the patient risk and is the basis of prognosis
Prognosis and predictive models Prognosis Possible outcomes of a disease and the probability with which they can be expected to occur. Predictive (or prognostic) factor Factor, from history, physical examination or diagnostic tests, associated with the risk (probability), or with the rate (incidence), of an event Predictive (or prognostic) models Decision-making tools for clinicians, combining several predictive factors to predict the patient risk of an event
Rate (Incidence) A measure of the average incidence of a clinical event. It is computed as total number of events / total follow-up time it measures the speed of the occurrence of an event. Example: Ten patients are followed for ten years each. At the end of follow-up, five patients develop the event of interest while the other five are censored. The total follow-up time is 10*10 = 100 person-years, and there are five events: the incidence rate of is 5 / 100 = 1 per 20 person-years
Risk (Hazard) It is the probability that a patient who has not experienced the outcome at time t will experience the outcome in the next time interval Kaplan-Meier curves show the cumulative probability of survival at each observation time. The cumulative risk is 1-survival
Proportion surviving 0.00 0.25 0.50 0.75 1.00 Single prognostic indicator of death in cirrhosis Ascites No ascites at diagnosis n=995 Ascites at diagnosis n=654 Log-rank test chi2(1) = 392.46 Pr>chi2 = 0.0000 Total n=1649 0 24 48 72 96 120 months Cumalative data from Dig Dis Sci 1986; 31: 468-75; Gastroenterology 2001; 120: A2
Proportion surviving 0.00 0.25 0.50 0.75 1.00 Single prognostic indicator of death in cirrhosis Esophageal varices No varices (n=631) Log-rank test chi2(1) = 44.01 Pr>chi2 = 0.0000 Varices (n=490) 0 24 48 72 96 120 months Cumalative data from Dig Dis Sci 1986; 31: 468-75; Gastroenterology 2001; 120: A2
Prognostic indicators of death in cirrhosis Ascites Worse outcome Cirrhosis No ascites Varices Better outcome Worse outcome Cirrhosis No varices Better outcome
Proportion surviving Multiple prognostic indicators of death in cirrhosis Esophageal Varices and Ascites 0.00 0.25 0.50 0.75 1.00 ascites, no varices n=141 ascites, and varices n=208 Log-rank test chi2(3) = 265.83 Pr>chi2 = 0.0000 0 24 48 72 96 120 months No varices, no ascites n=490 Varices, no ascites n=282 Cumalative data from Dig Dis Sci 1986; 31: 468-75; Gastroenterology 2001; 120: A2
Example of prognostic indicators No Ascites No Varices Mortality% 2yr 5yr 8 20 Varices 17 39 Cirrhosis No Varices 36 73 Ascites Varices 54 68
Prognostic index (or prognostic model) A prognostic index is derived by an appropriate multiple regression analyses In the general format, each signifcant variable is multiplied by a coefficient provided by the statistical analysis
How does it work Simulation of a prognostic index for mortality in cirrhosis Cox model multivariable analysis Variable rating Coefficient p Ascites Varices No=0 Yes=1 No=0 Yes=1 1.36 0.000 0.36 0.000 Prognostic index (PI) in the individual patient PI = (Ascites * 1.36) + (Varices* 0.36) Hazard = e PI
Prognostic index (PI) in the individual patient PI = (Ascites * 1.36) + (Varices* 0.36) Hazard = e PI No ascites, no varices 0 1 PI Hazard No ascites, varices 0+0.36 1.47 Ascites, no varices 1.36+0 3.90 Ascites and varices 1.36+0.36 5.58
Performance of prognostic models Discrimination Calibration Impact Relly BM. Ann Int Med 2006;144;201-209 Justice AC. Ann Int Med 1999;130:515-524
Discrimination Discrimination is the ability of a prediction tool to predict: a) a higher probability of the outcome for those patients who will ultimately suffer the outcome; and b) a lower probability of the outcome for those patients who will not.
Sensitivity Discrimination Model for End stage Liver Disease (MELD) MELD ability in discriminating alive from dead after one year 1.00 0.75 High MELD Low MELD 0.50 0.25 491 consecutive patients with cirrhosis ROC area: 0.78 0.00 0.00 0.25 0.50 0.75 1.00 1 - Specificity Data from a prospective cohort included in Kamath P Hepatology 2002;33:464-470
Calibration Calibration is the ability of a prognostic model to predict event rates on a new series of patients It is measured by comparing the observed and predicted event rates or risks
Kaplan-Meier observed HCC risk Calibration Prediction of HCC in HBV + The REACH-B score Mayo model for PBC Mean risk-score predicted HCC risk Yang HI. Lancet Oncol 2011;12:568 74 Grambsch PM & al. Hepatology 1989; 10:846-850
Impact The Ottawa ankle rule Predictive rule: possible fracture if pain at posterior edge or lower tip of medial or lateral malleolus or inhability to bear weigh (sensitivity 100%, specificity 40%) In a validation RCT of 12770 patients (sensitivity 99.4%) a 26% reduction of radiological examinations was obtained A subsequent cost-effectiveness study showed an economy of 3,145,000 $US per 100,000 pts Stiell IG. Br Med J 1995;311:594-597; Anis AH. Ann Emerg Med 1995;26:422-428; Perry JJ. Int J Care Inj 2006;37:1157-1165
How can I apply the results from prognostic studies to my single patient? Derivation sample Application to single patients Generalizability?
Appraising prognostic studies for use in individual patient Are the results valid? How does it work? Is the prediction tool applicable to Individual patients? Internal Validity Discrimination Calibration Use in individual patients
Internal validity: Domains of potential biases The Quality In Prognosis Studies (QUIPS) TOOL Domain Objective Indicators of bias 1. Study participation The study sample represents the population of interest 2. Study attrition Loss to follow-up independent of key characteristics Key characteristics not reported Incomplete follow-up 3. Prognostic factors Adequate measurement Definition and 4. Outcome 5. Confounding Potential confounders appropriately accounted for measurement not adequately reported 6. Analysis Appropriate for the study design Description of method and presentation of data Hayden JA Ann Int Med 2013;158:280-286
Can I use the prediction tool in my patient care? Item Generalizability External validation Importance My patient s characteristics Follow-up and outcome Assessment Reproducibility: validation of the results in patients from the identical population Transportability: validation in patients from different but related population Broad impact analysis of the prediction rule as a decision rule Are my patient s characteristics and management similar to those of patients in the validation studies? Was the follow-up long enough as to assess the outcome of interest in my patient?
Can I use the results in the management of my patient? Does the expected probability of outcome cross a decision threshold? 0% threshold threshold 100% Reassure Observe Intervene JAMAevidence 2011
Key concepts Diagnostic tests bear prognostic information Prognostic models allow to predict outcome risks or rates in patient-subgroups A prediction tool must be valid, reproducible and generalizable, to be applied To apply a prediction tool in individual patients, the patient characteristics must be similar to those of patient samples generating and validating the prediction tool Application of the prediction tool is expected to provide benefit to the patient