PREDICTIVE ANALYTICS AT THE BEDSIDE: The 5 W s. John W. Cromwell, MD

Transcription

1 PREDICTIVE ANALYTICS AT THE BEDSIDE: The 5 W s John W. Cromwell, MD

2 Disclosures Entac Medical, Inc. - Chief Medical Officer Apervita, Inc. - Consultant Dell, Inc. - Consultant

3 Cromwell Lab Analytics Developed machine learning device for predicting postoperative ileus Developed predictive analytics (PA) for readmissions Developed real-time analytics for predicting surgical site infections from the operating room Developing platform for predicting safety events Akpene Gbegnon, MD John Armstrong, MD

4 WHAT

5 ANALYTICS PAST Reporting Analytics PRESENT Dashboard Analytics FUTURE Predictive & Prescriptive Analytics

6 WHAT IS IT? Data Mining + Machine Learning = Predictive Analytics

7 Does it work? Google Self-Driving Car Over 6 years and 2 million miles on city streets in Mountain View, CA and Austin, TX Predictive analytics applied to imagery analysis 16 minor accidents None in which self-driving vehicle was responsible Blue CRUSH Memphis police began using PA to predict crime patterns based on historic data Re-distribute police based upon predicted crime 75% reduction in car-jackings, 67% reduction in business robberies

8 FEATURES OF PA MODELS Model can be trained and applied to wide range of populations or situations Model is constantly learning and refining through recalibration Models can be combined into ensembles for improved performance

9 WHY

10 Uses for predictive analytics Improve financial and administrative performance Improve operational effectiveness Improve clinical effectiveness

11 Why do we need bedside predictive analytics?

12 How far are we from error-free clinical decision-making? Over 12 months, 300,000 patients EHR screened for elevated prostate specific antigen, positive fecal occult blood test, rectal bleeding, and iron deficiency anemia Culled out patients with known cancer or appropriate follow up Chart review and follow up on 1500 patients without evidence of follow up Resulted in identifying 47 patients with high-grade cancer without appropriate follow up Translates to 13,600 diagnostic defects per 3.4 million patients Six-sigma defined as 1 defect in 3.4 million Murphy DR, Laxmisan A, Reis BA, et al. Electronic health record-based triggers to detect potential delays in cancer diagnosis. BMJ Qual Saf. 2014;23(1):8-16.

13 Cognitive Error Misdiagnosis is a source of medical error & waste Good example of a type of cognitive error

14 Clinical effectiveness by mitigating cognitive error Slips - incorrect execution of a correct action sequence Mistakes - correct execution of incorrect action sequence Reason, James. Human error. Cambridge University Press, 1990.

15 Cognitive Mistakes

16 ERRORS OF COGNITION Rate of diagnostic errors is conservatively 10-15% 40,000 deaths annually due to diagnostic errors in the ICU alone Diagnostic error rates are being measured in very few, if any, healthcare organisation in the USA. Graber, Mark L The Incidence of Diagnostic Error in Medicine. BMJ Quality & Safety Winters et al Diagnostic Errors in the Intensive Care Unit: A Systematic Review of Autopsy Studies. BMJ Quality & Safety

17 Physician Cognitive Error leads to inappropriate testing (over- or underuse) leads to inappropriate treatment leads to incorrect prognosis information for patients leads to patient death and injury

18

19 The most common types of preventable errors were technical errors (44%), errors in diagnosis (17%), failures to prevent injury (12%), and errors in the use of a drug (10%). Approximately 20% of technical errors, 71% of diagnostic errors, 50% of preventative errors, and 37% of errors in the use of a drug were judged to be negligent. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human. Washington DC: National Academy Press; 2015:1-311.

20 How do we currently deal with cognitive error? Re-education

21 Should we expect re-education to be effective? Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165(13):

22 Should we expect re-education to be effective? The failure to continue considering reasonable alternatives after an initial diagnosis was reached, was the single most common cognitive flaw. Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165(13):

23 Augmenting clinician cognition 1. Proactive 2. It works 3. Technology is available now

24 Effective Decision Support Kawamoto et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.

25 HOW SSI Illustration of Cognitive Limitations

26 Start with a problem - Readmissions Reason for readmission N % Surgical infection Gastrointestinal complication Dehydration Pulmonary complication Sepsis Wound disruption Deep vein thrombosis Hemorrhage Genitourinary complication Fever Urinary tract infection Renal failure Unknown 8 6.0

27 Surgical Site Infections Surgical site infections (SSIs) Infections that occur after surgery in the part of the body where the surgery took place Affect 2-11% of general surgery patients Increases postoperative morbidity (eg. pain, repeated procedures) $21,000-$29,000 of excess costs of care

28 Desired analytic to tailor preventive strategies to patients at high risk for surgical site infections prophylactic antibiotics wound care

29 Dataset Data warehouse developed from hospital EHR and NSQIP database (National Surgical Quality Improvement Program) Inclusion criteria: Patients who had undergone a general surgery operation, treated on a general surgery unit Training Set: 1/1/2011 9/30/13 Validation Set: 10/1/13 12/31/13 Outcome was any SSI (superficial, deep, organ space) within 30 days of operation

30 Dataset Basic Statistics Training Set Total cases: 2085 SSI rate: 8.4% Validation Set Total cases: 126 SSI rate: 8.7%

31 Predictor Variables Patient factors: Age Home ZIP code Sex Body mass index (BMI) Ethnicity Insurance company Location in hospital prior to operation Number of procedures during hospital encounter Presence of ostomy

32 Predictor Variables Disease factors: American Society of Anesthesiology score (ASA score)* Preoperative hemoglobin Last hemoglobin during hospitalization* Wound class* (clean, clean-contaminated, contaminated, or dirty/infected wound)

33 Predictor Variables Procedure factors: Surgical apgar score* (Intraoperative blood pressure, heart rate, estimated blood loss ) Blood transfusion* Minimum intraoperative temperature* Operating room* Duration of operation* Procedure type * (categorized by organ) Laparoscopic*or robotic vs open procedure Surgeon

34 Date Pre-Processing Text processed procedure name into 9 groups Appendix Colon Colonoscopy Gall bladder Hernia Pancreas Small intestines Stomach Other Laparoscopy designation

35 Grouping of Procedures Procedures Before text processing: 126 unique procedures After text processing: 9 unique procedure categories Colon Other Appendix Gall bladder Stomach Hernia Colonoscopy Pancreas Small Intestines Total SSI

36 Data Pre-Processing - Discretization Discretization Improves performance when data contains outliers ChiMerge algorithm Supervised discretization Create larger intervals by merging adjacent intervals that are similar according to a chi squared test (alpha=0.05)

37 Discretization of Continuous Variables Variable Discrete Intervals AGE PREHGB EBL TRANSFUSION BMI MIN_TEMP PROC_DURATION_MINS [ ] [ ] [ ] [ ] [ ] [ ] [ ] [0-31.5] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [6-41.5].. [ ] (36 intervals)

38 F Score and Cutoff F-Score It is an aggregated performance score A weighted average of the sensitivity and PPV (range 0-1) Cutoff F-score was calculated for each probability threshold The probability threshold with the highest Fscore was chosen as the cutoff

39

40 SVM Model Performance (All SSIs) Outcome (No SSI) Outcome (SSI) Prediction (no SSI) 80 4 Prediction (SSI) 35 7 n=126 AUC:.79 Sen: 0.64 ; PPV 0.17 Spec: 0.70 ; NPV 0.95

41 Cost Considerations Consider negative pressure wound therapy 60-80% effective at reducing SSI in high-risk wounds Total cost approximately $1500 We can address 64% of SSI by using therapy in ⅓ of patients

42 Variables not included in model that may be helpful: Co-morbidities Diabetes Smoker Malnutrition Preoperative colonization with Staph aureus Coexisting infection at remote body site Immunodeficiency Medications Steroid use Operative factors Antibiotic prophylaxis Duration of surgical scrub Foreign material at surgical site

43 How does this help with cognitive errors We are terrible at integrating more than a few data sources as humans Systems that integrate 10s or 100s or 1000s of variables can augment clinician decision-making to the advantage of patient and system PA is very good at classifying outliers

44 How does this help with cognitive errors Imagine an array of 10,000 variables from the EHR This array is generated on every patient Models are trained from these arrays on good & bad outcomes

45 WHEN

46 BARRIERS????

47 WHO

48 Who should be involved? Senior leadership representing VPMA, UIHC, College of Medicine, College of Nursing, UIP, ICTS Interdisciplinary data trust oversight team Content experts and leaders from all service lines Data mining, machine learning, and predictive analytics group Data warehouse support Human factors engineering Content Expert Data wrangler Analyst

49 Thank You!