Lessons Learned from Advanced Analytics at the VA VETERANS HEALTH ADMINISTRATION

Transcription

1 Lessons Learned from Advanced Analytics at the VA

2 Lessons Learned from Advanced Analytics at the VA SD Fihn MD MPH Office of Analytics and Business Intelligence Veterans Health Administration Professor of Medicine and Health Services University of Washington November 21, 2014

3 Veterans Health Administration 2013/14 Enrollees 8.9M Unique Patients Treated...6.5M Outpatient Visits...86M Outpatient Surgeries...29,000 Inpatient Admissions 695, Veteran Integrated Service Networks 152 Hospitals 985 Outpatient Clinics 819 Community-Based 150 Hospital-Based 10 Mobile 6 Independent 300 Vet Centers 70 Mobile Vet Centers 102 Domiciliary Residential Rehabilitation Programs 135 Community Living Centers 3

4 1 st Generation analytics Performance measures Multidimensional cubes Dashboards Clinical alerts and reminders Limited access and filtering Alert fatigue and information overload

5 Unintended Consequences of Well-Intended Measures (1) Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy J Gen Intern Med 2011 doi: /s

6 2 nd Generation Analytics Data Type TOTAL ENTRIES Entries per Day Orders 3.5 billion 1.2 million Images 3.7 billion 2.8 million Text (H&P, Notes) 2.3 billion 1.0 million Medications 1.9 billion 0.6 million Vital Signs 2.5 billion 0.9 million Corporate Data Warehouse: 1.5 Petabytes, 80B rows of data; 20,000 users 6

7 >800 data-related products in active use Enterprise Wide Search Portal for Performance Data 7

8 Productivity, Efficiency and Staffing 8

9 Veteran Experience and Satisfaction National AVG 90 th P-tile 10 th P-tile Access Composite: Primary Care Thru May Overall Rating of Primary Care Provider Thru May Overall Rating of Hospital Stay Thru Apr

10 3 rd Generation Improved Display Desktop BI High-level modeling Multiple data sources Mobile

11 SAIL Summary Reports 11

12 Predicted and Observed Likelihood of Death/Admission Updated and Distributed Weekly C=0.79 C=0.85 C=0.81 Veterans in highest %ile of risk have 62% probability of admission, 30% probability of death, and 72% probability of either event. 12

13 Risk Data Updated Weekly

14 Use of High Level Analytic Data for Population Management and Resource Planning 1-yr likelihood of admission or death 1-yr likelihood of admission 1.86% % 5.94% % 7.01% % 7.98% % 9.22% % 2.37% % 9.04% % 10.02% % 10.97% % 12.19% % 12

15 Current Predictive Modeling Activities Kidney Disease Risk of acute kidney injury at admission, before/after cardiac catheterization Risk of dialysis (6 mos and 5 yr) among pts with chronic kidney disease Cardiovascular Risk of stroke with atrial fibrillation Risk of coronary event or stroke within 5 years Infectious Disease Risk of drug-resistant Klebsiella infection Risk of infx. w/ mult. drug resist. organism (Acinetobacter, E coli, Pseudomonas) Risk of C. diff infection Mental health: Risk of suicide, ER visit, drug OD or mental health admission Response of prostate cancer to chemotherapy 15

16 Geospatial Mapping Linking Enrollment, Scheduling Patients on Wait-list within 40 Miles of VAMC/CBOC Not within 40 Miles 3,

17 Hospitalizations for Ambulatory Care Sensitive Conditions (ACSC) Predicted ACSC hospitalizations if PACT=0 Predicted hospitalizations for ACSCs Observed hospitalizations for ACSCs Hospitalizations avoided -4.2% Hebert et al. Health Aff 2014;33:

18 Return on investment Net discounted cash flow through 2019 = $1.175 billion

19 PACT Implementation Progress Index (Pi 2 ) Domains Source of data # of items Accessible, continuous and coordinated care Team-based care Access Waiting for care, after-hours care, non-face-to-face care CAHPS-PCMH 11 Continuity of care CDW 3 Coordination of care 8 Delegation, staffing, team functioning, working to top of competency Provider survey 18 Comprehensiveness 3 Self-management support 2 Patient-centered Patient centered care and CAHPS-PCMH care 6 communication Shared decision making 2 Total 53 Nelson et al. JAMA Intern Med

20 Pi 2 Scores, Patient Satisfaction, Provider Burnout PI 2 scores* No. of clinics Patient satisfaction (0 worst 10 best) Provider rating, CAHPS-PMCH Overall health care rating, SHEP Provider Burnout MBI** ED visits/ 1000 pts 5 to to to to to *Pi 2 score = number of domains in top and bottom quartiles for the domain scores, range 8 (all domain scores in top quartile) to -8 (all domain scores in bottom quartile ). **MBI emotional exhaustion scale Also significantly fewer visits for Ambulatory Care Sensitive Conditions 20

21 Higher clinical quality at higher implementation sites 48 clinical quality indicators Significantly higher (p<0.05) for 19/48 by high vs. low PI 2 Increase in average outcomes for sites w/ higher PI 2 scores as compared w/ those w/ lower scores (p <0.001) 21

22 Current Initiatives Capture new sources of data New provider generated data (EHR) Patient (and care giver) generated data Mobile Apps, Web, Kiosks) Connected devices (e.g. RFID) Geospatial data Staff, Veterans, assets, environmental events, bio events) Genomic data Off-premise data DoD, Pub/Priv Systems) Reduced latency Monthly Weekly Daily Near Real Time Improve Information Delivery Desktop BI (reduce reliance on static dashboards) Push, Just-in-Time, Exception reporting Enhance analytical capabilities Predictive analytics Risk analytics Prescriptive analytics Optimization Data mining Natural language processing Machine learning Clinical reasoning & point-of-care decision support

23 Mobile Data 23

24 I vs. T I T 24

25 How much do the following limit your ability to provide optimal, patient-centered care? (n=6,467) Proportion of respondents who say limits a great deal Clinical reminder volume 43.30% Recruiting and retaining providers 41.80% Lack of control over my schedule 38.80% CPRS view alerts volume (e.g., provider-to-provider communication) 34.40% Difficulty accessing specialty care 34.00% Recruiting and retaining non-provider clinicians 33.60% Poor communication with specialists within VA 32.60% Inadequate time allotted to provide counseling or education 32.40% Time & effort to input notes 32.30% Inadequate time allotted to provide follow-up care 31.90% Poor communication with specialists outside the VA 31.30% Recruiting and retaining non-clinicians 31.10% Adoption of EHR 90 min of additional work per 8 hrs of patient care

26 Universal Measurement Formula SQR of Sample Size Information Richness Confidence = (Signal Noise) x N How good is my decision? Messiness of Life Sackett, CMAJ 2001;165:

27 Randomness Makes Rank Mostly Meaningless

28 Hospital Rankings: Popular Entertainment and Growth Industry U.S. News Top 5 Why Not the Best Top 5 Hopkins Mayo Clinic UCLA Cleveland Clinic Mass General Hackettstown NJ NYC Community West Anaheim Flowers La Palma Top 100 Hospitals None of the Above!!

29 Cognitive Biases when using Numbers We see patterns in randomness (clustering illusion) We see what we want to see (confirmation bias) We see what we re used to seeing (availability bias) We extrapolate more than we should We underestimate the likelihood of randomness Gilovich, How We Know What Isn t So VHA Office of Informatics and Analytics

30 Measurement and Health IT potential pitfalls Key data are often missing Key data are often in unstructured text Structured data are often not coded consistently Capturing data in structured format is time consuming Structured data are often not interoperable between IT systems Data may not be consistent over time Bayley et al, Med Care 2013; 51:S80 30

31 Unintended Consequences of Well-Intended Measures (2) 31

32 Unintended Consequences of Well-Intended Measures (3) 32

33 Hospital Complications & NLP 33

34 What is still needed from Big Data High level search capability Intelligent aggregation of data Customized presentation of information based upon context (User/Patient) and importance Comparative effectiveness assessments Personalized Predictive analytics Data Mining to detect unrecognized relationships Platform to evaluate health care interventions Tools to effectively manage population health 34

35

36 36

37 Standard Orders 37

38 Developing Context-Sensitive Orders 30 million records 20 years of data mined to identify relationships between Problems Laboratory results Prescriptions Procedures Orders Location Provider Patient 15,000 conditions and 10,000 orders

39 Common Test and Drug Orders Epigastric pain CBC with diff Chemistry panel Helicobacter pylori Lipase Amylase Upper GI endoscopy Abdominal ultrasound CT abdomen w/o contrast Omeprazole Ranitidine Amphogel Rabeprazole Promethazine Sucralfate Metronidazole Metoclopramide Docusate

40 40

41 Lessons Importance of data consolidation Timeliness vs data integrity Stronger data governance Access Training Links to other data sources Integration of data collection and reporting into routine work flow