Using Electronic Medical Records Data for Health Services Research Case Study: Development and Use of Ambulatory Adverse Event Trigger Tools Hillary Mull VA Boston Healthcare System Boston University School of Public Health June 27, 2010 AcademyHealth ARM 2010 1
Outline Description of trigger study Domains of data merge challenges Data conversions Data availability Data de-identification Examples of merged data quality problems Conclusions AcademyHealth ARM 2010 2
Objectives Developed and tested a set of triggers to identify outpatient adverse events (AEs) Triggers are algorithms that use electronic patient data to identify patterns consistent with a possible AE E.g. the combination of a lab value threshold and an active prescription Requires high quality electronic patient data to 1. Develop the surveillance rule 2. Evaluate whether an AE occurred AcademyHealth ARM 2010 3
Develop Triggers Clinicians review literature to develop potential outpatient AE triggers Review these triggers with focus groups Hold Delphi process to refine trigger logic, develop new triggers Establish list of outpatient AE triggers to test List of triggers Merge Data Obtain de-identified data extracts from each site Merge variables from each site into TIDS database Iterative process of checking data quality and obtaining new data extracts Site 1 Site 2 TIDS database: Inpatient, outpatient, vitals, demographics, lab, pharmacy, notes Site 3 Program trigger algorithms Program triggers and perform chart classification Program trigger algorithms to run on data in TIDS database and flag cases Iterative process of running trigger and checking data quality Build mock EMR (TIDS viewer) to interface with TIDS database Perform chart classification TIDS Viewer Trigger-flagged cases Chart Review Analyze trigger performance Calculate positive predictive value and 95% confidence intervals for each trigger AcademyHealth ARM 2010 4
Data Merge Overview We developed a list of data elements for each site s data programmer EMRs were not designed for HSR; site-based programmers were not familiar with data extractions for HSR Discovered numerous inconsistencies with data Six revisions to guidelines over 8 months A problem at one site often required another data pull at a different site to ensure consistency AcademyHealth ARM 2010 5
Data Challenges: Conversions Same information, different coding by institutions Patient sex: M/F vs 1/2/3 Units of measure: metric vs US vs missing ICD-9-CM codes stored with or without periods Pharmacy dosages: 1 vial vs 50 ml Lab titles and results inconsistent across settings Lack of documentation on coding practices Numeric results within text data AcademyHealth ARM 2010 6
Data Challenges: Availability Missing pharmacy dosage/fill data Missing National Drug Codes (NDCs) in pharmacy data Free text vs. standardized daily dosage information ICD-9-CM procedure codes were unavailable for some procedures Lack of units in lab data AcademyHealth ARM 2010 7
Example of Pharmacy Sig TAKE ONE-HALF TABLET BY MOUTH EVERY DAY FOR 2 WEEKS, THEN TAKE ONE-HALF TABLET TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE ONE TABLET TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE TWO TABLETS TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE THREE TABLETS TWO (2) TIMES A DAY FOR 2 WEEKS, THEN TAKE FOUR TABLETS TWO (2) TIMES A DAY INCREASE DOSE GRADUALLY. WHEN GOING FROM 25 TO 50 MG START WITH INCREASING THE AM DOSE FOR 2WEEKS, THEN THE AM AND PM DOSE. DO THIS WHEN INCREASING FROM 50 TO 75 AND 75 TO 100. IF QUESTIONS PLEASE CALL. AcademyHealth ARM 2010 8
Data Challenges: De-identification Obtaining EMR data from each site required development and site-based validation of a deidentification algorithm Produced gaps in the data Fuzzy pattern and word matching removed some key clinical terms from clinical notes Removal of dates resulted in loss of information about clinical order De-identification made notes difficult to read AcademyHealth ARM 2010 9
Example of De-Identification from Patient Notes AcademyHealth ARM 2010 10
Review of Merged Data #1 AcademyHealth ARM 2010 11
Review of Merged Data #2 300,000 250,000 200,000 150,000 100,000 50,000 0 1st Data Pull 39 Note Types 2nd Data Pull 145 Note Types AcademyHealth ARM 2010 12
Conclusions What patterns in the data did we NOT look into? How can we be sure the electronic data is reliable? How can we better predict the time/complexity of merging electronic clinical data from multiple sites? AcademyHealth ARM 2010 13