1 pharmacoepidemiology and drug safety 2010; 19: Published online 11 October 2010 in Wiley Online Library (wileyonlinelibrary.com).2027 BRIEF REPORT Secondary use of electronic health record data: spontaneous triggered adverse drug event reporting Jeffrey A. Linder 1,2 *,JenniferS.Haas 1,2,AarthiIyer 1, Michael A. Labuzetta 1, Michael Ibara 3, Michael Celeste 3, George Getty 4 and David W. Bates 1,2 1 Division of General Medicine and Primary Care, Brigham and Women s Hospital, Boston, MA, USA 2 Harvard Medical School, Boston, MA, USA 3 Worldwide Safety and Regulatory Operations, Pfizer Inc., New York, NY, USA 4 Claricode, Waltham, MA, USA SUMMARY Purpose Physicians in the United States report fewer than 1% of adverse drug events (ADEs) to the Food and Drug Administration (FDA), but frequently document ADEs within electronic health records (EHRs). We developed and implemented a generalizable, scalable EHR-based system to automatically send electronic ADE reports to the FDA in real-time. Methods Proof-of-concept study involving 26 clinicians given access to EHR-based ADE reporting functionality from December 2008 to May Measurements Number and content of ADE reports; severity of adverse reactions (clinician and computer algorithm defined); clinician survey. Results During the study period, 26 clinicians submitted 217 reports to the FDA. The clinicians defined 23% of the ADEs as serious and a computer algorithm defined 4% of the ADEs as serious. The most common drug classes were cardiovascular drugs (40%), central nervous system drugs (19%), analgesics (13%), and endocrine drugs (7%). The reports contained information, pre-filled from the EHR, about comorbid conditions (207 reports [95%] listed 1899 comorbid conditions), concurrent medications (193 reports [89%] listed 1687 concurrent medications), weight (209 reports [96%]), and laboratory data (215 reports [99%]). It took clinicians a mean of 53 seconds to complete and send the form. In the clinician survey, 21 of 23 respondents (91%) said they had submitted zero ADE reports to the FDA in the prior 12 months. Conclusions EHR-based, triggered ADE reporting is efficient and acceptable to clinicians, provides detailed clinical information, and has the potential to greatly increase the number and quality of spontaneous reports submitted to the FDA. Copyright # 2010 John Wiley & Sons, Ltd. key words adverse drug reaction reporting systems; product surveillance; post-marketing; medical record systems; computerized; ambulatory care Received 9 December 2009; Revised 28 June 2010; Accepted 5 July 2010 INTRODUCTION About a quarter of outpatients experience adverse drug events (ADEs) and 13% of these are serious. 1 ADEs also represent common reasons for emergency department visits for adults and children. 2,3 However, because of barriers to reporting, physicians report as few as 0.3% of ADEs to the Food and Drug Administration (FDA). 4,5 A low reporting rate has contributed to the widespread agreement that it takes * Correspondence to: J. A. Linder, Division of General Medicine and Primary Care, Brigham and Women s Hospital, 1620 Tremont Street, BC-3-2X Boston, MA 02120, USA. too long to identify problem drugs and quantify risks of older drugs. 6 8 Consistent with recommendations from the Institute of Medicine, 9 proactive, automated reporting imbedded within electronic health records (EHRs), which are now used by about 35% of ambulatory physicians, 10 may improve public health by facilitating the submission of ADE reports. We developed the ADE spontaneous triggered electronic reporting system (ASTER), to submit ADE reports directly from EHRs using data collected as part of routine care to the FDA in a manner that was scalable and independent of a specific EHR. We conducted an initial pilot implementation of ASTER to assess the Copyright # 2010 John Wiley & Sons, Ltd.
2 1212 j. a. linder ET AL. feasibility of sending automated, real-time spontaneous, triggered reports from an EHR to the FDA and evaluated clinicians attitudes toward the system. METHODS We designed ASTER for use with any EHR that has a coded ADE documentation event or trigger. ASTER was remotely hosted by CRIX TM International, which could serve as a central aggregator of reports from multiple sources. For this proof-of-concept study, we used the Longitudinal Medical Record (LMR), an ambulatory EHR developed by Partners Healthcare, an integrated healthcare delivery system in Eastern Massachusetts. The LMR is an internally developed, web-based, fully functional, Certification Commission for Health Information Technology-certified EHR that uses a services-oriented architecture and includes notes from primary care and subspecialty practices; coded problem lists; a health maintenance grid for documenting preventive counseling and health habits; medication prescribing; and medication allergies. 11 ASTER used a flexible technical standard for data collection known as Retrieve Form for Data Capture (RFD) from the Clinical Data Interchange Standards Consortium (CDISC) and Integrating the Healthcare Enterprise (IHE). 12 RFD allowed remote hosting at CRIX, local aggregation and presentation of data within the EHR, and direct electronic submission to the FDA using the Individual Case Safety Report (ICSR) standard. ASTER was automatically triggered in the LMR when clinicians discontinued a medication due to an adverse reaction. When triggered, the LMR stored a unique identifier for the patient, clinician, and suspect medication. This identifier the key for which was retained within the LMR was passed to the ASTER application. ASTER then retrieved medication, demographic, vital sign, medical problem, and laboratory information from the LMR to populate an ADE reporting form. The ADE reporting form was available for the clinician to review within the LMR. Before submitting the form, the clinician had to provide two additional pieces of data: (1) the outcome of the ADE and (2) the earliest date of occurrence of the ADE. The clinician-defined outcomes were derived from the FDA MedWatch form. 13 We considered two types of seriousness: clinician coded and computer algorithm defined. For clinician coding, we considered the outcomes of death, life threatening, hospitalization, disability or incapacity, congenital anomaly or birth defect, and other serious outcomes as serious ADEs. For computer algorithm defined seriousness, events were automatically coded by CRIX using the Medical Dictionary for Regulatory Activities (MedDRA). 16 The event was then compared to a predefined list of designated always serious medical events created by the FDA. The study ran from 9 December 2008 to 22 May We selected, invited, and gave access to ASTER to 26 clinicians who frequently discontinued medications due to adverse reactions. As a comparison group, we used Partners Healthcare clinicians who wrote a prescription or had had at least one patient visit in the prior year. We compared these clinicians with the ASTER participants based on age, length of LMR use, and, in the year prior to the pilot, medication prescribing, and discontinuations due to adverse reactions. We classified drugs using First Data Bank s Enhanced Therapeutic Classes. 14 The main outcome of interest was the number of ADE reports received by the FDA. Additional measures of interest included the content, detail, and quality of the ADE reports and the speed, usability, and value of ASTER as reported by clinicians in a postpilot survey. We discussed the original concepts for ASTER with the FDA, which had input on the initial design. The design team met with the FDA a month prior to going live to review the project and processes. The Institutional Review Board and Health Information Systems Group of Partners Healthcare required signed data use agreements with CRIX and the FDA and approved the study protocol. RESULTS The 26 selected pilot clinicians, compared to 2606 nonparticipating clinicians, were older (mean age in years, 52 versus 45; p < 0.001); had used the EHR longer (mean years, 6.2 versus 4.8; p < 0.01); and, in the year prior to the pilot, prescribed more medications (mean, 4404 versus 578; p < 0.001), discontinued more medications (mean, 1422 versus 154; p < 0.001) and, by design, discontinued more medications due to adverse reactions (mean, 35 versus 2; p < 0.001). During the 5-month study period, the 26 pilot clinicians discontinued 319 medications due to ADEs. Reports for some of these ADEs were not submitted through ASTER because they were submitted before the system was saving data (13); errors in transmitting the data early in the study period (21); and either the clinician closed the pop-up form before submitting data or the form timed out because of delays (68). The FDA received 217 reports. The mean time from triggering to submitting the form was 53 (standard deviation [SD], 28) seconds.
3 The mean time between the ADE and report submission was 10 days (SD, 10). The most common suspect drug class was cardiovascular therapy agents (Table 1). The most common specific suspect drugs were lisinopril (n ¼ 34), simvastatin (n ¼ 10), nortriptyline (n ¼ 7), tramadol (n ¼ 6), and alendronate (n ¼ 5). Clinicians coded 168 (77%) of outcomes as non-serious and the majority of the remainder as other. The automatic algorithm coded 4% of reactions as serious. Reports contained information about demographics, comorbid conditions, concurrent medications, vital signs, and labs (Table 2). The mean number of comorbid conditions listed, on reports that had at least one comorbid condition, was 9 (SD, 6). The mean number of concurrent medications listed, on reports that had at least one concurrent medication, was 8 (SD, 5). The most commonly listed concurrent medications were lisinopril (n ¼ 26), aspirin (n ¼ 26), simvastatin ehr-based ade reporting 1213 Table 2. Example content of electronic adverse drug event reports (N ¼ 217) Characteristic Value Patient age, mean (SD) 58 (15) Women, N (%) 159 (73) Reports with any comorbid condition, N (%) 207 (95) Total number of comorbid conditions, N 1899 Report with any concurrent medication, N (%) 193 (89) Total number of concurrent medications, N 1687 Weight, N (%) 209 (96) Height, N (%) 177 (82) Reports with any laboratory data, N (%) 215 (99) Specific laboratory results, N (%) White blood cell count 211 (98) Creatinine 205 (95) Alkaline phosphatase 205 (95) Total bilirubin 185 (86) ALT 201 (94) Most common comorbid conditions were musculoskeletal problems (n ¼ 297), surgical procedures and tests (n ¼ 168), cardiovascular disease (n ¼ 159), lung disease (n ¼ 152), and cancer (n ¼ 111). Table 1. Suspect drug characteristics and outcomes (N ¼ 217) Characteristic N (%) Drug class Cardiovascular therapy agents 86 (40) Central nervous system agents 41 (19) Analgesic, anti-inflammatory or antipyretic 28 (13) Endocrine 14 (7) Anti-infective agents 13 (6) Chemical dependency, agents to treat 9 (4) Other classes 26 (12) Clinician-assigned outcomes No serious outcome 168 (77) Other serious outcome 36 (17) Inpatient hospitalization y 7 (3) Disability or incapacity z 4 (2) Life-threatening x 2 (1) Congenital anomaly or birth defect 0 (0) Death 0 (0) Automatically coded outcomes Serious reactions 8 (4) Non-serious reactions yy 158 (72) Not coded zz 53 (24) Most common other outcomes were musculoskeletal and connective tissue disorders (n ¼ 7), respiratory, thoracic, and mediastinal disorders (n ¼ 6), and psychiatric disorders (n ¼ 5). y Inpatient hospitalization outcome reactions were flu-like syndrome, seizure, heart block, rectal bleeding, myalgias, mental status change, and druginduced lupus (n ¼ 1 for all outcomes). z Disabling/incapacitating outcome reactions were flu-like syndrome, breathing difficulties, lung nodules, and rash (n ¼ 1); hepatotoxicity (n ¼ 1); insomnia (n ¼ 1); and chronic edema (n ¼ 1). x Life-threatening outcomes were bradycardia (n ¼ 1) and gastrointestinal intolerance (n ¼ 1). Serious reactions were angioedema (n ¼ 4), hyperkalemia (n ¼ 2), and myositis (n ¼ 2). yy Most common non-serious reactions were respiratory, thoracic, and mediastinal disorders (n ¼ 27); gastrointestinal disorders (n ¼ 26); psychiatric disorders (n ¼ 21); and nervous system disorders (n ¼ 20). zz Most common non-coded reactions were gastrointestinal disorders (n ¼ 30), nervous system disorders (n ¼ 5), and hepatobiliary disorders (n ¼ 4). (n ¼ 25), atorvastatin (n ¼ 15), and atenolol (n ¼ 12). Over 95% of reports had weight and laboratory data. After the pilot period, 23 of the 26 participants responded to the survey. Of the 23 respondents, 21 (91%) reported they had submitted no MedWatch forms in the prior year and two (9%) reported submitting three or more. Overall, 18 (78%) said they would like to view aggregate national data from the FDA about similar events. Importantly, 21 (91%) felt that ASTER would improve the care of their patients and 22 (96%) said that ASTER would improve their ability to accurately report drug risks. Among the freetext comments provided, 17 respondents (74%) stated that ASTER was either fast or easy to use and eight (35%) mentioned loading failures or ASTER taking time away from clinical care. DISCUSSION In this proof-of-concept study, clinicians who generally reported submitting no ADE reports in the prior year submitted 217 reports to the FDA in 5 months using an EHR-based, triggered, spontaneous reporting system. Despite initial technical challenges, the system met many of the requirements for a successful ADE reporting system: ease-of-use, acceptability to clinicians, and generation of reports containing a wide variety of data types. 15 Potential advantages of EHRbased, spontaneous triggered ADE reporting include secondary use of data collected as part of clinical care; integrating ADE reporting into clinicians workflow; the promptness and speed with which ADEs are reported; the ability to know the denominator of drug
4 1214 j. a. linder ET AL. prescriptions from which ADEs emerge; and increasing reports from clinicians rather than manufacturers, who presently submit over 80% of reports. 16 Potential disadvantages of EHR-based, spontaneous triggered ADE reporting include many of the biases inherent to spontaneous reports reporting bias, availability bias, channeling, and the Weber effect (the tendency of newly approved drugs to generate more ADE reports); 17 the potential volume of reports; the complexity of the data received; data incompatibility; 18 and the frequent submission of well-known, non-serious ADEs. 19 Finally, any reporting system must protect the privacy of patients and clinicians, who may be concerned about medical liability. 20 ASTER complements many other ongoing and emerging pharmacovigilance efforts such as the FDA Sentinel Initiative or the European Adverse Drug Reaction Project; 21,22 prospective and retrospective observational studies; 23,24 population-based outcome studies; 25 patient monitoring studies; 26 automated ADE detection techniques; 27 and independent reporting systems. 28,29 Compared to other methods of ADE identification, an EHR-based, spontaneous, triggered reporting system has several advantages. Clinicians, as opposed to researchers, identify and define potential ADEs ASTER is not dependent on pre-identified safety questions, has the potential to identify unexpected associations, 34,35 and takes advantage of the existing spontaneous reporting system. 19 ASTER could also be used to report vaccine-related or device-related adverse events to the FDA. 36 This study has limitations. First, by design, the clinicians we selected and who agreed to participate were different from the broader population of clinicians at Partners Healthcare. In addition, given that it is academically affiliated, our results may not generalize to broader community practice. Third, technical problems, now resolved, in conducting the pilot limited clinician satisfaction with ASTER. Finally, as a proof-of-concept study, the number of clinicians and reports was small. To address these limitations, we plan to continually monitor and quickly correct technical problems, expand participation to a larger, more representative set of clinicians, and expand participation beyond a single health system and EHR. During the project, we kept FDA staff apprised of our work and we have presented pilot results to the FDA. As we expand the number of clinicians with access to ASTER, we hope to continue working with the FDA to ensure that EHRbased, spontaneous triggered ADE reports provide high quality information. To that end, we may refine the ADE identification method, ensure the appropriate KEY POINTS EHR-based, triggered ADE reporting is efficient and acceptable to clinicians, and provides detailed clinical information. EHR-based, triggered ADE reporting has the potential to greatly increase the number and quality of submitted spontaneous reports. data elements are included, and, if the volume of reports becomes too large, limit submissions to only serious events. The true public health value of a system such as ASTER, will emerge when thousands of clinicians are submitting reports. We designed and implemented ASTER in a scalable way: a central aggregator allows collection of reports from multiple EHRs or electronic systems and can pre-process reports before sending them to the FDA. Such a system could also be bidirectional the FDA could request additional or follow-up information about specific ADEs and could serve as part of a national distributed health data network with functions other than ADE reporting. 37 Like any pharmacovigilance system, EHRbased, spontaneous triggered reporting should help identify new ADEs and accurately define the risks associated with drugs. 8,26 CONFLICT OF INTEREST Dr Ibara and Mr Celeste are employees of Pfizer and were involved in the planning, implementation, data collection, data analysis, manuscript drafting, and the decision to submit the manuscript for publication. Dr Linder had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. ACKNOWLEDGEMENTS This study was supported by a grant from Pfizer. REFERENCES 1. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003; 348(16): Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB, Schroeder TJ, Annest JL. National surveillance of emergency department visits for outpatient adverse drug events. JAMA 2006; 296(15): Cohen AL, Budnitz DS, Weidenbach KN, et al. National surveillance of emergency department visits for outpatient adverse drug events in children and adolescents. J Pediatr 2008; 152(3):
5 ehr-based ade reporting Health, Education, and Human Services Division. Adverse Drug Events: The Magnitude of the Health Risk Is Uncertain Because of Limited Incidence Data (Report GAO/HEHS-00-21), US General Accounting Office, Washington, DC, January Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of underreporting of adverse drug reactions: a systematic review. Drug Saf 2009; 32(1): Bates DW. Drugs and adverse drug reactions: how worried should we be? JAMA 1998; 279(15): Strom BL. How the US drug safety system should be changed. JAMA 2006; 295(17): Avorn J, Schneewiess S. Managing drug-risk information what to do with all those new numbers. N Engl J Med 2009; 361(7): Committee on the Assessment of the US Drug Safety System. The Future of Drug Safety: Promoting and Protecting the Health of the Public, National Academy of Sciences, Washington, DC, Hing E, Hsiao CJ. Electronic medical record use by officebased physicians and their practices: United States, National health statistics reports; no 23, National Center for Health Statistics, Hyattsville, MD, Certification Commission for Healthcare Information Technology. Partners HealthCare System Longitudinal Medical Record (LMR) Available at: [8 December 2009]. 12. Integrating the Healthcare Enterprise. Technical Framework Supplement Retrieve Form for Data Capture. Available at: Technical_Framework/upload/IHE_ITI_TF_Supplement_RFD_TI_ pdf [December 2009]. 13. MedWatch: The FDA Safety Information and Adverse Event Reporting Program. Available at: htm [December 2009]. 14. First Data Bank. Enhanced Therapeutic Classification System. Available at: [December 2009]. 15. Wu JH, Shen WS, Lin LM, Greenes RA, Bates DW. Testing the technology acceptance model for evaluating healthcare professionals intention to use an adverse event reporting system. Int J Qual Health Care 2008; 20(2): Moore TJ, Cohen MR, Furberg CD. Serious adverse drug events reported to the Food and Drug Administration, Arch Intern Med 2007; 167(16): Stephenson WP, Hauben M. Data mining for signals in spontaneous reporting databases: proceed with caution. Pharmacoepidemiol Drug Saf 2007; 16(4): Glass LM. Data incompatibility of adverse event reports electronically submitted to the AERS database. Pharmacoepidemiol Drug Saf 2008; 17(2): Wysowski DK, Swartz L. Adverse drug event surveillance and drug withdrawals in the United States, : the importance of reporting suspected reactions. Arch Intern Med 2005; 165(12): Lee LM, Gostin LO. Ethical collection, storage, and use of public health data: a proposal for a national privacy protection. JAMA 2009; 302(1): Platt R, Wilson M, Chan KA, Benner JS, Marchibroda J, McClellan M. The new Sentinel Network improving the evidence of medicalproduct safety. N Engl J Med 2009; 361(7): Trifiro G, Pariente A, Coloma PM, et al. Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf 2009; 18: Psaty BM, Vandenbroucke JP. Opportunities for enhancing the FDA guidance on pharmacovigilance. JAMA 2008; 300(8): Bennett CL, Nebeker JR, Yarnold PR, et al. Evaluation of serious adverse drug reactions: a proactive pharmacovigilance program (RADAR) vs safety activities conducted by the Food and Drug Administration and pharmaceutical manufacturers. Arch Intern Med 2007; 167(10): Booth CM, Mackillop WJ. Translating new medical therapies into societal benefit: the role of population-based outcome studies. JAMA 2008; 300(18): Harmark L, van Grootheest AC. Pharmacovigilance: methods, recent developments and future perspectives. Eur J Clin Pharmacol 2008; 64(8): Wang X, Hripcsak G, Markatou M, Friedman C. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study. J Am Med Inform Assoc 2009; 16(3): Zafar A, Hickner J, Pace W, Tierney W. An adverse drug event and medication error reporting system for ambulatory care (MEADERS). AMIA Annu Symp Proc. 2008: Miller MR, Clark JS, Lehmann CU. Computer based medication error reporting: insights and implications. Qual Saf Health Care 2006; 15(3): Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G. Detecting adverse events using information technology. J Am Med Inform Assoc 2003; 10(2): Honigman B, Light P, Pulling RM, Bates DW. A computerized method for identifying incidents associated with adverse drug events in outpatients. Int J Med Inform 2001; 61(1): Brown JS, Kulldorff M, Petronis KR, et al. Early adverse drug event signal detection within population-based health networks using sequential methods: key methodologic considerations. Pharmacoepidemiol Drug Saf 2009; 18(3): Brown JS, Kulldorff M, Chan KA, et al. Early detection of adverse drug events within population-based health networks: application of sequential testing methods. Pharmacoepidemiol Drug Saf 2007; 16(12): Platt R, Madre L, Reynolds R, Tilson H. Active drug safety surveillance: a tool to improve public health. Pharmacoepidemiol Drug Saf 2008; 17(12): Poluzzi E, Raschi E, Moretti U, De Ponti F. Drug-induced torsades de pointes: data mining of the public version of the FDA adverse event reporting system (AERS). Pharmacoepidemiol Drug Saf 2009; 18(6): Hinrichsen VL, Kruskal B, O Brien MA, Lieu TA, Platt R. Using electronic medical records to enhance detection and reporting of vaccine adverse events. J Am Med Inform Assoc 2007; 14(6): Maro JC, Platt R, Holmes JH, et al. Design of a national distributed health data network. Ann Intern Med 2009; 151(5):