Impact of Clinical Decision Support within Computerized Physician Order Entry A Systematic Review Jennifer Gillert Candidate for Honours B.Sc. in Health Studies with Health Informatics Option Faculty of Applied Health Science University of Waterloo Written in partial fulfillment for HLTH 452 taught by Professor Jose Arocha
1. INTRODUCTION Medication errors and adverse drug events (ADEs) are considered to be common, costly, clinically important and most importantly, preventable (1). Approximately 28% of ADEs are associated with medication errors, of which 56% occur during medication prescription (2). The potential impact of computer-assisted decision-making for clinicians during the ordering process has been widely discussed as a method to reduce errors and ADEs (1,3-5). Clinical decision support (CDS) within CPOE offers physician unsolicited advice and can include a range of functionality to aid in the electronic prescription of medications. The combination of computerized order entry and decision support has been acknowledged as promoting safe medication practice (7). The studies investigating the impact of CDS embedded within CPOE vary in setting, users, system design, implementation strategy and an unlimited number of other characteristics, and they measure a variety of different forms of outcome. The purpose of this systematic review is to examine the evidence available on the impact of clinical decision support within computerized physician order entry in an electronic medical record. 2. METHODS 2.1 Search and selection criteria Studies were identified through 3 electronic databases: PubMed, Web of Science and CSA Illumina Databases. In addition, the references of the selected articles were reviewed to identify additional studies meeting the inclusion criteria. A total of seventeen studies (N = 17) were included in the review. Inclusion criteria: Acute care, inpatient hospital Electronic medical record including a physician order entry with active, synchronous clinical decision support Studies of orders from staff physicians, fellows, residents or medical students in any clinical area Studies since 1996 Studies were not excluded based on study design. Quality screening of design and methods was not done due to relatively few available studies for the research question. Instead, concerns and limitations of some studies are discussed in the review. 3. RESULTS Information on the following study characteristics were extracted from the seventeen studies: population, duration, study design, relevant types of clinical decision support, outcomes measured, and results. Impact in the clinical setting can encompass many measures. In order to standardize the results, the impact outcomes of the studies were categorized by types of outcomes, presented in Table 2 with the number of studies that included at least one of the outcome types. Each relevant measure of impact outcome of the seventeen studies is summarized in Tables 3.1-3.4. The results from each specific measure from the studies have been categorized by impact type.
Table 2: Summary of impact outcome classifications Outcome Classification Type Impact Outcome Studies with one or more of the outcome type Practitioner Guideline compliance 10 Errors 6 Decision support utilization 1 Patient Adverse drug events 2 Intermediate outcomes 2 Hospital Administrative Length of stay 4 Cost 5 Resource utilization 2 Qualitative Opinions 1 Table 3.1: Impact outcomes: Practitioner Practitioner Guideline Compliance Errors Outcomes measured Specific Outcomes Ref Compliance to corollary orders (Immediate, 24-hr, Increased guideline compliance (19) and hospital-stay) (p < 0.0001) Appropriate prescription Increased rate of appropriate (9) orders (p < 0.001) Recommended Histamine2-blocker order selection Increase in recommended order (23) selection (p < 0.001) Orders exceeding recommended dose Fewer orders exceeding recommended dose (p < 0.001) Appropriate medication frequency for ondansetron Increased rate of appropriate frequency (p < 0.001) Heparin ordering consequent to bed rest order Increased guideline compliance (p < 0.001) Ordering of recommended asthma treatments Increased ordering of (8) recommended treatments (p < 0.001) Use of antiinfective agents Fewer doses of antiinfective (15) agents (p < 0.001) Use of traditional sliding scale orders compared to Fewer traditional sliding scale (11) minimal intervention orders orders (p < 0.0003) Hospital policy violations Fewer rule violations (p < 0.001) (20) Duration of vancomycin treatment Fewer treatment days per (22) physician (p = 0.05) Number of vancomycin orders Fewer treatment days per treatment (p = 0.05) Time for appropriate clinical response to alerts for untreated hypokalemia or hypomagnesemia, and digoxin, magnesium and potassium Unclear effect on compliance (dependent on orders, conditions) (16) Use of recommended early aspirin and betablockers Non-missed-dose medication errors Nonintercepted serious medication errors Unclear effect on compliance (dependent on orders, conditions) Fewer non-missed dose medication errors (p < 0.0001) Fewer nonintercepted serious medication errors (p = 0.01) Number of pharmacist interventions Fewer pharmacist interventions (19) (7) (6) (12)
CDS utilization (p = 0.003) Rate of illegible, incomplete and drug therapy Fewer total medication errors (p errors < 0.001) Frequency of order adjustments prompted by alert Lower order adjustment rate (p < 0.01) Medication prescribing errors Fewer medication prescribing errors (p < 0.001) Number of potential ADEs (duplicate therapy, Fewer potential ADEs (p < inappropriate dose/interval/route, wrong drug/unit, 0.001) allergy, drug interaction) Incorrect dose No difference in rate of incorrect Compliance with Acute Coronary Syndrome (ACS) order set (17) (18) (20) (14) dose (p = 0.4) Unclear impact (condition) (7) Table 3.2: Impact outcomes: Patient ADEs Patient Intermediate Outcomes Outcomes measured Specific Outcomes Adverse drug events (non-intercepted serious medication errors) Fewer ADEs (p = 0.0003) (6) Adverse drug events caused by antiinfective agents Fewer ADEs (p = 0.018) (15) Maximum serum creatinine levels No difference in intermediate (19) outcome (p = 0.28) Changes in renal function Improved intermediate outcome (p < 0.001) (9) Table 3.3: Impact outcomes: Hospital Administrative Outcomes measured Specific Outcomes Length of hospital stay No difference in LOS (p = 0.94) (19) Length of hospital stay No difference in LOS (p = N/A) (8) Length of hospital stay Reduced LOS (p = 0.009) (9) Hospital Administrative Length of stay Costs Resource Utilization Length of hospital stay Reduced LOS (p < 0.001) (15) Hospital charges No difference in costs (p = 0.68) (19) Hospital and pharmacy costs No difference in costs (p = 0.52) (9) Inpatient charges No difference in costs (p = N/A) (8) Cost of antiinfective treatment Reduced cost (p < 0.001) (15) Cost of hospitalization Reduced cost (p < 0.001) Charge savings Reduced cost (p = N/A) (13) Hospital-level vancomycin utilization No significant decrease in drug (22) utilization (p = N/A) Proportion of cancelled lab orders Fewer redundant lab orders (p < (13) 0.001)
Table 3.4: Impact outcomes: Qualitative Outcomes Qualitative Outcomes Opinions (Likert Scale) Outcomes measured Opinion of order sets on efficiency of order entry Opinion of guidelines and decision support interventions as an aid for quality of patient care Opinion of graphical representation of laboratory results on the test ordering Opinion of guidelines and decision support interventions on efficiency of order entry Opinion of guidelines and decision support interventions on the provider care Specific Outcomes Positive opinion Positive opinion Negative opinion No clear opinion No clear opinion (21) 4.3 Limitations Kaplan describes an informatics application as multi-faceted, including social, cultural, organizational, and cognitive aspects (25). The dynamic interaction between users of such a system and the technology itself is not constant between settings. Such a system is socio-technological in nature, dependent on not only the technology itself but also on, for example, the number of available workstations or technical support. Six of the seventeen studies included in this review were conducted at one large academic tertiary-care hospital (6,9,12,13,22,23) and the specific socio-technological characteristics of this particular hospital is thus overrepresented. The method of development, implementation and monitoring of guidelines influence the likelihood of adherence to clinical guidelines (24) and the outcomes from the particular hospital will not necessarily be generalizable to other settings. Comparing the study outcomes from different hospitals, it is difficult to determine impact from the forms of decision support when each setting has a system with varying combinations of clinical decision support. It is difficult to assess, based on the information collected for this review, which specific forms of support are effective and which are not. There may also be an interaction effect of the forms of support when implemented together. The system design can also influence the impact. For example if a decision support system alerts clinicians of non-patient-specific information (26) or fires too frequently, the alerts may be ignored. The generalizability for any of the outcomes may not be applicable to a setting outside that in which it was conducted in, as noted by several of the authors (8,12,19,23). In at least three of the studies simultaneous implementation and before-after outcome comparisons makes it difficult to distinguish outcomes due to the implementation of CDS within CPOE versus the impact due to the CPOE itself (8,12,20). Similarly, with decision support in the form of synchronous best-practice guidelines, it may be difficult to distinguish between the impact due to the decision support intervention and the impact due to the guideline it is based on (19). Randomized controlled trials, although considered the gold standard for evaluation, are not always possible due to the complex nature of the implementation of a CPOE system (6). A handful of the authors (7,12,16) discuss the before-after comparison as a potential threat to validity due to the possible interference of history, where another factor occurring in the study period that may influence the outcome. One example is the simultaneous implementation of a discharge planning tool (7). Additional threats to internal validity with this study design relate to accumulation of experience with a new system when a series of measures are taken over time. On the other hand, the importance of the randomized controlled trials in informatics may not be as great when investigating outcomes with contextual importance (25), like the impact of these interventions in busy clinical workflow. 5. CONCLUSION Studies investigating the impact of synchronous clinical decision support (CDS) within computerized physician order entry (CPOE) measure a variety of outcomes relating to the practitioner, the patient, hospital administration outcomes as well as qualitative perceptions of impact. Based on the available studies, CDS improves practitioner outcomes such as guideline compliance and errors. Limited availability
of studies and mixed study outcomes lead to an inconclusive impact on patient and hospital administrative outcomes. Clinician perceptions of the impact of these applications are also inconclusive. The impact outcomes are measured in a socio-technological setting, making it difficult to generalize one study outcome to another setting. Additional studies are required to further understand the impact of decision support within computerized physician order entry. 7. REFERENCES (1) Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Archives of internal medicine 2003 Jun 23;163(12):1409-1416. (2) Bates D, Cullen D, Laird N. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA 1995;274:29-34. (3) Rochon P, Field T, Bates D, Lee M, Gavendo L, Erramuspe-Mainard J, et al. Computerized Physician Order Entry with Clinical Decision Support in the Long-Term Care Setting: Insights from the Baycrest Centre for Geriatric Care. J Am Geriatr Soc 2005;53:1780-1789. (4) Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, Schulberg HC. A Randomized Trial Using Computerized Decision Support to improve Treatment of Major Depression in Primary Care. J Gen Intern Med 2002;17:493-503. (5) Palen TE, Raebel M, Lyons k, Magid DM. Evaluation of Laboratory Monitoring Alerts Within a Computerized Physician Order Entry System for Medication Orders. The American Journal of Managed Care ;12(7):389-395. (6) Bates D, Teich JM, Lee J, Seger DL, Kuperman GJ, Ma'luf N, et al. The Impact of Computerized Physician Order Entry on Medication Error Prevention. J Am Med Inform Assoc 1999;6(4):313-321. (7) Ozdas A, Speroff T, Waitman LR, Ozbolt J, Butler J, Miller RA. Integrating "best of care" protocols into clinicians' workflow via care provider order entry: impact on quality-of-care indicators for acute myocardial infarction. Journal of the American Medical Informatics Association : JAMIA. 2006 Mar-Apr;13(2):188-96. Epub: 2005 Dec 15. (8) Chislom D, McAlearney A, Veneris S, Fisher D, Holtzlander M, McCOy K. The role of computerized order sets in pediatric inpatient asthma treatment. Pediatric Allergy Journal 2006;17:199-206. (9) Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, et al. Guided Medication Dosing for Inpatients with Renal Insufficiency. JAMA 2001;286(22):2839-2844. (10) Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14(1):29. (11) Achtmeyer CE, Payne TH, Anawalt BD. Computer Order Entry System Decreased Use of Sliding Scale Insulin Regimens. Methods Inf Med 2002;41:277. (12) Bates D, Leape L, Cullen D, Laird N, Petersen LA, Teich JM, et al. Effect of Computerized Physician Order Entry on a Team Intervention on Prevention of Serious Medication Errors. JAMA 1998;280(15):1311-1316. (13) Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al. A Randomized Trial of a Computer-based Intervention to Reduce Utilization of Redundant Laboratory Tests. Am J Med 1999;106(104). (14) Eslami S, Abu-Hanna A, de Keizer N, de Jonge E. Errors Associated with Applying Decision Support by Suggesting Default Doses for Aminoglycosides. Drug Saf 2006;29(9):803.
(15) Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, et al. A computer-assister management program for antiotics and other antiinfective agents. New England Journal of Medicine 1998;338:232-238. (16) Galanter WL, Polikaitis A, Didomenico RJ. A Trial of Automated Safety Alerts for Inpatient Digoxin Use with Computerized Physician Order Entry. J Am Med Inform Assoc 2004;11(4):270-277. (17) Igboechi CA, Ng CL, Yang CS, Buckner AN. Impact of Computerized Prescriber Order Entry on Medication Errors at an Acute Tertiary Care Hospital. Hosp Pharm 2003;38(3):227. (18) Oppenheim MI, Vidal C, Velasco FT, Boyer AG, Cooper MR, Hayes JG, et al. Impact of a Computerized Alert During Physician Order Entry on Medication Dosing in Patients with Renal Impairment. AMIA 2002 Annual Symposium Proceedings; 2002. p. 577. (19) Overhage JM, Tierney WM, Zhou X, McDonald C. A Randomized Trial of "Corollary Orders" to Prevent Errors of Omission. J Am Med Inform Assoc 1997;4:364-375. (20) Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized Physician Order Entry and Medication Errors in a Pediatric Critical Care Unit. Pediatrics 2004;113(1):59. (21) Rosenbloom S, Talbert D, Aronsky D. Clinicians' perceptions of clinical decision support integrated into computerized provider order entry. International Journal of Medical Informatics 2004;73:433-441. (22) Shojania KG, Yokoe D, Platt R, Fisko J, Ma'luf N, Bates D. Reducing Vanomycin Use in Utilizing a Computer Guideline. J Am Med Inform Assoc 1998;5:554-562. (23) Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates D. Effect of Computerized Physician Order Entry on Prescribing Practrices. Arch Intern Med 2000;160:2741-2747. (24) Coiera E. Guide to health informatics. Second edition ed. London: Hodder Aarnold; 2003. (25) Kaplan B. Evaluating informatics applications - some alternative approaches: theory, social interactionism, and call for methodological pluralism. International Journal of Medical Informatics 2001;64:39-56. (26) Galanter WL, Didomenico RJ, Polikaitis A. A trial of automated decision support alerts for contraindicated medications using computerized physician order entry. Journal of the American Medical Informatics Association : JAMIA. 2005 May-Jun;12(3):269-74. Epub: 2005 Jan 31.