Community Pharmacies

Transcription

1 High-Alert Medication Modeling and Error-Reduction Scorecards (HAMMERS ) Workbook For Community Pharmacies Data Entry Errors (Patient) Data Entry Errors (Drug) Prescribing Errors Point of Sale Errors Drug Container Selection Errors A unique tool from the Institute for Safe Medication Practices designed to: Identify and quantify pharmacy dispensing system vulnerabilities Estimate the frequency of potentially harmful errors with high-alert medications Identify the most significant process steps and practices contributing to errors Determine how to reduce risks that have the highest probability of reaching patients Quantify the decrease or increase in risk after implementing new strategies This project was developed by the Institute for Safe Medication Practices in consultation with Outcome Engenuity, LLC. The project was supported by grant number R18HS from the Agency for Healthcare Research and Quality. This content is solely the responsibility of the authors and does not represent the official views of the Agency for Healthcare Research and Quality.

2 Forward Invitation to Use HAMMERS Dear Community Pharmacy Professional: The Institute for Safe Medication Practices (ISMP) is pleased to provide the nation s community pharmacies with the High-Alert Medication Modeling and Error-Reduction Scorecards (HAMMERS ). This tool, which was funded by the Agency for Healthcare Quality and Research (AHRQ) and created in consultation with Outcome Engenuity, will help community pharmacies assess and improve medication safety and protect consumers from the adverse effects of medication errors. Based on published studies, we estimate that approximately four errors reach consumers for every 250 prescriptions dispensed from each pharmacy. HAMMERS will help you assess the risks of HIGH-ALERT MEDICATION errors reaching your patients, identify opportunities for improvement, and suggest specific strategies for reducing these risks. If you use this tool, you will be able to compare the baseline data you collect with your results after you implement various risk reduction strategies. HAMMERS becomes a tool to track system performance over time. HAMMERS offers community pharmacies a simple yet effective way to identify risks, estimate how often these risks result in potentially harmful errors that reach customers, rank which system features and behaviors most often contribute to these risks, and quantify how the frequency of these events will change if interventions are implemented. We applaud your desire to use this tool proactively to reduce errors in your pharmacy. We are confident that you will recognize the substantial value of HAMMERS the very first time you use it. It is truly a unique tool the first of its kind in healthcare. Regular use of HAMMERS will help make community pharmacies safer and more efficient. We welcome all community pharmacies to join us as we work together on this important endeavor. Sincerely, Michael R. Cohen Judy Smetzer Donna Horn President Vice President Director, Patient Safety INSTITUTE FOR SAFE MEDICATION PRACTICES About ISMP The Institute for Safe Medication Practices (ISMP) is the nation's only 501c (3) nonprofit organization devoted entirely to medication error prevention. The organization is known and respected worldwide as the premier resource for impartial, timely, and accurate medication safety information. The Institute's medication error prevention efforts began in 1975 with a groundbreaking and contin uing column in Hospital Pharmacy. Today, a continuously expanding core of knowledge in medication safety fuels the Institute's initiatives, which fall into six key areas: public and professional advocacy, consults and collaboratives, error-reporting and analysis, targeted publications, research and best practice development, and website/social media outlets. As an independent watchdog organization, ISMP receives no advertising revenue and depends entirely on charitable donations, educational grants, newsletter subscriptions, and volunteer efforts to pursue its lifesaving work. For more information, visit ISMP online at: Words or terminology in BLUE SMALL CAPS are defined in the Glossary. I 2012 ISMP

3 Table of Contents I. Description of HAMMERS II. Directions for Using HAMMERS Important Details for Using HAMMERS Directions for Pharmacy Organizations with Multiple Pharmacies General Directions for All Pharmacies Using the Results to Improve III. HAMMERS Scorecards Prescribing Errors Data Entry Errors (Patient) Data Entry Errors (Drug) Drug Container Selection Errors Point of Sale Errors IV. Frequently Asked Questions and Glossary Frequently Asked Questions Glossary Appendix A: High-Alert Medications Dispensed from Community Pharmacies.. Appendix B: Documented Rates of Errors and At-Risk Behaviors Table 1. Human Error Rates Table 2. Incidence of Outpatient Prescribing Errors Table 3. Incidence of Community Pharmacy Dispensing Errors Table 4. Incidence of Missed Opportunities to Capture Errors Table 5. Incidence of At-Risk Behaviors Appendix C: Risk-Reduction Strategies Preventing and Detecting Prescribing Errors Preventing and Detecting Data Entry Errors (Patient) Preventing and Detecting Data Entry Errors (Drug) Preventing and Detecting Drug Container Selection Errors Preventing and Detecting Point of Sale Errors Appendix D: Understanding Errors and At-Risk Behaviors Appendix E: How HAMMERS Works References ISMP

4 I. Description of HAMMERS Purpose The High-Alert Medication Modeling and Error-Reduction Scorecards (HAMMERS ) are designed to help community pharmacies: Identify their unique set of system and behavioral risks associated with dispensing certain HIGH-ALERT MEDICATIONS Estimate how often an error or adverse drug event reaches a patient The Scorecards identify risk factors within the dispensing process and provide estimates of the impact of each risk factor on the overall likelihood that an error will reach the patient. By using the tool, pharmacists can estimate how often prescribing and dispensing errors reach patients and how the frequency will change if certain interventions are implemented. The tool will identify the exact process steps, human behaviors, equipment and technology, and system components that, when combined together, lead to the greatest risk of an adverse outcome. The tool is also unique because an individual pharmacy will be able to quantify the safety benefits of a particular intervention as estimated by the HAMMERS tool. Five Scorecards are available with the HAMMERS tool, each representing different types of errors. 1 Prescribing Errors (wrong drug, dose, or directions) 2 Data Entry Errors (wrong patient) 3 Data Entry Errors (wrong drug, dose, or directions) 4 Drug Container Selection Errors (wrong drug or dose) 5 Point of Sale Errors (medication dispensed to the wrong customer) The Scorecards are described in more detail in Section II: Instructions for Using HAMMERS. With the exception of wrong patient errors, the Scorecards prompt users to choose a HIGH-ALERT MEDICATION or class of medications on which to focus. HIGH-ALERT MEDICATIONS bear a high risk of causing serious injury or death to a patient if they are misused. A complete list of HIGH-ALERT MEDICATIONS dispensed from community pharmacies can be found in Appendix A. The tool s intended focus on HIGH-ALERT MEDICATIONS heightens its ability to impact the health and safety of consumers because it helps prevent potentially serious errors. However, HAMMERS can be used to assess the risks associated with any medication or class/group of medications. The tool requires pharmacy staff to answer predetermined questions about how often certain process steps and practices occur, and how likely staff would be to detect an error given the circumstances described in the questions. Once the questions have been answered, HAMMERS will estimate how often specific types of medication errors with the chosen medication(s) could reach patients. The Scorecards will: Provide accurate estimates regarding the frequency of the medication errors under evaluation Identify the most frequent and significant process steps and practices contributing to these errors Guide the user to interventions that can reduce the risks identified by the tool Provide accurate estimates of the frequency of the medication errors after key interventions have been implemented, thus quantifying the decrease or increase in risk that is estimated to occur See Appendix D and E for additional information about how errors happen and how the HAMMERS tool was created using a SOCIO-TECHNICAL PROBABILISTIC RISK ASSESSMENT PROCESS (ST-PRA). Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

5 II. Directions for Using HAMMERS Important Details for Using HAMMERS HAMMERS has been designed for use in any community pharmacy practice, regardless of the number of staff employed or the number of pharmacies in the organization. However, the tool must be used at the individual pharmacy level and is not intended for centralized use by a single group at the corporate level. The tool requires data from individual pharmacies that can only be provided by pharmacy staff who work in the pharmacies. Corporate-level data invalidates the tool s accuracy. Independent pharmacies can follow the General Directions for All Pharmacies provided below. Directions for Pharmacy Organizations with Multiple Pharmacies Although pharmacy organizations with multiple stores may employ standard operating procedures, management practices, and technology, there are important differences in each individual pharmacy that may influence error rates, including but not limited to: prescription volume, environmental issues such as space and lighting, customer demographics (including ethnicity and native languages spoken), types of medications most frequently dispensed, practice habits of individuals, differences in application of operating procedures, and services available such as drive-through windows. Thus, to maximize the value and accuracy of HAMMERS, large pharmacy organizations should follow the directions below as well as the General Directions for All Pharmacies provided below Whenever possible, all pharmacies within a pharmacy organization should use the tool either independently (following the Directions for All Pharmacies) or as a group. If all pharmacies within the organization cannot participate, the pharmacy organization should select one or more groups of sample pharmacies to use the tool for a targeted HIGH- ALERT MEDICATION. Each sample of pharmacies should be similar in regards to prescription volumes, staffing patterns, and services provided (e.g., drive-through windows, hours of operation), and should include enough pharmacies to provide results that can be considered representative of all similar pharmacies in the pharmacy organization. Small pharmacy organizations should consider including all pharmacies in the samples (grouped by similar demographics, when appropriate). Midsize pharmacy organizations with more than 50 pharmacies but less than 1,000 pharmacies should include at least 25 pharmacies in each sample. Large pharmacy organizations with more than 1,000 pharmacies should include a minimum of 50 pharmacies in each sample. Ask the participating pharmacies to submit the completed Scorecards to a designated management or corporatelevel staff member. Anonymous submissions are highly suggested to promote staff willingness to answer questions in a forthright manner. Gather a team for each sample of pharmacies to review the Scorecards completed by the participating pharmacies to analyze the system vulnerabilities identified and corporate-wide interventions for improvement. General Directions for All Pharmacies 1 Download HAMMERS. Download the HAMMERS software onto your personal computer or server from: The following computer requirements are needed to install and run the software: Windows Installer 4.5 Microsoft.NET Framework 4 Client Profile (x86 and x64) SQL Server Compact 4.0 SQL Server Compact 3.5 SP2 If these components are not currently installed on your computer, you can load them while downloading the software. Once the software has been downloaded, all data entered into the software and results tabulated by the software are the sole Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

6 II. Directions for Using HAMMERS General Directions for All Pharmacies continued property of the HAMMERS user. The Institute for Safe Medication Practices will NOT have access to your data, nor can any data be accidentally entered into the software until it has been downloaded onto your computer or server. Before downloading the tool, you will be asked to provide very basic demographics, but no identifiable information will be requested or collected. There are no limitations on the frequency of downloading the software, which is copyright protected. 2 Establish a team. For each individual pharmacy, establish a team consisting of at least one pharmacist and one PHARMACY ASSOCIATE to answer the questions associated with HAMMERS. Depending on which Scorecard is used, the team can expect to spend about 1-2 hours to complete the assessment. The Scorecard for Prescribing Errors includes the most questions 38 in total. The Scorecard for Point of Sale Errors has the least questions 10 in total. 3 Prepare the team. Read the User s Workbook ( before using HAMMERS. 4 Select an area of focus. Narrow the focus of work by selecting a specific HIGH-ALERT MEDICATION or class of medications to evaluate. A complete list of HIGH-ALERT MEDICATIONS dispensed from community pharmacies can be found in Appendix A. The two Scorecards associated with wrong patient errors (Data Entry Errors-wrong patient, Point of Sale Errors) can include all dispensed medications. 5 Select the type of errors to evaluate and corresponding Scorecard. Select the type of error with the HIGH-ALERT MEDICATION(S) you want to evaluate from among the five Scorecards (Table 1). Table 1. HAMMERS Scorecards Prescribing Errors (wrong drug, dose, or directions) This Scorecard determines how often an error with the prescription gets through the dispensing system and reaches patients. Data Entry Errors (wrong patient) This Scorecard determines how often a prescription that has been entered into the wrong patient s profile gets through the dispensing system and reaches patients. Data Entry Errors (wrong drug, dose, or directions) This Scorecard determines how often mistakes get through the dispensing system and reach patients as a result of mistakes made while entering the drug, dosage form, dose, or directions into the pharmacy computer system. Drug Container Selection Errors (wrong drug or dose) This Scorecard determines how often mistakes get through the dispensing system and reach patients due to selecting the wrong drug product or container while filling the prescription. Point of Sale Errors (medication dispensed to the wrong customer) This Scorecard determines how often a customer leaves the pharmacy counter with the wrong patient s filled prescription at the time of purchase. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

7 II. Directions for Using HAMMERS General Directions for All Pharmacies continued 6 Gain access to the correct Scorecard. The Scorecards can be found in Section III of the workbook (or accessed electronically at: HAMMERS) and within the HAMMERS software on the Home Page. On the HAMMERS Home Page, click on the correct error type to gain access to the corresponding set of questions. 7 Gather dispensing information Gather information regarding how many prescriptions your pharmacy dispenses (weekly, monthly, quarterly, or yearly) for the selected HIGH-ALERT MEDICATION(S). The prescription volume of the medication is necessary as a denominator so HAMMERS can calculate estimates of risk and errors. 8 Discuss and answer each question in the tool. There are four types of questions in the Scorecards (Table 2 on next page). Consider each question in the Scorecard and evaluate the best answer. Always answer the questions based upon the specific HIGH-ALERT MEDICATION and type of error under evaluation. Many will find it helpful to answer the Scorecard questions on paper in the workbook (Section III) before entering data into the HAMMERS software. Helpful Hints Appendix B offers quick reference to research data, which can be used as a realistic starting point and then adjusted up or down according to your pharmacy s unique experiences when determining your pharmacyspecific rates. When estimating initial error rates, be sure to include all errors that are captured and corrected before the patient leaves the pharmacy counter or drive-through window. Many, but not all, questions are phrased negatively How many errors will be missed or processes skipped? for example so be sure to answer the questions as asked. 9 Access your saved Scorecard. Scorecards will be saved automatically as soon as you answer the first question, and then updated every time you answer a subsequent question. You can exit the Scorecard before completing each question and return to complete it at a later date without losing the information previously entered. As appropriate, the Scorecard name will include the medication type. A creation date will also be assigned and updated each time the Scorecard is changed. Saved Scorecards can be accessed via the tab along the right margin of the screen. To access a saved Scorecard, just doubleclick on the Scorecard name. To skip to the Scorecard results or to delete or copy a Scorecard, use the right click function. 10 Questions? If you have any questions while using HAMMERS, click on the Frequently Asked Question (FAQ) button which appears on most screens in the HAMMERS software. The most common Frequently Asked Questions can also be found in Section IV of the workbook. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

8 II. Directions for Using HAMMERS General Directions for All Pharmacies continued Table 2. Question Types in Scorecards Question Type Error frequency Exposure rates Missed CAPTURE OPPORTUNITIES AT-RISK BEHAVIORS Description You will be asked to estimate how often you observe or learn about the errors under evaluation. Support from studies that offer widely generalized data regarding error frequency can be found in Appendix B. However, you will be asked to estimate error rates with specific drugs, for which there are no error frequency data available. Thus, you will be asked to use your own experiences to estimate an error rate after referencing Appendix B and adjusting the rate up or down according to your pharmacy s experiences. You will be asked to estimate how often certain practices occur, or how often certain conditions exist. These data are often captured electronically. For instance, most pharmacies collect data on whether prescriptions are new or refills. When possible, HAMMERS calculates some exposure rates based on previous answers. For example, the tool asks only how often PHARMACY ASSOCIATES enter prescriptions, while the frequency of pharmacists entering prescriptions is calculated by the tool based on how often associates enter orders. You will be asked to estimate how often pharmacy staff or patients fail to capture an error that has been introduced into the dispensing system, given very specific conditions. For example, you may be asked how often a particular error would not be noticed when counseling occurs at the drive-through window, and the patient has not been asked for a second identifier (e.g., birth date) other than name. You will be asked to estimate how often staff engage in well-meaning but dangerous shortcuts, work-arounds, procedural violations, or other unsafe behaviors. AT-RISK BEHAVIORS are often employed and tacitly encouraged as a workaround for various system, technological, equipment, and environmental weaknesses or flaws. Using the Results to Improve Once all questions have been answered in the Scorecard, a report will be viewable on the screen to help guide your improvement activities. You can print each of the Results screens by clicking on the PRINT button in the upper left corner of the screen. When printing reports, the orientation of the page must be set to landscape. Your report will consist of four sections, each provided on a different screen in the software. Scorecard Results Screens Screen 1 (Scorecard Results) This screen provides an informed estimate regarding how many errors for the HIGH-ALERT MEDICATION and/or error type will reach patients from YOUR PHARMACY or PHARMACY ORGANIZATION within the specified timeframe. Screen 2 (Scorecard Metrics) This screen displays Table 1 and Bar Graph 1, which identify the events that most frequently and significantly contribute to the errors under evaluation. The Scorecard questions associated with these events are also provided. These events represent priority risks that should be reduced to improve safety. Screen 3 (Scorecard Metrics) This screen displays Table 2 and Bar Graph 2, which identify risk-reduction strategies and their contribution to the errors under evaluation. The Scorecard questions associated with these strategies are also provided. These strategies represent priorities that should be increased to improve safety. Screen 4 (Scorecard Metrics) This screen displays Table 3 and Bar Graph 3, which identify AT-RISK BEHAVIORS that most frequently and significantly contribute to the errors under evaluation. The Scorecard questions associated with these AT-RISK BEHAVIORS are also provided. These AT-RISK BEHAVIORS must be reduced in scope and frequency to further improve safety, which can only be accomplished after the underlying system-based causes of the behaviors have been addressed. See Appendix D for additional information about AT-RISK BEHAVIORS. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

9 II. Directions for Using HAMMERS Using the Results to Improve continued Additional risk-reduction strategies can be found in subsequent screens and Appendix C. HAMMERS users are strongly encouraged to implement any of the suggested strategies or a different strategy they feel might be effective. Helpful Hint Do not overestimate the scope of implementing a new strategy or its effects. New behaviors, technology, process steps, system changes, or environmental changes take time to be forged, and practitioners cannot expect 100% compliance because there will always be unexpected situations when the intervention will not be applicable or be able to be carried out. For example, if you plan to increase the frequency of patient counseling for a particular HIGH-ALERT MEDICATION, we suggest a 75% rate of compliance, even if the counseling is mandated, to account for conditions when counseling cannot be provided (e.g., the patient is in a hurry, the patient is in pain, the caregiver has a sick child, a patient surrogate picks up the prescription). Before and/or after implementation of risk-reduction strategies, the team that evaluated the initial risk should use the Scorecard to answer the questions again, this time anticipating or realizing the effects of the strategy. Even if you are only implementing a single strategy, be sure to review every question in the Scorecard to determine if the new strategy will have an impact on the answer. Once the questions have been answered, a new report will be generated, which can be used for comparison to the initial report, thus demonstrating the overall impact of the strategy. Please remember to copy and rename the Scorecard (using a right click function) before changing the answers to the questions so that the original risk and error rate data can be used for comparison. Helpful Hint To determine the percent of change between the initial error rate and the new error rate, use this formula: New Error Rate Old Error Rate x 100 Old Error Rate A negative percent of change means the error rate has been reduced by that percent. A positive percent of change means the error rate has been increased by that percent. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

10 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) # Question Answer Special Instructions or User Comments Drug Selection and Prescription Volume You have selected the Scorecard associated with prescribing errors (wrong drug, dose, or directions). Now you must specify which HIGH-ALERT MEDICATION(S) you want to evaluate, and how often your pharmacy fills prescriptions for these medications. While this tool can be used to evaluate prescribing errors with any drug, focusing on HIGH-ALERT MEDICATIONS helps to reduce the risk of errors that can cause harm to patients. 1a Please list the name(s) of the medication(s) or class of medications involved in the prescribing errors you want to evaluate. List Medication(s) See Appendix A for a list of HIGH-ALERT MEDICATIONS dispensed from community pharmacies. 1b Please provide the number of prescriptions (new prescriptions and refills combined) filled within the specified time interval for the medication(s) involved in these errors. # of Prescriptions Time Interval: Week, Month, Quarter, Year Include filled prescriptions that are picked up from the pharmacy or delivered to patients. Add all generics, brands, and strengths together. Error Frequency You will need to estimate how often you receive prescriptions for the specified medications with the wrong drug, dose, or directions. This may include a drug to which the patient is allergic or otherwise contraindicated, unsafe or ineffective directions for use, sub-therapeutic doses, doses that exceed safe limits, and doses that may not be appropriate for the specific patient (based on weight, lab monitoring values, condition being treated, etc.). Your answer should be provided as a percent of prescriptions with the wrong drug, dose, or directions per all prescriptions for the given medication(s) per week, month, quarter, or year. While prescribing error rates vary among different drugs, please consider the data from published research provided in Appendix B and adjust the rates up or down accordingly. Keep in mind that people tend to underestimate error rates, particularly forgetting to count errors that may not be easily captured. 2 In a normal week, month, quarter, or year, what percent of the time do you see prescriptions for this drug with the wrong dose, wrong directions, or wrong drug (not the intended drug)? 0.1% to 100% This includes errors that are detected before they reach the patient. For example, include prescribing errors that were corrected after you called the prescriber. Receipt of Prescription The next set of questions is associated with how prescriptions for the medication(s) are received in the pharmacy: hard-copies brought into the pharmacy, telephone prescriptions, FAXED PRESCRIPTIONS, or ELECTRONIC PRESCRIPTIONS. ELECTRONIC PRESCRIPTIONS include only those that are transmitted directly to the pharmacy computer and populate the required data entry fields. The sum of percentages for all forms of receiving prescriptions into the pharmacy (3a, 3b1, 3c, 3d) should equal 100%. 3a 3b1 3b2 What percent of prescriptions for this medication(s) are hard-copies brought into the pharmacy by either the patient or the patient s representative? What percent of prescriptions for this medication(s) are called into the pharmacy? Of the prescriptions for this medication(s) that are called into the pharmacy, what percent are left on voic ? 0% to 100% 0% to 100% Enter 0% if no prescriptions for the medication(s) are called into the pharmacy. 0% to 100% 3c 3d What percent of prescriptions for this medication(s) are faxed to the pharmacy? What percent of prescriptions for this medication(s) are ELECTRONIC PRESCRIPTIONS? 0% to 100% Please include prescriptions that are printed and entered after being faxed or transmitted by other electronic means as FAXED PRESCRIPTIONS. 0% to 100% ELECTRONIC PRESCRIPTIONS DO NOT require pharmacy staff to enter the drug name, dose/strength, and directions for use. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

11 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or User Comments Receipt of Prescription (continued) For the next set of questions, you will need to estimate how often staff might detect the prescribing error while receiving the prescription into the pharmacy. One question also asks how often prescriptions are read back to the caller when taking DIRECT CALLS from physicians or office staff. READBACK involves transcribing the prescription on a prescription blank as it is being given, and then reading it back to the caller to verify accuracy. 4 What percent of these prescribing errors will be missed when pharmacy staff takes an oral (verbal) prescription on the phone? 5 Of the DIRECT CALLS (not voic ), what percent of the time are the prescriptions NOT read back to the prescriber or office staff to verify accuracy? 6 What percent of the prescribing errors will be missed despite READBACK of an oral (verbal) prescription? 0% to 100% This includes oral prescriptions left on voic and DIRECT CALLS, but it does not include the process of reading back the prescription to the caller. Detecting errors during READBACK is a separate step in the process, which is addressed later. Enter 0% if no oral prescriptions for the medication(s) are called into the pharmacy. 0% to 100% Forgetting to carry out the READBACK process, rather than skipping it, is already factored into the tool and should not be included in your answer. Common reasons for skipping the READBACK process may include: prescriber is in a hurry; the pharmacy is busy; the call is interrupted by a time-urgent task; prior experiences with READBACK have been poorly received; office staff who call in the prescription are unlikely to detect an error. Enter 0% if no oral prescriptions for the medication(s) are called into the pharmacy and/or if READBACK of prescriptions does not occur. 0% to 100% This includes READBACK of prescriptions to office staff, not just prescribers. Enter 0% if no oral prescriptions for the medication(s) are called into the pharmacy and/or if READBACK of prescriptions does not occur. Data Entry of Prescription The next set of questions explores how often pharmacists and PHARMACY ASSOCIATES enter prescriptions for these medications into the pharmacy computers, and how many prescriptions are for refills or for NEW PATIENTS (patients new to the pharmacy or new to the drug therapy). You will also be asked to estimate how often the prescribing errors are missed during data entry given specified conditions. These questions apply only to the data entry process. The data entry verification process and drug utilization review process are addressed later. 7 What percent of prescriptions for this medication(s) are entered into the pharmacy computer by a pharmacy associate? 8 What percent of prescriptions for this medication(s) are for NEW PATIENTS (new to therapy or new to the pharmacy/ chain)? 9 A PHARMACIST is entering a prescription into the profile of an EXISTING PATIENT who has previously taken the same drug or another drug within the same class. On average, what percent of the prescribing errors will be missed by a pharmacist during data entry without the help of a computer alert? 10 A PHARMACIST is entering a prescription into the profile of a NEW PATIENT (new to therapy or new to the pharmacy). On average, what percent of the prescribing errors will be missed by a pharmacist during data entry without the help of a computer alert? 0% to 100% Please be sure to consider daytime, nighttime, and weekend/holiday staffing when determining the percentage. 0% to 100% <1% to 100% Alerts are factored in during the pharmacists drug utilization review (DUR). Enter <1% if pharmacists never enter prescriptions into the pharmacy computer. <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

12 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or User Comments Data Entry Verification The next set of questions deals with the process of verifying the data entry of prescriptions. You will be asked whether your computer system requires data entry verification in order to continue the dispensing process. The questions that follow will ask you to estimate how often data entry verification is performed by the same pharmacist who entered the prescription, and how often the verification process occurs under less than ideal conditions. You will also be asked to estimate how often a pharmacist will miss the prescribing error during the verification process given specified conditions. 11 Does your computer system require some action by the operator (e.g., enter information, press a function key) during data entry verification in order to proceed with the dispensing process? Yes No 12 After a PHARMACIST has entered a prescription, how often is the data entry verification process rushed, inattentive, incomplete, or skipped? 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because an experienced pharmacist initially entered the prescription. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 5% (lowest score) if pharmacists never enter prescriptions into the pharmacy computer. 13 If data entry verification is required after a pharmacist has entered a prescription into the computer, how often do pharmacists have to verify their own data entry (self-check of data entry)? 14 After a PHARMACY ASSOCIATE has entered a prescription, how often is the data entry verification process rushed, inattentive, incomplete, or skipped? 15 For EXISTING PATIENTS who have been taking the same drug or a similar drug within the same class: What percent of the prescribing errors will be missed by a pharmacist verifying another pharmacist's data entry (INDEPENDENT CHECK)? 16 For NEW PATIENTS (new to the pharmacy or new to the drug therapy): What percent of the prescribing errors will be missed by a pharmacist verifying another pharmacist's data entry (INDEPENDENT CHECK)? 17 For EXISTING PATIENTS who have been taking the same drug or a similar drug within the same class: What percent of the prescribing errors will be missed by a pharmacist verifying a PHARMACY ASSOCIATE'S data entry? 0% to 100% Don t forget times when a pharmacist is working alone. Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 0% if pharmacists never enter prescriptions into the pharmacy computer. 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because the checking pharmacist believes the PHARMACY ASSOCIATE S accuracy is very high. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Enter 5% (lowest score) if PHARMACY ASSOCIATES never enter prescriptions into the pharmacy computer. <1% to 100% Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter <1% if pharmacists never enter prescriptions into the pharmacy computer. <1% to 100% <1% to 100% Enter <1% if PHARMACY ASSOCIATES never enter prescriptions into the pharmacy computer. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

13 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or User Comments 18 For NEW PATIENTS (new to the pharmacy or new to the drug therapy): What percent of the prescribing errors will be missed by a pharmacist verifying a PHARMACY ASSOCI- ATE'S data entry? Data Entry Verification (continued) <1% to 100% Enter <1% if PHARMACY ASSOCIATES never enter prescriptions into the pharmacy computer. Drug Utilization Review (DUR) The next set of questions deals with the drug utilization review (DUR) process during which prescribed therapy is evaluated because initial concerns have surfaced. You will be asked about various computer alerts that may appear during data entry to which a pharmacist has access to during the DUR process. You will also be asked how often these alerts are ignored or not given the pharmacists full attention, and, if they are acknowledged by the pharmacist, how often the prescribing error will still be missed during the DUR process. 19 What percent of prescriptions with the prescribing error will cause a computer alert directly related to the type of prescribing error, which can be viewed by a pharmacist during DUR? 20 What percent of prescriptions with the prescribing error will cause a DUPLICATE THERAPY alert that can be viewed by a pharmacist during DUR? 0% to 100% Examples include an OUT-OF-RANGE DOSE alert for prescribing errors related to the dose or directions; or an allergy alert, a contraindication alert, or a look-alike drug name alert related to prescribing the wrong drug. Exclude prescriptions that cause an alert if the alert is not or cannot be viewed by the pharmacist or otherwise communicated to the pharmacist during DUR. 0% to 100% Exclude prescriptions that cause an alert if the alert is not or cannot be viewed by the pharmacist or otherwise communicated to the pharmacist during DUR. 21 What percent of prescriptions with the prescribing error will cause neither a DUPLICATE THERAPY alert nor an alert directly related to the type of prescribing error (e.g., OUT-OF- RANGE DOSE alert for prescribing errors related to the dose or directions; allergy alert; contraindication alert; look-alike drug name alert related to prescribing the wrong drug) during data entry and/or data entry verification? 22 What percent of these prescribing errors will a pharmacist miss when the computer issues an alert directly related to the type of prescribing error, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? 23 What percent of these prescribing errors will a pharmacist miss when the COMPUTER FLAGS the prescription for DUPLI- CATE THERAPY, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? 0% to 100% <1% to 100% Acting on the alert simply means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. Enter <1% if you previously indicated (Question 19) that no (0%) alerts related to the type of prescribing error would be issued or available to the pharmacist conducting DUR. <1% to 100% Acting on the alert simply means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. Enter <1% if you previously indicated (Question 20) that no (0%) DUPLICATE THERAPY alerts would be issued or available to the pharmacist conducting DUR. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

14 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or User Comments 24 What percent of time does a pharmacist fail to give an alert directly related to the type of prescribing error his or her full attention or ignore the alert and quickly bypass it? 25 What percent of time does a pharmacist fail to give a DUPLICATE THERAPY alert for these medication(s) his or her full attention or ignore the alert and quickly bypass it? Drug Utilization Review (DUR) (continued) 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. Enter 5% (lowest score) if you previously indicated (Question 19) that no (0%) alert directly related to the prescribing error would be issued or available to the pharmacist conducting DUR. 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. Enter 5% (lowest score) if you previously indicated (Question 14) that no (0%) DUPLICATE THERAPY alerts would be issued or available to the pharmacist conducting DUR. Insurance Adjudication For the next question, you will need to know the percent of prescriptions typically adjudicated online for third-party payment. Because insurance prescription drug coverage is patient specific, not drug specific, this question applies broadly to all medications you dispense from the pharmacy. This is the only question in this Scorecard that you can answer broadly for all drugs, rather than for the specific medication(s) being evaluated with the prescribing errors. 26 What percent of prescriptions are adjudicated online for third-party payment? 0% to 100% Filling the Prescription The next set of questions deals with filling prescriptions for the medication(s) under evaluation, from selecting the drug off the shelf to applying the label on the product. You will be asked about automated and manual filling of the prescriptions and whether a pharmacist might detect the prescribing errors while filling a prescription. 27 What percent of prescriptions for the medication(s) involved in these prescribing errors are filled using AUTOMATED DIS- PENSING EQUIPMENT (e.g., robotics, dispensing machines)? 0% to 100% 28 For manually filled prescriptions for the medication(s) under evaluation, what percent are filled by PHARMACY ASSOCIATES? 29 What percent of these prescribing errors will be missed while a PHARMACIST is filling the prescription (e.g., counting tablets, filling vials, preparing cartons)? 0% to 100% <1% to 100% Final Product Verification The next few questions involve verification of the filled prescription before placing it in the will-call area and/or dispensing it. You will be asked to consider whether a pharmacist would detect the prescribing error during this step of the dispensing process. 30 What percent of prescriptions associated with this type of prescribing error might a pharmacist recognize as a potential problem without the aid of a computer alert? <1% to 100% 31 If the pharmacist recognizes a potential problem with these prescriptions, what percent of the errors will still be missed during final product verification? <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

15 III. HAMMERS Scorecards Prescribing Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or Rationale for Question 32 How often does a pharmacist rush through the final product verification process, or skip it in part or entirely, before filled prescriptions for the medication(s) are ready for pick-up or delivery? Final Product Verification (continued) <1% to 100% Reasons for rushing through or skipping the final product verification of the filled prescription may include the fact that another pharmacist (not a PHARMACY ASSOCIATE) filled the prescription, the same person who is supposed to conduct the final verification also filled the prescription, and over-reliance on PHARMACY ASSOCIATES during the pharmacy s busiest hours. Patient Counseling The final set of questions deals with counseling patients who pick up filled prescriptions or receive pharmacy deliveries for the medication(s) under evaluation. You will be asked how often this occurs at the counter or at the drive-through window, and whether you open the bag, prescription vial, or carton to view the product during the counseling session. You will also be asked how often you might miss these prescribing errors during counseling given specified conditions. 33 Considering both new prescriptions and refills of the medication(s) that are picked up from the pharmacy or delivered to patients, what percent of the time does patient counseling occur? 34 When patient counseling occurs, what percent of customers receiving the medication(s) are at the drivethrough window? 35 When patient counseling occurs at the counter, what percent of the time is the prescription vial or container opened to view the actual medication(s) under evaluation? 36 What percent of prescribing errors for this medication(s) would be missed when patient counseling occurs at the drive-through window? 37 While counseling a patient at the counter, a pharmacist HAS opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the prescribing error, and the patient leave the counter with the wrong medication, dose, or directions for use? 38 While counseling a patient at the counter, a pharmacist HAS NOT opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the prescribing error, and the patient leave the counter with the wrong medication, dose, or directions for use? 0% to 100% 0% to 100% If you do not offer drive-through services, enter a score of 0%. 0% to 100% <1% to 100% If you do not offer drive-through services, enter a score of <1%. <1% to 100% Enter <1% if you previously indicated (Question 35) that you never (0%) open the prescription vial or container while counseling the patient. <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

16 III. HAMMERS Scorecards Data Entry Errors (Wrong Patient) # Question Answer Special Instructions or Rationale for Question Prescription Volume You have selected the Scorecard associated with data entry errors wrong patient. Because this type of error could involve any prescription filled in the pharmacy, you do not need to choose a specific medication or class of medications for evaluation all medications can be included. However, you will need to provide the volume of prescriptions filled in your pharmacy. Please provide a name for this Scorecard: 1 Please provide the number of prescriptions (new prescriptions and refills combined) filled within the specified time interval in your pharmacy. # of Prescriptions Time Interval: Week, Month, Quarter, Year Include filled prescriptions that are picked up from the pharmacy or delivered to patients. Add all generics, brands, and strengths together. Error Frequency You will need to estimate how often a prescription is initially entered into the wrong patient s profile. This includes errors that may be detected during the dispensing process. Your answer should be provided as a percent of data entry errors among all prescriptions per week, month, quarter, or year. Please consider the data from published research provided in Appendix B, and adjust the rates up or down accordingly. Keep in mind that people tend to underestimate error rates, particularly forgetting to count errors that may not be easily captured. 2 In a normal week, month, quarter, or year, what percent of the time is a prescription or group of prescriptions entered into the wrong patient s profile during data entry? 0.1% to 100% This includes errors that are detected before they reach the patient. For example, include data entry errors that are detected and corrected during order entry verification, final product verification, at the point of sale, or during patient counseling. Data Entry of Prescription The next set of questions explore how often pharmacists and PHARMACY ASSOCIATES enter prescriptions for these medications into the pharmacy computer, and how many prescriptions are for refills or for NEW PATIENTS (patients new to the pharmacy or new to the drug therapy). You will also be asked to estimate how often the data entry errors (wrong patient) are missed given specified conditions. These questions apply only to the data entry process. The data entry verification process and drug utilization review process are addressed later. 3 What percent of prescriptions are entered into the pharmacy computer by a PHARMACY ASSOCIATE? 0% to 100% Please be sure to consider daytime, nighttime, and weekend/holiday staffing when determining the percentage. 4 What percent of all prescriptions are for NEW PATIENTS (new to therapy or new to the pharmacy/chain)? 0% to 100% Data Entry Verification The next set of questions deals with the process of verifying the data entry of prescriptions. You will be asked whether your computer system requires data entry verification in order to continue the dispensing process. The questions that follow will ask you to estimate how often data entry verification is performed by the same pharmacist who entered the prescription, and how often the verification process occurs under less than ideal conditions. You will also be asked to estimate how often a pharmacist will miss the data entry error during the verification process given specified conditions. 5 Does your computer system require some action by the operator (e.g., enter information, press a function key) during data entry verification in order to proceed with the dispensing process? 6 If data entry verification is required after a pharmacist has entered a prescription into the computer, how often do pharmacists have to verify their own data entry (self-check of data entry)? Yes No 0% to 100% Don t forget times when a pharmacist is working alone. Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 0% if pharmacists never enter prescriptions into the pharmacy computer. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

17 III. HAMMERS Scorecards Data Entry Errors (Wrong Patient) continued # Question Answer Special Instructions or Rationale for Question 7 For EXISTING PATIENTS (with an existing pharmacy profile): What percent of the wrong patient data entry errors will be missed by a pharmacist verifying another pharmacist s data entry (INDEPENDENT CHECK)? Data Entry Verification (continued) <1% to 100% Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter <1% if pharmacists never enter prescriptions into the pharmacy computer. 8 For NEW PATIENTS (new to the pharmacy): What percent of the wrong patient data entry errors will be missed by a pharmacist verifying another pharmacist s data entry (INDEPENDENT CHECK)? <1% to 100% 9 For EXISTING PATIENTS (with an existing pharmacy profile): What percent of wrong patient data entry errors will be missed by a pharmacist verifying a PHARMACY ASSOCIATE S data entry? 10 For NEW PATIENTS (new to the pharmacy): What percent of the wrong patient data entry errors will be missed by a pharmacist verifying a PHARMACY ASSOCIATE S data entry? 11 After a PHARMACY ASSOCIATE has entered a prescription, how often is the data entry verification process rushed, inattentive, incomplete, or skipped? 12 After a PHARMACIST has entered a prescription, how often is the data entry verification process rushed, inattentive, incomplete, or skipped? <1% to 100% Enter <1% if PHARMACY ASSOCIATES never enter prescriptions into the pharmacy computer. <1% to 100% 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because the checking pharmacist believes the PHARMACY ASSO- CIATE S accuracy is very high. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Enter 5% (lowest answer) if PHARMACY ASSO- CIATES never enter prescriptions into the pharmacy computer. 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because an experienced pharmacist initially entered the prescription. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Answer 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 5% (lowest score) if PHARMACISTS never enter prescriptions into the pharmacy computer. Drug Utilization Review (DUR) The next set of questions deals with the drug utilization review (DUR) process during which prescribed therapy is evaluated because initial concerns have surfaced. You will be asked about various computer alerts that may appear during data entry to which a pharmacist has access during the DUR process. You will also be asked how often these alerts are ignored or not given the pharmacists full attention, and, if they are acknowledged by the pharmacist, how often the data entry error (wrong patient) will still be missed during the DUR process. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

18 III. HAMMERS Scorecards Data Entry Errors (Wrong Patient) continued # Question Answer Special Instructions or Rationale for Question 13 What percent of the data entry errors will a pharmacist miss if the COMPUTER FLAGS the prescription for an OUT-OF- RANGE DOSE, an allergy, or contraindication, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? Drug Utilization Review (continued) <1% to 100% Acting on the alert means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. 14 On average, what percent of the time does a pharmacist ignore an alert for an OUT-OF-RANGE DOSE, allergy, or contraindication, or fail to give the alert his/her full attention? 15 What percent of the data entry errors will a pharmacist miss if the COMPUTER FLAGS the prescription for DUPLICATE THERAPY, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? 16 On average, what percent of the time does a pharmacist ignore a DUPLICATE THERAPY alert or fail to give the alert his/her full attention? 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. <1% to 100% Acting on the alert simply means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. Insurance Adjudication For the next question, you will need to know the percent of prescriptions typically adjudicated online for third-party payment. This question applies broadly to all medications you dispense from the pharmacy. 17 What percent of prescriptions are adjudicated online for third-party payment? 0% to 100% Filling the Prescription The next set of questions deals with filling prescriptions for the medication(s) under evaluation, from selecting the drug off the shelf to applying the label on the product. You will be asked about automated and manual filling of the prescriptions and whether a pharmacist might detect the data entry error while filling a prescription. 18 What percent of prescriptions are filled using AUTOMATED DISPENSING EQUIPMENT (e.g., robotics, dispensing machines)? 19 For manually filled prescriptions, what percent are filled by PHARMACY ASSOCIATES? 20 What percent of the data entry errors will be missed while a PHARMACY ASSOCIATE Is filling the prescription (e.g., counting tablets, filling vials, preparing cartons)? 0% to 100% 0% to 100% <1% to 100% 21 What percent of the data entry errors will be missed while a PHARMACIST is filling the prescription (e.g., counting tablets, filling vials, preparing cartons)? <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

19 III. HAMMERS Scorecards Data Entry Errors (Wrong Patient) continued # Question Answer Special Instructions or Rationale for Question Final Product Verification The next few questions involve verification of the filled prescription before placing it in the will-call area and/or dispensing it. You will be asked to consider whether a pharmacist would detect the data entry error during this step of the dispensing process. 22 What percent of wrong patient data entry errors will be missed by pharmacists during final product verification? 23 How often does a pharmacist rush through the final product verification process, or skip it in part or entirely, before filled prescriptions are ready for pick-up or delivery? <1% to 100% If the original prescription, a scanned image, or ELECTRONIC PRESCRIPTION is not available to view and compare to the pharmacy label during the final product verification, do not enter a score lower than 90%. <1% to 100% Reasons for rushing through or skipping the final product verification of the filled prescription may include the fact that another pharmacist (not a PHARMACY ASSOCIATE) filled the prescription, the same person who is supposed to conduct the final verification also filled the prescription, or over-reliance on PHARMACY ASSOCIATES during the pharmacy s busiest hours. Patient Counseling The final set of questions deals with counseling patients who pick up medications or receive pharmacy deliveries of filled prescriptions. You will be asked how often this occurs at the counter or at a drive-through window, and whether you open the bag, prescription vial, or carton to view the actual product during the counseling session. You will also be asked how often you might miss the data entry errors (wrong patient) during counseling given specified conditions. 24 Considering both new prescriptions and refills of all medication(s) that are picked up from the pharmacy or delivered to patients, what percent of the time does patient counseling occur? 25 When patient counseling occurs, what percent of these customers are at the drive-through window? 26 What percent of the data entry errors (wrong patient) would be missed when patient counseling occurs at the drive-through window? 27 When patient counseling occurs at the counter, what percent of the time is the prescription vial or container opened to view the actual product? 28 While counseling a patient at the counter, a pharmacist HAS NOT opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication? 29 While counseling a patient at the counter, a pharmacist HAS opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication? 0% to 100% 0% to 100% If you do not offer drive-through services, enter a score of 0%. <1% to 100% If you do not offer drive-through services, enter a score of <1%. 0% to 100% <1% to 100% <1 to 100% If you never open the prescription vial or container while counseling the patient, enter a score of <1%. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

20 III. HAMMERS Scorecards Data Entry Errors (Wrong Drug, Dose, or Directions) # Question Answer Special Instructions or Rationale for Question Drug Selection and Prescription Volume You have selected the Scorecard associated with data entry errors wrong drug, dose, or directions. Now you must specify which HIGH-ALERT MEDICATION(S) you want to evaluate, and how often your pharmacy fills prescriptions for these medications. While this Scorecard can be used to evaluate data entry errors with any drug, focusing on HIGH-ALERT MEDICATIONS helps to reduce the risk of errors that can cause harm to patients. 1a Please list the name(s) of the medication(s) or class of medications involved in the data entry errors (wrong drug, dose, directions) you want to evaluate. List Medication(s) 1b Please provide the number of prescriptions (new prescriptions and refills combined) filled within the specified time interval for the medication(s) involved in these errors. # of Prescriptions Time Interval: Week, Month, Quarter, Year Include filled prescriptions that are picked up from the pharmacy or delivered to patients. Add all generics, brands, and strengths together. Error Frequency You will need to estimate how often data entry errors (wrong drug, dose, directions) occur when entering prescriptions for the medication(s) being evaluated. This includes errors that may be detected and corrected during the dispensing process. Your answer should be provided as a percent of data entry errors among all prescriptions for the targeted medication(s) per week, month, quarter, or year. While the frequency of data entry errors (wrong drug, dose, directions) often varies based on the medication(s) under evaluation, please consider the data from published research provided in Appendix B, and adjust the rates up or down accordingly. Keep in mind that people tend to underestimate error rates, particularly forgetting to count errors that may not be easily captured. 2 In a normal week, month, quarter, or year, what percent of the time is the wrong drug, wrong dose, or wrong directions entered into the patient s profile during data entry? 0.1% to 100% Include errors that are detected before they reach the patient. For example, include data entry errors that are detected and corrected during data entry verification, final product verification, at the point of sale, or during patient counseling. Data Entry of Prescription The next set of questions explore how often pharmacists and PHARMACY ASSOCIATES enter prescriptions for these medications into the pharmacy computer, and how many prescriptions are for refills or for NEW PATIENTS (patients new to the pharmacy or new to the drug therapy). You will also be asked to estimate how often the data entry errors (wrong drug, dose, directions) will be missed given the specified conditions. These questions apply only to the data entry process. The data entry verification process and drug utilization review process are addressed later. 3 What percent of prescriptions for the medication(s) are entered into the pharmacy computer by a PHARMACY ASSOCIATE? 0% to 100% Please be sure to consider daytime, nighttime, and weekend/holiday staffing when determining the percentage. 4 What percent of prescriptions for the medication(s) are for NEW PATIENTS (new to therapy or new to the pharmacy/ chain)? 0% to 100% Data Entry Verification The next set of questions deals with the process of verifying the data entry of prescriptions. You will be asked whether your computer system requires data entry verification in order to continue the dispensing process. The questions that follow will ask you to estimate how often data entry verification is performed by the same pharmacist who entered the prescription, and how often the verification process occurs under less than ideal conditions. You will also be asked to estimate how often a pharmacist will miss the data entry error during the verification process given specified conditions. 5 Does your computer system require some action by the operator (e.g., enter information, press a function key) during data entry verification in order to proceed with the dispensing process? Yes No 6 If data entry verification is required after a pharmacist has entered a prescription into the computer, how often do pharmacists have to verify their own data entry (self-check of data entry)? 0% to 100% Don t forget times when a pharmacist is working alone. Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 0% if pharmacists never enter prescriptions into the pharmacy computer. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

21 III. HAMMERS Scorecards Data Entry Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or Rationale for Question 7 After a PHARMACIST has entered a prescription for this medication(s), how often is the data entry verification process rushed, inattentive, incomplete, or skipped? Data Entry Verification (continued) 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because an experienced pharmacist initially entered the prescription. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter 5% (lowest score) if pharmacists never enter prescriptions into the pharmacy computer. 8 After a PHARMACY ASSOCIATE has entered a prescription for this medication(s), how often is the data entry verification process rushed, inattentive, incomplete, or skipped? 5% to 100% Forgetting to carry out the data entry verification process, rather than skipping it or hurrying through it, has already been factored into the tool and should not be included in your answer. Sometimes data entry verification is rushed or skipped because the checking pharmacist believes the PHARMACY AS- SOCIATE S accuracy is very high. Other times, multiple patients waiting for prescriptions and a hectic pace in the pharmacy can lead to rushed, incomplete, inattentive, or skipped data entry verification. Enter 5% (lowest score) if PHARMACY ASSO- CIATES never enter prescriptions into the pharmacy computer. 9 For EXISTING PATIENTS who have been taking the same drug or a similar drug within the same class: What percent of the data entry errors will be missed by a pharmacist verifying another pharmacist s data entry (INDEPENDENT CHECK)? <1% to 100% Enter 100% if your dispensing process does not require data entry verification after a PHARMACIST has entered a prescription. Enter <1% if pharmacists never enter prescriptions into the pharmacy computer. 10 For NEW PATIENTS (new to the pharmacy or new to the drug therapy): What percent of the data entry errors will be missed by a pharmacist verifying another pharmacist s data entry (INDEPENDENT CHECK)? <1% to 100% 11 For EXISTING PATIENTS who have been taking the same drug or a similar drug within the same class: What percent of the data entry errors will be missed by a pharmacist verifying a PHARMACY ASSOCIATE S data entry? <1% to 100% Enter <1% if PHARMACY ASSOCIATES never enter prescriptions into the pharmacy computer. 12 For NEW PATIENTS (new to the pharmacy or new to the drug therapy): What percent of the data entry errors will be missed by a pharmacist verifying a PHARMACY ASSOCIATE S data entry? <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

22 III. HAMMERS Scorecards Data Entry Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or Rationale for Question Drug Utilization Review (DUR) The next set of questions deals with the drug utilization review (DUR) process during which prescribed therapy is evaluated because initial concerns have surfaced. You will be asked about various computer alerts that may appear during data entry to which a pharmacist has access during the DUR process. You will also be asked how often these alerts are ignored or not given the pharmacists full attention, and, if they are acknowledged by the pharmacist, how often the data entry error (wrong drug, dose, directions) will still be missed during the DUR process. 13 What percent of the data entry errors will cause an OUT- OF-RANGE DOSE alert during data entry that can be viewed by a pharmacist during DUR? 14 What percent of the data entry errors will cause a DUPLI- CATE THERAPY alert that can be viewed by a pharmacist during DUR? 15 What percent of the data entry errors will cause neither an OUT-OF-RANGE DOSE alert nor a DUPLICATE THERAPY alert during data entry and/or data entry verification? 16 What percent of the data entry errors will a pharmacist miss when the COMPUTER FLAGS the prescription for an OUT- OF-RANGE DOSE, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? 17 What percent of the data entry errors will a pharmacist miss when the COMPUTER FLAGS the prescription for DUPLI- CATE THERAPY, the alert is available to the pharmacist conducting DUR, and the pharmacist acts on the alert? 18 What percent of time does a pharmacist fail to give an OUT- OF-RANGE DOSE alert for these medication(s) his or her full attention or ignore the alert and quickly bypass it? 0% to 100% Exclude data entry errors that cause an OUT-OF-RANGE DOSE alert if the alert is not or cannot be viewed by the pharmacist or otherwise communicated to the pharmacist during DUR. 0% to 100% Exclude data entry errors that cause a DUPLICATE THERAPY alert if the alert is not or cannot be viewed by the pharmacist or otherwise communicated to the pharmacist during DUR. 0% to 100% <1% to 100% Acting on the alert simply means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. Enter <1% if you previously indicated (Question 13) that no (0%) OUT-OF-RANGE DOSE alerts would be issued or would be available to the pharmacist conducting DUR. <1% to 100% Acting on the alert simply means that the pharmacist was aware of the alert and considered its importance. Action to correct an error may or may not occur. For example, a pharmacist may notice an alert, consider its importance, but still believe the order is acceptable. Or a pharmacist could call the prescriber s office, but the office nurse communicates that the prescription is correct so no further action is taken. Enter <1% if you previously indicated (Question 14) that no (0%) DUPLICATE THERAPY alerts would be issued or would be available to the pharmacist conducting DUR. 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. Enter 5% (lowest score) if you previously indicated (Question 13) that no (0%) OUT-OF-RANGE DOSE alerts would be issued or would be available to the pharmacist conducting DUR. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

23 III. HAMMERS Scorecards Data Entry Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or Rationale for Question 19 What percent of time does a pharmacist fail to give a DUPLICATE THERAPY alert for these medication(s) his or her full attention or ignore the alert and quickly bypass it? Drug Utilization Review (DUR) (continued) 5% to 100% Sometimes an alert is ignored or bypassed because the presenting issue appears to be inconsequential, is thought to have been addressed previously, or does not adequately capture the pharmacist s attention due to ALERT FATIGUE. Enter 5% (lowest score) if you previously indicated (Question 14) that no (0%) DUPLICATE THERAPY alerts would be issued or would be available to the pharmacist conducting DUR. Insurance Adjudication For the next question, you will need to know the percent of prescriptions typically adjudicated online for third-party payment. Because insurance prescription drug coverage is patient specific, this question applies broadly to all medications you dispense from the pharmacy. This is the only question in this Scorecard that you can answer broadly for all drugs, rather than for the specific medication(s) being evaluated with the data entry errors. 20 What percent of prescriptions are adjudicated online for third-party payment? 0% to 100% Filling the Prescription The next set of questions deals with filling prescriptions for the medication(s) under evaluation, from selecting the drug off the shelf to applying the label on the product. You will be asked about automated and manual filling of the prescriptions and whether a pharmacist might detect the data entry error while filling a prescription. 21 What percent of prescriptions for the medication(s) involved in these data entry errors are filled using AUTO- MATED DISPENSING EQUIPMENT (e.g., robotics, dispensing machines)? 0% to 100% 22 For manually filled prescriptions for the medication(s) under evaluation, what percent are filled by PHARMACY ASSOCIATES? 23 What percent of the data entry errors will be missed while a PHARMACIST is filling the prescription (e.g., counting tablets, filling vials, preparing cartons)? 0% to 100% <1% to 100% Final Product Verification The next few questions involve verification of the filled prescription before placing it in the will-call area and/or dispensing it. You will be asked to consider whether a pharmacist would detect the data entry error during this step of the dispensing process. 24 What percent of the data entry errors might lead a pharmacist to recognize an OUT-OF-RANGE DOSE for a particular patient without the aid of a computer alert? 25 If the dose is recognized by the pharmacist as outside an acceptable range of possible doses, what percent of the data entry errors involving this medication(s) will still be missed during final product verification? 26 How often does a pharmacist rush through the final product verification process, or skip it in part or entirely, for the medication(s) under evaluation? <1% to 100% <1% to 100% <1% to 100% Reasons for rushing through or skipping the final product verification of the filled prescription may include the fact that another pharmacist (not a PHARMACY ASSOCIATE) filled the prescription, the same person who is supposed to conduct the final verification also filled the prescription, or over-reliance on PHARMACY ASSOCIATES during the pharmacy s busiest hours. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

24 III. HAMMERS Scorecards Data Entry Errors (Wrong Drug, Dose, or Directions) continued # Question Answer Special Instructions or Rationale for Question Patient Counseling The final set of questions deals with counseling patients who pick up filled prescriptions or receive pharmacy deliveries for the medication(s) under evaluation. You will be asked how often this occurs at the counter or at a drive-through window, and whether you open the bag, prescription vial, or carton to view the actual product during the counseling session. You will also be asked how often you might miss the data entry errors (wrong drug, dose, directions) during counseling given specified conditions. 27 Considering both new prescriptions and refills of this medication(s) that are picked up from the pharmacy or delivered to patients, what percent of the time does patient counseling occur? 28 When patient counseling occurs, what percent of customers receiving the medication(s) are at the drivethrough window? 29 When patient counseling occurs at the counter, what percent of the time is the prescription vial or container opened to view the actual medication(s) under evaluation? 30 What percent of the data entry errors with this medication(s) would be missed when patient counseling occurs at the drive-through window? 31 While counseling a patient at the counter, a pharmacist HAS opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication, strength, or directions for use? 32 While counseling a patient at the counter, a pharmacist HAS NOT opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication, strength, or directions for use? 0% to 100% 0% to 100% If you do not offer drive-through services, enter a score of 0%. 0% to 100% <1% to 100% If you do not offer drive-through services, enter a score of <1%. <1% to 100% If you never open the prescription vial or container while counseling the patient, enter a score of <1%. <1% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

25 III. HAMMERS Scorecards Drug Container Selection Errors (Wrong Drug or Wrong Dose) # Question Answer Special Instructions or Rationale for Question Drug Selection and Prescription Volume You have selected the Scorecard associated with drug container selection errors (wrong drug or dose). Now you must specify which HIGH-ALERT MEDICATION(S) you want to evaluate, and how often your pharmacy fills prescriptions for these medications. While this tool can be used to evaluate drug container selection errors with any drug, focusing on HIGH-ALERT MEDICATIONS helps to reduce the risk of errors that can cause harm to patients. 1a Please list the name(s) of the medication(s) or class of medications involved in selecting the wrong product containers you want to evaluate. List Medication(s) 1b Please provide the number of prescriptions (new prescriptions and refills combined) filled within the specified time interval for the medication(s) involved in these errors. # of Prescriptions Time Interval: Week, Month, Quarter, Year Include filled prescriptions that are picked up from the pharmacy or delivered to patients. Add all generics, brands, and strengths together. Error Frequency You will need to estimate how often you initially select the wrong product container when filling prescriptions for the medication(s) under evaluation. This may involve selecting the wrong dose, concentration, dosage form, or drug. Your answer should be provided as a percent of product container selection errors given all prescriptions for the targeted medication(s) per week, month, quarter, or year. While drug selection error rates often vary among different drugs, please consider the data from published research provided in Appendix B, and adjust the rates up or down accordingly. Keep in mind that people tend to underestimate error rates, particularly forgetting to count errors that may not be easily captured. 2 In a normal week, month, quarter, or year, what percent of the time is the wrong drug or dose initially selected from the shelf or refrigerator when manually filling a prescription for this drug? 0.1% to 100% Include all product selection errors even if they are noticed immediately or captured and corrected at any time before filling the prescription. For example, if barcode scanning is used, include product selection errors that are detected when the barcode is scanned. Filling the Prescription The next set of questions deals with filling prescriptions for the medication(s) under evaluation, from selecting the drug off the shelf to applying the pharmacy label on the product. You will be asked about automated and manual filling of the prescriptions, the use of barcode scanning for verification of product selection, and whether a pharmacist might detect the drug selection error while filling a prescription. 3 What percent of prescriptions for the medication(s) under evaluation are filled using AUTOMATED DISPENSING EQUIP- MENT (e.g., robotics, dispensing machines)? 4 For manually filled prescriptions for the medication(s) under evaluation, what percent are filled by PHARMACY ASSOCIATES? 5 While manually filling prescriptions for this medication(s), how often would a barcode scanning process to verify product selection be bypassed? 6 How often might you use a CHEAT SHEET to scan the barcode of the prescribed drug and dose for this type of medication(s)? 0% to 100% 0% to 100% <1% to 100% Exclude times when the NDC number on the stock bottle or product (not from the patient label) has been manually entered into the bar-coding system due to problems with a particular barcode. However, scanning the same barcode several times, rather than each individually, when dispensing more than one container of the same medication should be factored in to the percentage of bypassed medications. If barcode scanning technology is not available in your pharmacy to verify product selection, enter a score of 100%. 0% to 100% If barcode scanning technology is not available in your pharmacy for product verification, enter a score of 0%. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

26 III. HAMMERS Scorecards Drug Container Selection Errors (Wrong Drug or Wrong Dose) continued # Question Answer Special Instructions or Rationale for Question 7 What percent of these product container selection errors will be missed while a PHARMACIST is filling the prescription (e.g., counting tablets, filling vials, preparing cartons)? Filling the Prescription (continued) <1% to 100% If barcode scanning technology is available to verify product selection, answer this question as though the barcode scanning process had been bypassed. Your answer will only be applicable to the percent of time you said the barcode scanning process was bypassed. Final Product Verification The next few questions involve verification of the filled prescription before placing it in the will-call area and/or dispensing it. You will be asked whether an image of the pill or package is available to the pharmacist during product verification and to consider whether a pharmacist would detect the drug selection error during this step of the dispensing process. 8 For the medication(s) involved in these product container selection errors, how often would a tablet, capsule, or package image be available on the screen during final product verification? 9 What percent of the product container selection errors will be missed during final product verification if the pharmacist HAS an image of the prescribed drug on the screen? 10 What percent of the wrong drug/dose selection errors will be missed during final product verification if the pharmacist does NOT have an image of the prescribed drug on the screen? 11 How often does a pharmacist rush through the final product verification process, or skip it in part or entirely, for the medication(s) under evaluation? 0% to 100% If pill imaging is not available in your pharmacy, enter a score of 0%. <1% to 100% If pill imaging is not available, enter a score of <1%. <1% to 100% If pill imaging technology is available in your pharmacy, answer the question as if you did NOT have an image on the screen. Your answer will only be applicable to the percent of time you said there may not be an image available. <1% to 100% Reasons for rushing through or skipping the final product verification of the filled prescription may include the fact that another pharmacist (not a PHARMACY ASSOCIATE) filled the prescription, the same person who filled the prescription is also conducting the final verification, and over-reliance on PHARMACY ASSOCIATES during the pharmacy s busiest hours. Patient Counseling The final set of questions deals with counseling patients who pick up filled prescriptions or receive pharmacy deliveries for the medication(s) under evaluation. You will be asked how often this occurs at the counter or at a drive-through window, and whether you open the bag, prescription vial, or carton to view the actual product during the counseling session. You will also be asked how often you might miss the drug container selection errors during counseling given specified conditions. 12 Considering both new prescriptions and refills of this medication(s) that are picked up from the pharmacy or delivered to patients, what percent of the time does patient counseling occur? 0% to 100% 13 When patient counseling occurs, what percent of customers receiving the medication(s) are at the drivethrough window? 0% to 100% If you do not offer drive-through services, enter a score of 0%. 14 What percent of the product container selection errors for this medication(s) would be missed when patient counseling occurs at the drive-through window? <1% to 100% If you do not offer drive-through services, enter a score of <1%. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

27 III. HAMMERS Scorecards Drug Container Selection Errors (Wrong Drug or Wrong Dose) continued # Question Answer Special Instructions or Rationale for Question 15 While counseling a patient at the counter, a pharmacist HAS NOT opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication or strength/form of the drug? Patient Counseling (continued) <1% to 100% 16 When patient counseling occurs at the counter, what percent of the time is the prescription vial or container opened to view the actual medication(s) under evaluation? 0% to 100% 17 While counseling a patient at the counter, a pharmacist HAS opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong medication or strength/form of the drug? <1% to 100% If you never open the prescription vial or container while counseling the patient, enter a score of <1%. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

28 III. HAMMERS Scorecards Point of Sale Errors (Wrong Customer) # Question Answer Special Instructions or Rationale for Question Prescription Volume You have selected the Scorecard associated with point of sale errors (medication dispensed to the wrong patient). Because this type of error could involve any prescription filled in the pharmacy, you do not need to specify a specific medication or class of medications for evaluation all medications can be included. However, you will need to provide the volume of prescriptions filled in your pharmacy. Please provide a name for this Scorecard: 1 Please provide the number of prescriptions (new prescriptions and refills combined) filled within the specified time interval in your pharmacy. # of Prescriptions Time Interval: Week, Month, Quarter, Year Include filled prescriptions that are picked up from the pharmacy or delivered to patient. Add all generics, brands, and strengths together. Error Frequency You will need to estimate how often you initially retrieve the wrong patient s filled prescription(s) from the will-call area or counter and begin to initiate the sales transaction with the wrong customer. You will also be asked how often staff accidentally place one patient s medications into the bag of another patient s medications. Your combined answer should be provided as a percent of errors given all prescriptions per week, month, quarter, or year. Please consider the data from published research provided in Appendix B, and adjust the rates up or down accordingly. Keep in mind that people tend to underestimate error rates, particularly forgetting to count errors that may not be easily captured. 2 In a normal week, month, quarter, or year, what percent of the time does pharmacy staff accidentally retrieve the wrong patient s bag from the will-call area or filled prescription from the counter, and begin to initiate the sales transaction with the wrong customer? 0.1% to 100% Include all errors that are captured and corrected. For example, if the customer or PHARMACY ASSOCIATE notices the mistake while still at the pharmacy counter, include these errors when estimating the percent of prescriptions initially given to the wrong customer. 3 In a normal week, month, quarter, or year, what percent of the time does pharmacy staff accidentally place one patient s medications into the bag of another patient s medications (bagging error)? 0.1% to 100% Include all bagging errors, even those that are captured and corrected before the patient leaves the pharmacy counter. Point of Sale The next set of questions deals with the interaction between the customer and pharmacy staff at the point of sale. You will be asked about the processes used to identify patients at the point of sale and ensure that the correct medications are in the pharmacy bag. 4 What percent of the time does pharmacy staff fail to ask the patient (or customer) for two unique patient identifiers (full name and date of birth or address) when retrieving the patient s prescription from the will-call area or counter? <1% to 100% Reasons for not asking the customer for two unique patient identifiers include staff familiarity with the patient; hectic pace of the pharmacy; time urgency; and perceived ability to remember patients names, particularly if the customer was asked for name and address/birth date when dropping off the prescription to be filled. 5 At the point of sale, how often do pharmacy staff open the bag that contains the patient s medications to view each prescription vial, carton, tube, or other container, and read the label to ensure that the medication is for the intended patient? 0% to 100% Patient Counseling The final set of questions deals with counseling patients who pick up filled prescriptions or receive pharmacy deliveries of filled prescriptions. You will be asked how often this occurs at the counter or at a drive-through window, and whether you open the bag, prescription vial, or carton to view the actual product during the counseling session. You will also be asked how often you might miss wrong patient errors (giving the patient medication intended for another patient) during counseling given specified conditions. 6 Considering both new prescriptions and refills that are picked up from the pharmacy or delivered to patients, what percent of the time does patient counseling occur? 0% to 100% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

29 III. HAMMERS Scorecards Point of Sale Errors (Wrong Customer) continued # Question Answer Special Instructions or Rationale for Question 7 When patient counseling occurs, what percent of these customers are at the drive-through window? Patient Counseling (continued) 0% to 100% If you do not offer drive-through services, enter a score of 0%. 8 When patient counseling occurs at the counter, what percent of the time is the prescription vial or container opened to view the actual product? 0% to 100% 9 While counseling a patient at the counter, a pharmacist HAS NOT opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong patient s medications? <1% to 100% 10 While counseling a patient at the counter, a pharmacist HAS opened the prescription vial or container to show the patient the actual product. During the counseling session, what percent of the time would the pharmacist or patient fail to notice the error, and the patient leave the counter with the wrong patient s medications? <1% to 100% If you never open the prescription vial or container while counseling the patient, enter a score of <1%. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

30 IV. Frequently Asked Questions and Glossary Frequently Asked Questions 1 Why do the answer choices for some questions begin at 0% and others begin at 0.1% or <1%? Why can t I choose 0% for some questions? Multiplication is one of the mathematical calculations performed by the HAMMERS tool. Any number multiplied by zero equals zero. Thus, when dealing with missed CAPTURE OPPORTUNITIES, HUMAN ERROR, equipment failure, and AT- RISK BEHAVIORS, your answer must be greater than zero to avoid canceling out important data when all risks are combined. Only questions that are associated with exposure rates, such as how often you receive ELECTRONIC PRESCRIP- TIONS, allow an answer choice of zero if the element under investigation is wholly absent. In these cases, the tool appropriately cancels out all risks associated with the absent element. Selecting an answer of <1%, even if the most accurate answer would be zero, does not result in measurable differences in the overall calculated risk. 2 Why are there lower limits for some answer choices? For example, why do some answer choices start at 5% and not 0%? Lower limits (5%) were set for just a few questions that ask about the frequency of AT-RISK BEHAVIORS to align with the lowest documented rate of participation in the behaviors. Users can choose a higher rate of participation in the AT-RISK BEHAVIOR, but the lower limit has been set in the tool. 3 How should I answer questions associated with Drug Utilization Review (DUR) if I anticipate receiving several alerts at the same time? In the HAMMERS tool, each alert is handled separately. So, please answer the questions as though you have received just the alert under evaluation. 4 It seems like some important questions are missing. How does the tool account for the missing questions? Some risks have been preset in the HAMMERS tool, particularly HUMAN ERROR rates established in the literature, if there is little to no variability of the conditions associated with the risk between pharmacies or for different types of medications. Also, some exposure rates are calculated by the tool based on the answer to an associated question. For example, the tool asks how often PHARMACY ASSOCIATES enter prescriptions into the pharmacy computer. Once that rate has been selected, the tool can calculate the remaining percent of prescriptions entered by pharmacists, so that question does not need to be asked. 5 How does the tool calculate the risk? After you have answered all the questions in the Scorecard, the HAMMERS tool used this information for two primary purposes. First, the tool defines all the different chain of events that could lead to the specific error type under evaluation. For each type of error, there will be thousands of different pathways to the error defined by your answer choices. A typical data entry error, for example, will result in hundreds of thousands of different failure pathways that can lead to the error, given the number of steps in that process and variations on how they are carried out. A typical point of sale error in which you provide a customer with another customer s filled prescription medication may result in tens of thousands of failure pathways, given that there are fewer steps involved in the entire process. Once all the possible pathways to the error have been defined, the tool used the estimates from your answer choices, along with preset values in the tool that do not vary from pharmacy to pharmacy, to calculate how often each failure pathway may lead to the error. Then, the risk associated with each failure pathway is combined to provide an overall estimated risk of the error reaching the patient. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

31 IV. Frequently Asked Questions and Glossary Frequently Asked Questions continued 6 Based on the sequence of questions in the Scorecards, the process of data entry verification occurs right after the prescription has been entered into the pharmacy computer. How should I answer the questions about data entry verification if, in my pharmacy, this process occurs after the prescription has been filled and is combined with the final product verification? When determining overall risk, it makes no difference in the HAMMERS Scorecard where in the dispensing process data entry verification occurs. Thus, simply answer all questions associated with data entry verification as it applies to your pharmacy workflow. 7 How do I answer questions about patient identification at the point of sale and patient counseling if the patient is not the person picking up the prescriptions? The inability to obtain accurate patient birth dates, addresses, or telephone numbers from customers who are picking up prescriptions for family or friends is a challenge in many pharmacies, as is the inability to counsel the patient under these circumstances. Thus, when answering questions related to patient identification or patient counseling, consider how often this challenge occurs and how you are currently managing it, and answer the questions accordingly. 8 Why doesn t the Scorecard provide Results when I click on the NEXT button after answering the last question in the Scorecard? Scorecard Results will only be calculated once you have answered all the questions. If you believe you have answered the last question in the Scorecard and nothing happens when you click NEXT, you have inadvertently left one of the Scorecard questions unanswered. Look at the bottom of the screen, where the number of questions you have answered is listed, to confirm that this is the problem. Use the PREVIOUS button to return to the questions to find and answer any unanswered questions. The Results can be accessed once all questions have been answered. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

32 IV. Frequently Asked Questions and Glossary Glossary Alert fatigue: Overuse of alerts issued during data entry whereby the practitioner, after receiving too many alerts, begins to ignore and/or override the alerts, risking the chance that an important alert will be overlooked. At-risk behavior: A behavioral choice that increases risk where risk is not recognized or is mistakenly believed to be justified. AT-RISK BEHAVIORS are often employed and tacitly encouraged as workarounds for various system, technological, or environmental weaknesses. Example: The patient is not identified at the point of sale using the full name and a second unique identifier, such as date of birth or address, because pharmacy staff know the patient. Automated dispensing equipment and robotics: Computer-controlled mechanical devices used to fill prescriptions for oral tablets and capsules and sometimes other forms of medications. Examples include Baker s Cells and ScriptPro Robotics, and Scriptech and AutoScript dispensing equipment. Important note: Automated dispensing equipment does NOT include automatic capsule/tablet counters. Capture opportunities: Activities or conditions that actively or passively help staff detect and correct errors before they reach patients. Failed capture opportunities may be caused by HUMAN ERROR, system risk, or AT-RISK BEHAVIOR. Example: A pharmacist fails to capture a prescribing error during drug utilization review. Many missed capture opportunities do not represent HUMAN ERROR, but the opportunity to catch a mistake that was not realized. Example: Patient fails to capture a dispensing error at the point of sale. Cheat sheet: An easily accessible sheet of scannable product barcodes or retained product carton(s) that are scanned inappropriately instead of the barcode on the actual product, which may be difficult to scan or inconvenient to access (e.g., barcode on the original outer carton of a refrigerated product). Use of the cheat sheet prevents detection of misidentified products, as a surrogate barcode for the correct product is scanned instead of the actual product container barcode. Computer flag: An alert or warning issued by the pharmacy computer system during data entry and/or data entry verification. Direct calls: Real-time telephone conversations between pharmacy staff, usually a pharmacist, and a prescriber or prescriber s representative, such as a physician assistant, nurse, or designated office staff. Duplicate therapy: Therapeutic or pharmacologic repetitions of two or more drugs (or the same drug in different doses/strengths) that create an increased potential for additive toxicity or adverse effects, or cause therapeutic redundancy without increased benefit. Electronic prescription: A prescription transmitted directly to the pharmacy computer that populate the required data entry fields and do not require data entry of the drug name, dose/strength and/or directions for use by a pharmacist. Electronic prescriptions undergo data entry verification only. Existing patient: A patient for whom an active medication profile exists in the pharmacy computer and who has previously taken the same drug currently prescribed or another drug within the same class. Faxed prescription: A prescription that has been sent to the pharmacy via a facsimile machine, including handwritten prescriptions, typed prescriptions, and ELECTRONIC PRESCRIPTIONS that are printed after being faxed to a queue or transmitted by other electronic means. (ELECTRONIC PRESCRIPTIONS include only those that are transmitted directly to the pharmacy computer and populate the required data entry fields and do not require data entry of drug name, dose/strength and/or directions for use.) Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

33 IV. Frequently Asked Questions and Glossary Glossary continued High-alert medications: Medications that have a high risk of causing serious injury or death to a patient if they are misused. Errors with these products are not necessarily more common but their results are more devastating. Examples of high-alert medications dispensed from community pharmacies include: antiretroviral agents, chemotherapy, opioids, heparin, metformin, and warfarin. A complete list can be found in Appendix A. Human error: Inadvertently doing other than what should have been done; the failure of a planned action to be completed as intended (error of execution) or the use of a wrong plan to achieve an aim (error of planning). Example: A physician prescribed the wrong dose of a fentanyl transdermal patch due to a knowledge deficit. Independent check (second-party double check): A procedure in which one individual performs a task and a different individual independently checks each component of the work to ensure it is correct. An example would be one person entering a prescription into the pharmacy computer and another person verifying that the prescription has been entered correctly. New patient: A patient who is a new customer in the pharmacy for whom an active medication profile does not exist in the pharmacy computer or a patient who has never taken the drug currently prescribed or another drug in the same class (new to therapy). Out-of-range dose: A dose of a medication that lies outside the lower and upper limits that are normally found in professional compendia and/or manufacturer recommendations. Lower and upper dose limits may vary according to age, weight, or diagnosis. Performance shaping factors (PSFs): Internal or external influences that enhance or degrade human performance. Internal PSFs are influences that the individual brings to the situation such as mood, fitness, stress level, fatigue, skill, and knowledge. External PSFs are influences in the situation or environment that affect the individual such as noise, workflow, workload, job aids, and staffing patterns. Pharmacy associate: Trained pharmacy staff, including pharmacy technicians, pharmacy students, and pharmacy residents, who assist with certain steps in the dispensing process under the direct supervision of a licensed pharmacist. Readback: A redundant safeguard in which the pharmacist who receives an oral (verbal) prescription transcribes the information onto a pharmacy prescription pad and then reads back the prescription to the prescriber or prescriber s agent to verify accuracy. Readback requires a transcribed prescription from which to read for verification, which differs from repeat back or echoing the prescription from memory. Robotics and automated dispensing equipment: Computer-controlled mechanical devices used to fill prescriptions for oral tablets and capsules and sometimes other forms of medications. Examples include Baker s Cells and ScriptPro Robotics, and Scriptech and AutoScript dispensing equipment. Important note: Automated dispensing equipment does NOT include automatic capsule/tablet counters. Socio-technical probabilistic risk assessment (ST-PRA): An advanced, prospective risk assessment process that models multiple combinations of HUMAN ERROR, AT-RISK BEHAVIORS, system and equipment failures, and potential CAPTURE OPPORTUNITIES leading to a single adverse outcome. ST-PRA provides an estimate of how often the adverse outcome occurs, determines which tasks or conditions most often lead to the adverse outcome, and demonstrates the impact of making changes to the tasks or conditions. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

34 Appendix A: High-Alert Medications High-Alert Medications Dispensed from Community Pharmacies HIGH-ALERT MEDICATIONS are drugs that bear a heightened risk of causing significant patient harm when they are used in error. Although mistakes may or may not be more common with these drugs, the consequences of an error are clearly more devastating to patients. The list was developed through in-depth evaluation of harmful medication errors reported to the ISMP National Medication Errors Reporting Program (ISMP MERP), the Pennsylvania Patient Safety Reporting System, and the FDA MedWatch database. Harmful medication errors published in the literature, public litigation data, and data from a survey of community pharmacy practitioners were also evaluated by a team of medication safety experts before the list was compiled. Community pharmacy practitioners can use this list to determine which medications require special safeguards to reduce the risk of errors and minimize harm. This may include strategies like providing mandatory patient education; improving access to information about these drugs; using auxiliary labels and automated alerts; employing automated or INDEPENDENT CHECKS when necessary; and standardizing the prescribing, storage, dispensing, and administration of these products. High-Alert Medications in Community Pharmacy Drug Class/Category Examples Antiretroviral agents abacavir atazanavir deaviridine lamivudine ritonavir zidovudine Chemotherapy, oral (exclusion: hormonal agents) Hypoglycemic agents, oral busulfan chlorambucil cyclophosphamide lomustine melphalan chlorpropamide glipizide Immunosuppressant agents azathioprine cyclosporine daclizumab mycophenolate Insulin NPH/Regular aspart detemir Opioids, all formulations butorphanol fentanyl HYDROmorphone meperidine Pregnancy category X drugs atorvastatin bosentan estazolam Pediatric liquid medications that require measurement Individual Drugs carbamazepine chloral hydrate liquid (for sedation of children) heparin (unfractionated and low-molecular weight) metformin methotrexate (non-oncologic use) midazolam liquid (for sedation of children) propylthiouracil warfarin Combination products such as: Combivir Atripla Epzicom Kaletra mercaptopurine methotrexate procarbazine temozolomide glyburide repaglinide pimecrolimus sirolimus tacrolimus glargine glulisine lispro methadone morphine opium tincture oxycodone ISOtretinoin simvastatin temazepam Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

35 Appendix B: Documented Rates Documented Rates of Errors and At-Risk Behaviors The HAMMERS Scorecards require users to estimate the frequency of certain events. Some of the needed information can be obtained from operational data (e.g., how often technicians enter prescriptions into the computer, how often prescriptions are received via fax, what percent of a specific drug is filled via automation). The remaining information regarding how often specific components of the dispensing system fail, how often OPPORTUNITIES TO CAPTURE ERRORS miss the mark, and how often staff make unsafe behavioral choices (AT-RISK BEHAVIORS) can only be obtained from estimates provided by pharmacists and PHARMACY ASSOCIATES who possess first-hand knowledge of their dispensing process. Most healthcare practitioners do not have actual rate data regarding failures in the components of the dispensing process. At best, pharmacy data collection systems only capture a small fraction of adverse outcomes, with the rate of intermediate failures relatively unknown. Therefore, to use the HAMMERS tool effectively, pharmacists and PHARMACY ASSOCIATES should reference the research data compiled in Tables 1-5 (pages 35-40), and use this information as a realistic starting point, adjusting the rates up or down according to the pharmacy s unique experiences. Table 1 provides HUMAN ERROR rates, which become progressively higher as working conditions worsen. Examples from community pharmacies are also provided to better explain the error type. Table 2 includes study findings related to outpatient prescribing errors. When multiple studies are cited, a range and rough estimate of an average are provided as scores that correspond to answer choices in the HAMMERS tool. Table 3 displays study findings related to community pharmacy dispensing errors, including rates for very specific types of errors when available (e.g., data entry errors involving the wrong strength). Table 4 provides evidence regarding missed OPPORTUNITIES TO CAPTURE ERRORS during the dispensing process. Table 5 includes studies that document the frequency of some common AT-RISK BEHAVIORS. When multiple studies are cited in Tables 2-5, a range and rough average are provided as scores that correspond to answer choices in the HAMMERS tool. Using the data provided in the Tables as a starting point, judgments about pharmacy specific failure rates, AT-RISK BEHAVIOR frequencies, and missed OPPORTUNITIES TO CAPTURE ERRORS should factor in current PERFORMANCE SHAPING FACTORS that have a positive or negative influence on work. For example, staff training can influence performance either positively (e.g., when training emphasizes the appropriate learned responses) or negatively (e.g., when training is absent). Illegible handwritten prescriptions, look-alike product names, and complex tasks are examples of negative PERFORMANCE SHAPING FACTORS. Additional PERFORMANCE SHAPING FACTORS appear in Table 1 in Appendix D. The number of PERFORMANCE SHAPING FACTORS identified and their degree of influence will help HAMMERS users adjust the published rates upward or downward to reflect their own pharmacy. Estimates derived in this manner are often more accurate than rates predicted by senior management. 1 Perhaps the greatest value of the HAMMERS Scorecards lies less in the estimates of potential adverse events and more in the prioritization of the process steps, behavioral choices, and sequence of events that most often contribute to an adverse outcome. Its advantages over current risk assessment methods, its ability to forecast combinations of risk leading to preventable adverse drug events, and its use in facilitating implementation of effective interventions to reduce medication errors are substantial. Even with uncertain estimates, HAMMERS can lead to learning and improvements not possible with other risk assessment processes. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

36 Appendix B: Documented Rates Documented Rates of Errors and At-Risk Behaviors continued Table 1. HUMAN ERROR Rates Description of Error Probabilities Error Probability Error Rate (%) Examples of Task in Community Pharmacy High Probability of Error Unfamiliar task performed at speed with no idea of likely consequences % New PHARMACY ASSOCIATE entering prescription data at speed to maintain a pharmacy-imposed quota Failed task involving high stress levels % Performing product verification of stacked baskets (backlog) Failed inspection/verification of tasks with moderate stress 2, % Performing data entry verification while distracted by customers interrupting with questions Failed complex task requiring high level of comprehension and skill % Compounding a product that requires special handling Failed task involving complex math computation % Calculating mg/kg dose for infant Failed task conducted in the first 30 minutes of an emergency % Treating a patient with an allergic reaction to a flu vaccine Failure to detect an error after it has happened % Entering a prescription and unable to detect own error when double-checking Fairly simple task performed rapidly or given scant attention % Entering days supply for ml but selecting directions in teaspoonfuls instead Moderate Probability of Error Misidentify/misdiagnose given like symptoms/appearance % Selecting the wrong product from the shelf due to a look-alike label or drug name Failure to select ambiguously labeled control/package % Selecting the wrong strength of a product with a confusing label Failure to perform a check correctly % Missing a computer entry error during verification Wrong conclusion drawn with competing/unclear information % Believing one product is the same as another due to label similarities and confusing dose expressions Failed task with cognitive or task complexity % Attempting to determine the appropriateness and safety of a powerful opioid prescribed for pain Failure to act correctly after the first few hours in a high stress situation % Starting a shift after learning about a serious prescription error from the day before Symptoms noticed, but wrong interpretation % Misreading patient s hesitancy to ask questions as shyness rather than a language barrier Failed task related to values/units/scales/indicators % Missing signs of narcotic abuse with patient with early refills Failed task related to selection of items from among groups of items % Selecting the wrong dosage form among mixed dosage forms listed on an order-entry screen Failed routine, highly practiced, rapid task involving a relatively % Misreading a barcode scan during restocking of a robotic device low level of skill 2 General mental slip without knowledge deficit % Selecting the wrong patient profile when entering a prescription into the computer Failed task related to known hazards/damage % Dispensing a familiar HIGH-ALERT MEDICATION in the wrong dose Failed communication among workers % Mishearing a prescription via telephone Failed task involving both diagnosis and action % Misunderstanding a drug interaction alert as insignificant and failing to contact the prescriber or question the patient Failed diagnosis task % Dispensing a drug prescribed for a patient with a known and documented allergy Error in a routine operation where care is required % Selecting the wrong bag for patient at pick-up Set a switch in wrong position % Entering the wrong allergy code Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

37 Appendix B: Documented Rates Documented Rates of Human Error and At-Risk Behaviors continued Table 1. HUMAN ERROR Rates continued Description of Error Probabilities Error Probability Error Rate (%) Examples of Task in Community Pharmacy Low Probability of Error Procedural omission % Forgetting to reconstitute an antibiotic prior to dispensing Errors during READBACK % Misstating the transcribed dose when reading back a prescription to the physician Counting/volume errors % Miscounting the number of tablets when filling a prescription Selection of the wrong control/package (well-labeled) % Selecting the wrong drug from the shelf despite the presence of a shelf talker Completely familiar, well-designed, highly practiced, routine task occurring several times per hour, performed to highest possible standards by a highly motivated, highly trained and experienced person, totally aware of implications of failure, with time to correct potential error, but without the benefit of significant job aids 2 Lowest Limits of Human Error % (4 errors per 10,000) Human-performance limit: single person working alone % (1 error per 10,000) Human-performance limit: team of people performing a welldesigned % task 3 (1 error per 100,000) Table 2. Incidence of Outpatient Prescribing Errors Error Type Study Error Rate All Error Types Combined Nanji KC, et al (ELECTRONIC PRESCRIPTIONS) Gandhi, et al (adults) Kozer E, et al (pediatric patients) Pharmacist (who enjoys patient counseling and engages in the activity several times an hour) providing the patient with the wrong information regarding well-known side effects of a drug with which the pharmacist is familiar Highly trained, well rested, motivated pharmacy technician or pharmacist working under the most ideal conditions entering the wrong dose into the computer Team from a pharmacy chain developing erroneous dosing guidelines for a new drug being added to the pharmacy computer system 11.7% (5.1% to 37.5%, depending on prescribing system) 7.6% (total errors) 4.3% (errors with ELECTRONIC PRESCRIPTIONS) 11.0% (errors with handwritten prescriptions) 10% Corresponding Scores for HAMMERS ++ Range 4-35% Average 10% Buurma H, et al % (prescriptions that required clarification) Telephone Prescription Errors Camp SC, et al % Range 10-15% Wrong Dose Wrong Frequency Wrong Route Two or More Prescriptions with Drug-Drug Interactions Gandhi, et al (adults) 4.1% (dose) 1.4% (frequency) 1.0% (route) Range 1-4% Average 2% Solberg LI, et al % to 6.7% Range 5-10% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

38 Appendix B: Documented Rates Documented Rates of Human Error and At-Risk Behaviors continued Table 2. Incidence of Outpatient Prescribing Errors continued Error Type Study Error Rate Two or More Prescriptions with Drug-Disease Interactions Dosing Errors with Pediatric Medications Zhan C, et al (elderly patients) McPhillips HA, et al (pediatric patients) 2.58% 15% (all pediatric patients) 33% (children weighing less than 35 kg) Corresponding Scores for HAMMERS ++ 3% Range 15-35% Average 25% Overdoses of Analgesics Underdoses of Antiepileptics McPhillips HA, et al (pediatric patients) 15% (analgesics) 20% (antiepileptics) Range 15-20% ++Ranges and averages are estimates, rounded to available answer choices in Scorecards. The estimates were roughly derived from published rates in the literature. Ranges and averages are intended to help guide Scorecard use only. Table 3. Incidence of Community Pharmacy Dispensing Errors Error Type Study Error Rate Overall Error Rates All Error Types Friesner DL, et al % Combined 1.3% Remote Telepharmacies 0.8% Traditional Community Pharmacies Franklin BD, et al (Community pharmacies without barcoding technology) 3.32% (total undetected errors) 1.7% (content error) 1.6% (label error) Flynn EA, et al % and 1.7% (without automated dispensing system)* 1.8% and 1.9% (with automated dispensing system)** Flynn EA, et al % Range: 0 to 12.8% Grasha AF % (includes detected and corrected errors) Errors with New Prescriptions Flynn EA, et al % and 3.9% (without automated dispensing system)* 4.7% and 4.6% (with automated dispensing system)** Errors with Refills Flynn EA, et al % and 0.4% (without automated dispensing system)* 0.3% and 0.1% (with automated dispensing system)** Errors By Process Step Errors During Receipt of Prescription Friesner DL, et al % (traditional pharmacies and (prescriptions with missing information) telepharmacies) Corresponding Scores for HAMMERS ++ Range 1-7% Average 3% Range 4-5% Average 5% Range 0.1-1% Average 0.5% 0.1% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

39 Appendix B: Documented Rates Documented Rates of Human Error and At-Risk Behaviors continued Table 3. Incidence of Community Pharmacy Dispensing Errors continued Error Type Study Error Rate Data Entry Errors Friesner DL, et al (traditional pharmacies and telepharmacies) Flynn EA, et al Grasha AF Cohen MR, et al Errors During the Filling Process Friesner DL, et al (traditional pharmacies and telepharmacies) Error When Bagging the Filled Prescription Errors By Process Step (continued) 0.67% (all types) 0.11% (wrong drug) 0.09% (wrong strength) 0.06% (wrong patient) 0.18% (wrong directions) 0.12% (others) 0.90% (wrong directions) 2.7% (wrong directions) 1.0% (wrong patient) (includes detected and corrected errors) 0.5% (wrong patient) 0.23% (all types) 0.07% (wrong drug) 0.07% (wrong strength) 0.01% (wrong label placed on container) 0.05% (wrong quantity) 0.03% (others) Corresponding Scores for HAMMERS ++ Range 0.1-3% Average All Types 0.7% Specific Type 0.7% Range % Average All Types 0.3% Specific Types 0.1% Cohen MR, et al % Range Ashcroft DM % 0.1-3% Grasha AF % (includes detected and corrected errors) Select Wrong Bag From Will-Call Area Cohen MR, et al % 0.3% General Categories of Errors Average 1% General Labeling Errors Ashcroft DM % Range % Wrong Drug (combines data entry and drug container selection errors) Flynn EA, et al % Friesner DL, et al % (traditional pharmacies and telepharmacies) Flynn EA, et al % Average 0.1% Range 0.1-3% Average 1% Grasha AF % (includes detected and corrected errors) Wrong Strength Friesner DL, et al % Range (traditional pharmacies and 0.1-3% telepharmacies) Flynn EA, et al % Average Grasha AF % (includes detected and corrected errors) 1% Wrong Form of Drug Flynn EA, et al % 0.1% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

40 Appendix B: Documented Rates Documented Rates of Human Error and At-Risk Behaviors continued Table 3. Incidence of Community Pharmacy Dispensing Errors continued Error Type Study Error Rate Corresponding Scores for HAMMERS ++ Special Conditions Drug-Drug Interactions Missed Malone DC, et al Range: 0.2% to 5% Range 0.1-5% Error rate with one or more interruptions or distractions per hour Flynn EA, et al % to 6.65% (average of 6 interruptions and distractions per hour) ++Ranges and averages are estimates, rounded to available answer choices in Scorecards. The estimates were roughly derived from published rates in the literature. Ranges and averages are intended to help guide Scorecard use only. * The automated dispensing system included a semiautomated machine for dispensing the top 200 oral solids, barcode aided medication verification, and a color photograph of the medication to compare with the vial contents. **The primary source of errors after installation of an automated dispensing system was a deliberate overriding the automated system. Table 4. Incidence of Missed OPPORTUNITIES TO CAPTURE ERRORS Range 5-10% Opportunities to Detect Errors Study Error Rate Corresponding Scores for HAMMERS ++ All Pharmacy Inspections Flynn EA, et al % of all errors missed 50% Data Entry Verification Flynn EA, et al % of errors missed 70% Final Product Verification Flynn EA, et al % of errors missed 30% Typical Dispensing Tasks Grasha AF % of self-made errors missed while carrying out typical dispensing tasks 15% ++Ranges and averages are estimates, rounded to available answer choices in Scorecards. The estimates were roughly derived from published rates in the literature. Ranges and averages are intended to help guide Scorecard use only. Table 5. Incidence of AT-RISK BEHAVIORS At-Risk Behavior Study Incidence Rate General Procedural Violations Fogarty & McKeon % of errors due to procedural violations Gandhi TK, et al Rule violations when prescribing medications: 9.5% (ELECTRONIC PRESCRIPTIONS) 12.1% (handwritten prescriptions) 10.8% (combined) Failure to Counsel Patients Kimberlin CL, et al % not counseled 78% not provided with risk information (adverse effects or precautions) Flynn EA, et al % not counseled 77% not counseled if you exclude consumers who initiated the counseling session Nau & Erickson Svarstad BL, et al % not counseled (patient reported) Range of 6% to 60%, with lowest rates of not counseling patients in states with intensive counseling regulations Corresponding Scores for HAMMERS ++ Range 10-15% Range 6-80% Average 60% Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

41 Appendix B: Documented Rates Documented Rates of Human Error and At-Risk Behaviors continued Table 5. Incidence of AT-RISK BEHAVIORS continued At-Risk Behavior Study Incidence Rate Bypass Computer Alerts Due to ALERT FATIGUE (After receiving too many alerts, user begins to ignore and/or override the alerts) Bypass Barcode Scanning/Use of a CHEAT SHEET/Override Alerts Paterno MD, et al Isacc T, et al Van der Sijs H, et al Koppel R, et al (hospital study) 66% (severe drug interactions in nontiered interaction system) 0% (severe drug interactions, in tiered interaction system) 77% (allergies) 91% (overall) 98% (drug interactions) 89% (overdoses) 80% (DUPLICATE THERAPY) 1 override per 5 prescriptions Medium-level drug alerts bypassed more often than low-level or high-level alerts 10.3% override of system alerts 4.2% bypass of technology (not used) Corresponding Scores for HAMMERS ++ Range 0-100% Average All Types 90% Specific Types 80% (excluding 0% from first study) Bypass Technology 5% Bypass Alerts 10% Failure to Ask Customer for Two Cohen MR, et al % Unique Identifiers for Identification 20% at the Point of Sale ++Ranges and averages are estimates, rounded to available answer choices in Scorecards. The estimates were roughly derived from published rates in the literature. Ranges and averages are intended to help guide Scorecard use only. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

42 Appendix C: Risk-Reduction Strategies Preventing and Detecting Prescribing Errors Prevention Strategies 1 Provide prescribers with ongoing education about medication errors and their prevention. Provide feedback to the community of prescribers (e.g., newsletter for physicians in the area, presentations at local medical association meetings) regarding common and potentially serious prescribing errors detected while filling prescriptions, including errors made with ELECTRONIC PRESCRIPTIONS (e.g., ambiguous patient directions that contain both a sig and special instructions that may conflict). Include external information about prescribing errors available in journals and newsletters (e.g., ISMP and FDA publications). Accept and encourage ELECTRONIC PRESCRIPTIONS generated from computer programs that offer prescribers clinically relevant decision support and safety features. Detection Strategies 1 Obtain and consider essential information about the patient when dispensing medications. Encourage prescribers to include a diagnosis or indication on all prescriptions to help ascertain if the medication prescribed is appropriate for the patient. In all NEW and EXISTING PATIENT profiles in the pharmacy computer, include basic information about the patient s date of birth, comorbid and/or chronic conditions, and allergies, which is updated annually (via a questionnaire, for example). Make patient allergies a required field in the pharmacy computer so that prescriptions cannot be entered until allergies have been documented and properly coded to allow for computer screening. When a prescription is brought into the pharmacy, verify with the patient or caregiver any clinical information about the patient that is necessary to confirm the appropriateness of the medication and dose (e.g., allergies, weight, opioid tolerance, indication for drug). Highlight the date of birth for children less than 6 years old in the computer system and on the prescription hard copy as a reminder to verify doses and dosage forms. Incorporate special prompts in the pharmacy computer systems for selected HIGH-ALERT MEDICATIONS to obtain or verify critical information about the patient necessary to confirm the appropriateness of prescribed medications, doses, dosage forms, and directions for use (e.g., past opioid use for patients receiving transdermal fentanyl greater than 25 mcg/hour, concentrated morphine solutions, long-acting opioids). Ascertain the clinical purpose of each medication being dispensed to assure that the prescribed therapy is appropriate for the patient s condition. 2 Improve access to information about the prescribed drug. Provide easy, seamless access from the pharmacy computer to current electronic drug information (e.g., Micromedex, Clinical Pharmacology) and the Internet for pharmacists to search for information about disease processes, drug dosing and availability, unusual uses of drugs, and other drug-related information. At least quarterly, receive and load in the pharmacy computer updated drug information from a drug information vendor. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

43 Appendix C: Risk-Reduction Strategies Preventing and Detecting Prescribing Errors continued Ensure that the computer system automatically performs adult and pediatric dose range checks and warns pharmacy staff about potential overdoses and underdoses of targeted HIGH-ALERT MEDICATIONS or narrow therapeutic index medications. Ensure that the computer system warns pharmacy staff about clinically significant drug interactions, dose-related interactions, DUPLICATE THERAPY, and allergies. For high-severity alerts with selected HIGH-ALERT MEDICATIONS (e.g., methotrexate daily instead of weekly, U-500 insulin instead of U-100 insulin), configure computer alerts as a hard stop that requires verification and meaningful action (more than a key stroke) by a pharmacist to proceed. 3 Safeguard methods of communicating, clarifying, and receiving prescriptions. If a prescription is called into the pharmacy, write the information legibly on a telephone prescription blank with prompts for date of birth, allergy information, weight in kg for children less than 6 years old, and drug indication. If speaking directly with the prescriber or agent, request this information and confirm the prescription by reading it back to the prescriber or agent. Follow up directly with prescribers regarding concerns about the safety of a prescription. Establish a clear, written policy to guide the process that should be followed to resolve conflicts easily and effectively when discrepancies about the safety of a prescription arise between prescribers and pharmacists. Persist with follow up if an explanation provided does not resolve the initial concerns. Establish a formal policy to assess and clarify any unusual drugs, doses, and unusual uses of drugs before dispensing the medication. 4 Create a physical environment that supports safety. Make any changes necessary in the workflow and environment to support consistent, cognitive data entry verification, drug utilization review, and final product verification in which the checking pharmacist is relatively free from distractions and competing time demands. Examine the prescription volume periodically to determine appropriate staffing levels, even during peak times. 5 Provide staff training and education. Routinely provide information to all pharmacy staff (including PHARMACY ASSOCIATES) about common prescribing errors and strategies to detect and correct them. 6 Employ redundancies that support INDEPENDENT CHECKS and/or automated processes to detect and correct errors. Configure the pharmacy computer to require action by the operator during drug utilization review to confirm that all aspects of the prescription have been reviewed (e.g., prompts for each aspect verified, checklist that requires keystroke action). Should a computer alert appear during PHARMACY ASSOCIATE data entry, ensure that pharmacists are aware or allow alerts to appear only upon pharmacist review. Printing bypassed alerts on labels along with product labels is an option to consider. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

44 Appendix C: Risk-Reduction Strategies Preventing and Detecting Prescribing Errors continued Protect against ALERT FATIGUE through fewer, more appropriate alerts that need consideration by pharmacists before filling the prescriptions. Optimize the sensitivity of alert systems by carefully selecting alert severity levels and allowing only the most significant alerts to appear on the screen during data entry. Encourage the reporting of invalid or insignificant warnings so they can be altered or removed from the computer system. For selected patient groups (e.g., pediatric patients, patients receiving drugs dosed according to age, weight, or body surface area), conduct a manual check of the prescriber s calculated dose before preparing and dispensing the medications. Provide the prescription (e.g., scanned image, hard copy) for a pharmacist to reference while conducting data entry verification, drug utilization review, and the final product verification. On a daily basis, compare the previous day s prescription orders to hard copies (or scanned image) of prescriptions to verify accuracy. Follow up with pharmacy staff and patients (and prescribers, if necessary) regarding any discrepancies. Run reports of bypassed alerts for a pharmacist to review daily, and follow up with prescribers and patients who might have received an erroneous prescription. Require pharmacists to periodically perform quality control checks by reviewing completed prescriptions in the will-call area; examining typed labels, computer entries, and location of stock bottles replaced in inventory; and other forms of random checks that promote detection of errors. 7 Include patients as active partners in care through education. Increase the frequency of patient counseling at the point of sale, particularly for HIGH-ALERT MEDICATIONS (for free patient handouts on numerous HIGH-ALERT MEDICATIONS, go to: Link the need for patient counseling with the cash register so that the transaction cannot be completed until the pharmacist provides counseling. For selected HIGH-ALERT MEDICATIONS that are delivered to patients, counsel patients by phone when possible. During the patient counseling sessions: Tell the patient the name of the drug (generic and brand if applicable) being dispensed and its purpose. Open the vial or container to show the patient the actual product. For certain HIGH-ALERT MEDICATIONS, require patients to come into the pharmacy to pick up a prescription rather than using the drive-through. Specifically ask patients if the information provided matches their expectations based on their conversations with the prescriber and prior drug therapy. Verify the medication if the patient suggests that the product looks different than expected or expresses any uncertainty with the information provided. Obtain any necessary information from the patient to confirm that the prescribed medication and dose is appropriate for the patient s condition (e.g., prior opioid tolerance). Provide written medication information at an appropriate reading level to patients, and encourage them to read it. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

45 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Patient) Prevention Strategies 1 Obtain and consider essential information about the patient when dispensing medications. If a prescription is brought into the pharmacy, verify the patient information on the prescription with the patient or caregiver, (e.g., spelling of patient s name, address, date of birth). For visually or hearing impaired patients, provide alternate means of communication so patient information may be obtained. When necessary, clarify with the prescriber or agent all illegible or poorly legible prescriptions or prescriptions without necessary information (e.g., date of birth). Incorporate an interactive voice response (IVR) system with prompts to require the physician or agent to spell all names and numbers (prescriber, patient, drug, strength) when leaving a spoken prescription order. Be sure date of birth is noted on every patient s profile and prescription. Highlight the date of birth for children less than 6 years old if possible in the pharmacy computer system. Ascertain the clinical purpose of each medication being dispensed to assure that the prescribed therapy is appropriate for the patient s condition. 2 Safeguard methods of communicating and receiving prescriptions. If a prescription is called into the pharmacy, write the information legibly on a telephone prescription blank that has prompts for all the necessary information (e.g., patient s full name, address, date of birth). If speaking directly with the prescriber or agent, ask the caller to spell the patient s name and provide a date of birth. Confirm the prescription by reading it back to the prescriber or agent, including the patient s name and a second unique identifier. Also verify the indication for the medication. Reduce the need for pharmacy data entry by accepting and encouraging ELECTRONIC PRESCRIPTIONS. 3 Create a physical environment that supports safety. Create an environment that facilitates accurate data entry (e.g., a device to hold prescriptions near computer screen at eye level; scanned, re-sizeable prescription image on computer screen; magnification or zoom capability; adjustable lighting). Scan paper prescriptions so they are displayed on the pharmacy computer screen at eye level. Enter one patient s prescription(s) at a time, with no other patient s prescription(s) nearby or within the line of vision. Utilize a basket or other system to keep patient orders separate from one another during the dispensing process. Encourage patients to use fax, voic , or to request refills to minimize staff interruptions during data entry and data entry verification. Utilize a remote order entry service to help reduce workload at peak hours in the pharmacy. Maintain a proper balance between operational objectives (e.g., cost-containment, productivity, turn-around time, profit and loss) and safe pharmaceutical care responsibilities. Avoid prescription quotas or required dispensing rates, Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

46 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Patients) continued unrealistic promises to fill prescriptions within a certain time, and pharmacy staff or store penalties for delayed prescription filling caused by compliance with safety standards. Make any changes necessary in the workflow and environment to support consistent, cognitive data entry verification in which the checking pharmacist is relatively free from distractions and competing time demands. Isolate order entry tasks whenever possible. Examine the prescription volume periodically to determine appropriate staffing levels, even during peak times. 4 Minimize the risk of data entry errors with patients who have similar or the same names and/or addresses. For EXISTING PATIENTS, initially select the patient s profile by entering the patient s date of birth, and then verify the patient s identity by comparing two unique identifiers on the prescription and pharmacy profile. For NEW PATIENTS, verify patient identity by comparing two unique identifiers on the prescription and the newly created pharmacy profile before completing data entry of the first prescription. Implement policies and procedures or system enhancements to insure one profile per patient exists in the system. Be cognizant of name suffixes (Jr.), first/last name interchanges, and incorrect assignment of first and last name (e.g., James John, Ikembe Fintumbo). If applicable, utilize the patient s legal name or name on the insurance card. 5 Provide staff training and education. Allow PHARMACY ASSOCIATES to perform data entry only if they have documented training related to the order entry process. Detection Strategies 1 Employ redundancies that support INDEPENDENT CHECKS and/or automated processes to detect and correct errors. When possible, require a second pharmacist to verify the data entry of any prescriptions initially entered by a pharmacist. Configure the pharmacy computer to require action by the operator during data entry verification to confirm that all aspects of the prescription have been reviewed (e.g., prompts for each aspect verified, checklist that requires keystroke action). Provide the prescription (e.g., scanned image, hard copy) for a pharmacist to reference while conducting data entry verification, drug utilization review, and the final product verification. On a daily basis, compare the previous day s prescription orders to hard copies (or scanned image) of prescriptions to verify accuracy. Follow up with pharmacy staff and patients (and prescribers, if necessary) regarding any discrepancies. Run reports of bypassed alerts for a pharmacist to review daily, and follow up with prescribers and patients who might have received an erroneous prescription. Require pharmacists to periodically perform quality control checks by reviewing completed prescriptions in the will-call area; examining typed labels, computer entries, and location of stock bottles replaced in inventory; and Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

47 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Patients) continued other forms of random checks that promote detection of errors. 2 Include patients as active partners in care through education. Increase the frequency of patient counseling at the point of sale, particularly for HIGH-ALERT MEDICATIONS (for free patient handouts on numerous HIGH-ALERT MEDICATIONS, go to: Link the need for patient counseling with the cash register so that the transaction cannot be completed until the pharmacist provides counseling. For selected HIGH-ALERT MEDICATIONS that are delivered to patients, counsel patients by phone when possible. During the patient counseling sessions: Tell the patient the name of the drug (generic and brand if applicable) being dispensed and its purpose. Open the vial or container to show the patient the actual product. For certain HIGH-ALERT MEDICATIONS, require patients to come into the pharmacy to pick up a prescription rather than using the drive-through. Specifically ask patients if the information provided matches their expectations based on their conversations with the prescriber and prior drug therapy. Verify the medication if the patient suggests that the product looks different than expected or expresses any uncertainty with the information provided. Obtain any necessary information from the patient to confirm that the prescribed medication and dose is appropriate for the patient s condition (e.g., prior opioid tolerance). Provide written medication information at an appropriate reading level to patients, and encourage them to read it. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

48 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Drug, Dose, Directions) Prevention Strategies 1 Obtain and consider essential information about the patient when dispensing medications. Encourage prescribers to include a diagnosis or indication on all prescriptions to help distinguish medication names that look alike. In all NEW and EXISTING PATIENT profiles in the pharmacy computer, include basic information about the patient s date of birth, comorbid and/or chronic conditions, and allergies, which is updated annually (via a questionnaire, for example). Ascertain the clinical purpose of each medication being dispensed to assure that the prescribed therapy is appropriate for the patient s condition. 2 Obtain accurate information about the prescribed drug. Clarify with the prescriber all illegible or poorly legible prescriptions, or prescriptions that do not include important information (e.g., metric dose of liquid medications). Incorporate an interactive voice response (IVR) system with prompts to require the physician or agent to spell all names and numbers (prescriber, patient, drug, strength) when leaving a spoken prescription order. 3 Improve access to information about the prescribed drug. Ensure that the computer system automatically performs adult and pediatric dose range checks and warns pharmacy staff about potential overdoses and underdoses of targeted HIGH-ALERT MEDICATIONS or narrow therapeutic index medications. Ensure that the computer system warns pharmacy staff about clinically significant drug interactions, dose-related interactions, DUPLICATE THERAPY, and allergies. For high-severity alerts with selected HIGH-ALERT MEDICATIONS (e.g., methotrexate daily instead of weekly, U-500 insulin instead of U-100 insulin), configure computer alerts as a hard stop that requires verification and meaningful action (more than a key stroke) by a pharmacist to proceed. At least quarterly, receive and load in the pharmacy computer updated drug information from a drug information vendor. Consult the original prescription or prescription image for refills. 4 Safeguard methods of communicating and receiving prescriptions. Do not accept oral (telephone) prescriptions for chemotherapy. Use only the metric system (ml) for dose volumes and directions for use on labels forbid the use of household measures such as teaspoonful and tablespoonful as well as the abbreviations tsp, tbsp, and tbs. Reduce the need for pharmacy data entry by accepting and encouraging ELECTRONIC PRESCRIPTIONS. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

49 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Drug, Dose, Directions) continued 5 Create a physical environment that supports safety. Create an environment that facilitates accurate data entry (e.g., a device to hold prescriptions near computer screen at eye level; scanned, re-sizeable prescription image on computer screen; magnification or zoom capability; adjustable lighting). Scan paper prescriptions so they are displayed on the pharmacy computer screen at eye level. Enter one patient s prescription(s) at a time, with no other patient s prescription(s) nearby or within the line of vision. Utilize a basket or other system to keep patient orders separate from one another during the dispensing process. Encourage patients to use fax, voic , or to request refills to minimize staff interruptions during data entry and data entry verification. Utilize a remote order entry service to help reduce workload at peak hours in the pharmacy. Maintain a proper balance between operational objectives (e.g., cost-containment, productivity, turn-around time, profit and loss) and safe pharmaceutical care responsibilities. Avoid prescription quotas or required dispensing rates, unrealistic promises to fill prescriptions within a certain time, and pharmacy staff or store penalties for delayed prescription filling caused by compliance with safety standards. Make any changes necessary in the workflow and environment to support consistent, cognitive data entry verification in which the checking pharmacist is relatively free from distractions and competing time demands. Isolate order entry tasks whenever possible. Examine the prescription volume periodically to determine appropriate staffing levels, even during peak times. 6 Minimize the risk of data entry errors with drugs that have similar names, multiple concentrations/forms, error-prone dosing, unusual uses, or associated abbreviations. To avoid product selection errors where mnemonics are allowed and used, program computer entry screens to display the specific brand names along with the generic names whenever a stem or mnemonic is entered. Prevent look-alike drug names from appearing in alphabetical order on the same screen, and/or differentiate lookalike drug names from one another through the use of tall man letters (HumALOG vs. HumuLIN, NovoLOG vs. NovoLIN) or other font enhancements. View the ISMP-FDA list of drug name pairs and standard tall man letters ( for examples. Clarify the presentation of certain HIGH-ALERT MEDICATIONS to reduce the risk of selecting the wrong drug or concentration from the screen during data entry (e.g., for U-500 insulin and other concentrated drugs, list concentrated immediately following the drug name and preceding the strength; for insulin product mixtures, emphasize the word mixture or mix ; for pediatric strength medications, include pediatric as part of the drug name). Build special alerts in the computer system to warn pharmacy staff about very serious, error-prone data entry situations (e.g., potentially dangerous look-alike drug names, unusual directions for use), and print the warning with the product label or communicate it by another means so the risk is known to the pharmacist verifying data entry. Allow pharmacy staff to enter, or request the addition of, these targeted drug warnings into the pharmacy computer system. View the ISMP List of Confused Drug Names ( for examples of product names that could lead to mix-ups. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

50 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Drug, Dose, Directions) continued Avoid abbreviated product names (e.g., AZT, PTU, DTO, MS, MTZ), error-prone abbreviations (e.g., U for units), and unsafe dose expressions (e.g., trailing zeros, naked decimal points) when drug names and related information are displayed on the screen. Prohibit staff from coining abbreviations for drug names or entering sig or speed codes at the store level. 7 Provide staff training and education. Routinely provide information to all pharmacy staff (including PHARMACY ASSOCIATES) about data entry errors, error-prone situations, errors occurring in other pharmacies, and strategies to prevent such errors. Educate all staff, including PHARMACY ASSOCIATES, about new drug products coming to market and/or being stocked in the pharmacy. Allow PHARMACY ASSOCIATES to perform data entry only if they have documented training related to the order entry process and prescription types. Detection Strategies 1 Employ redundancies that support INDEPENDENT CHECKS and/or automated processes to detect and correct errors. When possible, require a second pharmacist to verify the data entry of any prescriptions initially entered by a pharmacist. Configure the pharmacy computer to require action by the operator during data entry verification to confirm that all aspects of the prescription have been reviewed (e.g., prompts for each aspect verified, checklist that requires keystroke action). Should a computer alert appear during PHARMACY ASSOCIATE data entry, ensure that pharmacists are aware or allow alerts to appear only upon pharmacist review. Printing bypassed alerts on labels along with product labels is an option to consider. Protect against ALERT FATIGUE through fewer, more appropriate alerts that need consideration by pharmacists before filling the prescriptions. Optimize the sensitivity of alert systems by carefully selecting alert severity levels and allowing only the most significant alerts to appear on the screen during data entry. Encourage the reporting of invalid or insignificant warnings so they can be altered or removed from the computer system. Provide the prescription (e.g., scanned image, hard copy) for a pharmacist to reference while conducting data entry verification, drug utilization review, and the final product verification. On a daily basis, compare the previous day s prescription orders to hard copies (or scanned images) of prescriptions to verify accuracy. Follow up with pharmacy staff, prescribers, and patients regarding any discrepancies. Run reports of bypassed alerts for a pharmacist to review daily, and follow up with prescribers and patients who might have received an erroneous prescription. Routinely run reports of system speed codes in use and review for potential risk (e.g., codes that can be inter- Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

51 Appendix C: Risk-Reduction Strategies Preventing and Detecting Data Entry Errors (Wrong Drug, Dose, Directions) continued changed and linked to unintended products). Use the ISMP List of Confused Drug Names ( confuseddrugnames.pdf) for examples of look-alike drug product names that could lead to mix-ups when using speed codes. Require pharmacists to periodically perform quality control checks by reviewing completed prescriptions in the will-call area; examining typed labels, computer entries, and location of stock bottles replaced in inventory; and other forms of random checks that promote detection of errors. 2 Include patients as active partners in care through education. Increase the frequency of patient counseling at the point of sale, particularly for HIGH-ALERT MEDICATIONS (for free patient handouts on various HIGH-ALERT MEDICATIONS, go to: Link the need for patient counseling with the cash register so that the transaction cannot be completed until the pharmacist provides counseling. For selected HIGH-ALERT MEDICATIONS that are delivered to patients, counsel patients by phone when possible. During the patient counseling sessions: Tell the patient the name of the drug (generic and brand if applicable) being dispensed and its purpose. Open the vial or container to show the patient the actual product. For certain HIGH-ALERT MEDICATIONS, ask patients to come into the pharmacy to pick up a prescription rather than using the drive-through. Specifically ask patients if the information provided matches their expectations based on their conversations with the prescriber and prior drug therapy. Verify the medication if the patient suggests that the product looks different than expected or expresses any uncertainty with the information provided. Obtain any necessary information from the patient to confirm that the prescribed medication and dose is appropriate for the patient s condition (e.g., prior opioid tolerance). Provide written medication information at an appropriate reading level to patients, and encourage them to read it. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

52 Appendix C: Risk-Reduction Strategies Preventing and Detecting Drug Container Selection Errors Prevention Strategies 1 Store, dispense, and return medications to stock in a manner that reduces the risk of errors. Provide adequate space to safely organize and separate the storage of drugs and drug supplies on shelves and in cabinets, narcotic cases, and refrigerators. Utilize dividers on crowded stock shelves, in narcotic cabinets, and in refrigerators as needed. Organize drug inventory according to frequency and volume of use to separate less frequently used doses from those dispensed more frequently (e.g., two dosage strengths of a product are separated if one strength is used predominantly). Always stock products with the manufacturer s label showing. Never place the product face down due to crowding on shelves. When stocking shelves, ensure that the wholesaler s price label does not interfere with critical information on a product label. Never place stickers or cross-out lines on any part of a stock bottle where it could obliterate key information. Always label all stock, including return-to-stock vials, with the drug name, strength, expiration date, and national drug code (NDC) number or barcode. Avoid abbreviated product names (e.g., AZT, PTU, DTO) on shelf labels. Safeguard access to targeted HIGH-ALERT MEDICATIONS such as anticoagulants, oral hypoglycemic drugs, and other problem products, through constraints (e.g., placing drugs in locked areas, removal from fast mover areas where it might be grabbed accidentally). Limit bulk chemicals used for pharmacy compounding and non-drug supplies (e.g., alcohol) to products used regularly. Clearly label all containers, and store them separately from all other medications and supplies. Avoid storage that requires staff to reach over their heads or to climb up to retrieve products. If filled prescriptions are not dispensed to patients, return the medications to stock in a consistent manner that reduces the risk of an error. 2 Minimize the risk of product container selection errors with drugs that have similar names, multiple concentrations/forms of the drug, and look-alike packaging. Establish a system to identify product name and packaging similarities or label ambiguity that might lead to mixups or confusion (e.g., follow ISMP and FDA reports; review event and hazard reports; proactively examine the package and label of new drugs added to inventory; work with the drug purchasing group to establish product selection criteria). Store products with known look-alike names or packaging apart from each other and not alphabetically if names are similar or begin with same syllable (e.g., metronidazole and metformin). Use shelf dividers to separate the products, or otherwise clearly differentiate them if they must remain next to each other (e.g., warning labels). Institute a shelf talker or signing program that brings attention to confusing or look-alike names or labels during Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

53 Appendix C: Risk-Reduction Strategies Preventing and Detecting Drug Container Selection Errors continued stocking and retrieval procedures. Call attention to important information on the manufacturer s label that could be missed (e.g., enhancement with pen or marker, auxiliary labels, stickers), particularly for products with look-alike names or packaging, and products that have overlapping strengths, 10-fold differences between strengths, extended and immediate-release formulations (e.g., oxycodone), or a concentrated form of the drug (e.g., oral liquid morphine). When possible, avoid stocking generic manufacturers products that incorporate the same size stock bottles, label colors, and fonts in their complete product line. Never stock any part of a product line with sound- or look-alike drug names in the fast mover section (unless automation is employed). Store different types of insulin and other similar items in separate bins in the refrigerator. When two products exist with dangerously similar labeling/packaging, attempt to purchase one of the products from a different manufacturer. Work with group purchasing organizations as appropriate. Build special alerts in the computer system for the most serious container mix-ups, and print the warning with the product label so the risk of a mix-up is known to those filling the prescription. 3 Create a physical environment that supports safety. Fill one prescription at a time, affix the label to the patient s prescription container, and return the stock bottle or other bulk container to the storage area before filling the next prescription. When filling multiple prescriptions for one patient, separate each by dividers, baskets, or other means to ensure the prescriptions are not mixed up. Ensure adequate space, storage, and lighting (10,000 ft. candles) in medication stock and dispensing areas. Maintain workstations that are free of clutter. Make any changes necessary in the workflow and environment to support a consistent, cognitive final product verification processes in which the checking pharmacist is relatively free from distractions and competing time demands (e.g., physical layout of pharmacy minimizes distractions, areas free of unnecessary chatter). Maintain a proper balance between operational objectives (e.g., cost-containment, productivity, turn-around time, profit and loss) and safe pharmaceutical care responsibilities. Avoid prescription quotas or required dispensing rates, unrealistic promises to fill prescriptions within a certain time, and pharmacy staff or store penalties for delayed prescription filling caused by compliance with safety standards. 4 Improve access to information about the prescribed drug. To guide proper selection, provide a computer image or description on the screen for each drug to show or describe the appearance of the product. Include the purpose of the medication on the patient s prescription label, if provided by the prescriber. When dispensing unit-of-use packaging to patients, avoid placing a pharmacy label on top of pertinent manufacturer s information. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

54 Appendix C: Risk-Reduction Strategies Preventing and Detecting Drug Container Selection Errors continued 5 Provide staff training and education. Provide and train staff on policies and procedures to be implemented when a stock product does not have a barcode or has a barcode that is not readable. Routinely provide information to all pharmacy staff (including PHARMACY ASSOCIATES) about potential look-alike drug names, product selection errors, errors occurring in other pharmacies, and strategies to prevent such errors. Detection Strategies 1 Employ redundancies that support INDEPENDENT CHECKS and/or automated verification processes to detect and correct errors. When appropriate, increase AUTOMATED DISPENSING of products that incorporates robotics and/or barcode scanning verification. Employ barcode scanning technology consistently to verify manual product selection, or at a minimum, establish a system to compare the computer-generated national drug code (NDC) on prescription labels and the NDC on the manufacturer s containers to verify product selection. Scan each product container s barcode when manually filling prescriptions that require medication from more than one container. Eliminate technology workarounds (e.g., use of CHEAT SHEETS, bypassing the scanning process) by uncovering and correcting the environmental-, equipment-, or system-related causes. Fill prescriptions using the original prescription order and the computer-generated drug label together. Compare the label and product with the original prescription before drugs are dispensed to patients. Employ tablet imaging technology so pharmacists can visualize what the product should look like during final product verification. When a patient arrives to pick up a prescription, require a pharmacist to conduct an additional INDEPENDENT CHECK of selected HIGH-ALERT MEDICATION prescriptions when they are pulled from the will-call area. 2 Include patients as active partners in care through education. Increase the frequency of patient counseling at the point of sale, particularly for HIGH-ALERT MEDICATIONS (for free patient handouts on various HIGH-ALERT MEDICATIONS, go to: Link the need for patient counseling with the cash register so that the transaction cannot be completed until the pharmacist provides counseling. For selected HIGH-ALERT MEDICATIONS that are delivered to patients, counsel patients by phone when possible. During the patient counseling sessions: Open the vial or container to show the patient the actual product. For certain HIGH-ALERT MEDICATIONS, require patients to come into the pharmacy to pick up a prescription rather than using the drive-through. Compare the product description on the pharmacy label to the appearance of the actual product. Verify the medication if the patient suggests that the product looks different than expected or expresses any uncertainty. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

55 Appendix C: Risk-Reduction Strategies Preventing and Detecting Point of Sale Errors Prevention Strategies 1 Obtain and consider essential information about the patient when dispensing medications. Include in each patient s profile his or her full name, address, telephone number(s), gender, and birthdate to facilitate proper identification. Increase pharmacy staff compliance with requesting two unique patient identifiers from customers before beginning the sales transaction. Unique patient identifiers include full name and date of birth (year may be excluded). If the patient s date of birth is unknown, address or telephone number may be substituted as deemed appropriate by pharmacy staff. Avoid verifying patient identity by passive agreement (e.g., Your birthdate is October 10, right?). Compare the identifiers provided by customers to information about the patient on the pharmacy computer and prescription receipts. Tie the point of sale system to the pharmacy operating system and require manual entry of the patient s birthdate (month and day) at the point of sale; do not put birthdate on the pharmacy receipt pharmacy staff must ask the patient for the date of birth to enter. Explain to customers who are well known to pharmacy staff why two unique identifiers are required before every sale. Teach patients how to actively participate in proper identification before accepting medication at pick up. Store medications for pick up in clear bag hangers so containers with patients names are visible to staff and patients. 2 Create a physical environment that supports safety. Verify and bag one customer s prescription(s) at a time in a clutter-free area where no other patient s medication(s) are present. Maintain a prescription pick-up/will-call area that is free from clutter and contains enough space to prevent spillage into the next basket or bin. Institute return to stock procedures; physically remove filled prescriptions not picked up or no longer wanted by patients. Maintain separate refrigerators for stock and prepared prescriptions waiting to be dispensed to patients. Detection Strategies 1 Employ redundancies that support INDEPENDENT CHECKS and/or automated processes to detect and correct errors. Open the bag at the point of sale and read the name on each prescription label and receipt to verify that the medication(s) being dispensed are for the intended patient. Never allow a patient to leave pharmacy without examining each container and label. Confirm with the customer how many filled prescriptions are being picked up (if known) for comparison to the actual number of filled prescriptions being dispensed. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

56 Appendix C: Risk-Reduction Strategies Preventing and Detecting Point of Sale Errors continued 2 Include patients as active partners in care through education. Increase the frequency of patient counseling at the point of sale, particularly for HIGH-ALERT MEDICATIONS (for free patient handouts on various HIGH-ALERT MEDICATIONS, go to: Link the need for patient counseling with the cash register so that the transaction cannot be completed until the pharmacist provides counseling. For selected HIGH-ALERT MEDICATIONS that are delivered to patients, counsel patients by phone when possible. During the patient counseling sessions: Tell the patient the name of the drug (generic and brand if applicable) being dispensed and its purpose. Open the vial or container to show the patient the actual product. For certain HIGH-ALERT MEDICATIONS, require patients to come into the pharmacy to pick up the prescription rather than using the drive-through. Specifically ask patients if the information provided matches their expectations based on their conversations with the prescriber and prior drug therapy. Verify the medication if the patient suggests that the product looks different than expected or expresses any uncertainty with the information provided. Provide written medication information at an appropriate reading level to patients, and encourage them to read it. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

57 Appendix D Understanding Errors and At-Risk Behaviors Retail pharmacies dispense an estimated 4 billion prescriptions annually, and projections indicate growing volumes for the foreseeable future. 36 Typical volumes in US urban and suburban pharmacies range from less than a 1,000 to more than 3,000 prescriptions dispensed per week. Four well-designed retrospective studies examining community pharmacy dispensing errors reported a wide range of error rates from %. 20,29,37 Applying the lowest dispensing error rate of 1.7% translates to approximately 4 errors reaching the patient per 250 prescriptions per pharmacy per day. 38 Errors occur when one or more process, mechanical, or behavioral elements in a system fails or deviates from standard, producing an undesirable outcome. In the case of a community pharmacy, perhaps the pharmacy computer system is misprogrammed (process failure), Table 1. Performance Shaping Factors or a pharmacist mistakenly selects the wrong strength of a drug (behavioral (PSFs) 42,43 error), or the barcode reader isn t working properly (mechanical failure). EXTERNAL When we look at systems, we also look for active controls in place that will either prevent an error from occurring or detect it before it can cause harm, like separating look-alike products to discourage wrong drug or dose errors. 39 Active controls are deliberate steps in the process that specifically help manage the risk of errors, such as independent data entry verification of prescriptions entered into the computer. Passive controls are features inherent in the system that might help control risks, but are not specifically set up for that purpose, such as differences in tablet appearance that may alert a pharmacist or PHARMACY ASSOCIATE to an incorrect medication. 40 Strong community pharmacy dispensing systems typically have many control elements built into their procedures, reducing the probability that an error can reach the patient. Please bear in mind that while error frequencies can be substantially reduced, human systems are inherently fallible. 41 Human beings make mistakes. Most behavioral errors are inadvertent. We call these HUMAN ERRORS to denote that they are intrinsic and do not involve any conscious choice on the part of the individual. HUMAN ERROR rates are often influenced by system design issues and PERFORMANCE SHAPING FACTORS (PSFS) like fatigue, lighting, distractions, or interruptions (Table 1). 42,43 PSFS can have a positive or negative effect on performance. For example, staff training can influence performance either positively (e.g., when it emphasizes the appropriate response) or negatively (e.g., when absent or done badly). Task complexity Information complexity Ergonomics/Human-machine interface Procedures, including job aids Work environment Communication/Information exchange Workflow/Work processes Stress Time available/time urgency Design of products and labels Organizational culture/management INTERNAL Training Experience Familiarity with task Mental and physical health/fitness for duty Task tension and engagement Stress Motivation Previous actions A second type of behavioral error is AT-RISK BEHAVIOR (Table 2). These are behavioral choices where the increased risk isn t recognized or is mistakenly believed to be justified. AT-RISK BEHAVIORS are very common we often cut a corner, take a shortcut, or work around a system feature in order to get our work done. We engage in AT-RISK BEHAVIORS because we feel pressured to use our time wisely, to do more with less, and any risk associated with the behavior Table 2. The Two Types of Behavioral Errors HUMAN ERROR AT-RISK BEHAVIOR Inadvertently doing other than what should have been done; the failure of a planned action to be completed as intended (error of execution) or the use of a wrong plan to achieve an aim (error of planning). Examples: Physician orders wrong dose of a fentanyl transdermal patch due to knowledge deficit Pharmacist mistakes a glulisine vial for a glargine vial and dispenses the wrong insulin analog Behavioral choice that increases risk where risk is not recognized or is mistakenly believed to be justified. AT-RISK BEHAVIORS are often employed and tacitly encouraged as a workaround for various system, technological, or environmental weaknesses. Example: Patient identification process not followed at the point of sale Skipping verification of a trusted colleague s data entry Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

58 Appendix D Understanding Errors and At-Risk Behaviors continued quickly fades as we reap the rewards of engaging in the AT-RISK BEHAVIORS. The positive rewards for taking shortcuts, for example, rapidly foster continuance despite our knowledge on some level that it could risk patient safety. In fact, taking shortcuts could even be labeled as efficient behavior. Yet, these AT-RISK BEHAVIORS often emerge because of system-based problems. Talking on a cell phone while driving or choosing to quickly bypass computer warnings are AT-RISK BEHAVIORS, for example. People don t believe that their choices will cause harm and are surprised when an accident occurs. Time or production pressures often drive increased AT-RISK BEHAVIORS, as people adopt shortcuts to meet their goals. It s not unusual for people to be reluctant to admit that they are cutting corners in some fashion, but understanding when and why these behaviors occur is key to building safer systems. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

59 Appendix E How HAMMERS Works The HAMMERS tool is designed to help community pharmacies reduce errors with HIGH-ALERT MEDICATIONS using an advanced risk-reduction method borrowed from aerospace, nuclear power, and petrochemical safety engineering called SOCIO-TECHNICAL PROBABILISTIC RISK ASSESSMENT (ST-PRA). ST-PRA is derived from a probabilistic risk assessment (PRA) methodology to improve safety that has been used to support decision-making in many complex, high-risk industries 43,44,45 because multiple failures can be considered in combination with one another. While PRA is used to assess predominantly mechanical/procedural system failures, ST-PRA includes assessment of HUMAN ERRORS and behavioral choices such as intentional procedural deviations (e.g., AT-RISK BEHAVIORS), making it an effective tool for managing errors in complex environments like NASA, 40 hospitals, and community pharmacies. 22 ST-PRA examines combinations of events, processes, and behaviors that increase or decrease risk. The process starts with the development of an event tree that includes every small step involved in the system under evaluation. In preparation for building the HAMMERS tool, for example, a basic event tree template that depicted every step in the dispensing process was created. The event trees built using the template depict the events or combination of events leading to an undesired outcome, i.e., the patient receives a wrong drug, a wrong dose, the wrong directions, or someone else s drugs. The event tree is very detailed and includes all the different combinations that a single task can entail. For example, in the event tree associated with dispensing the wrong drug, the patient counseling process includes distinct events for opening and not opening the prescription vial during the counseling session at the counter, and not opening the vial during a counseling session at the drive-through window. In the event tree associated with a prescribing error that gets through the pharmacy dispensing system, the prescription receipt process includes distinct events associated with receiving the prescription electronically, by fax, in person at the pharmacy counter, or by telephone. Even prescriptions received by telephone are subdivided based on whether the prescription was left on voic or communicated directly to a pharmacy staff member. The event trees are very detailed pictorial descriptions of all the possible ways that medications can be dispensed from the pharmacy. Each event in the event tree is assigned an estimated frequency of occurrence. For example, an event tree associated with dispensing a correctly filled prescription to the wrong patient may assign a 30% rate of occurrence to a patient counseling event (and a 70% rate of non-occurrence to the event related to no patient counseling). An event associated with failing to capture the error when counseling the patient at the counter while opening the prescription vial may be assigned a very low rate of 0.1%. Estimated frequencies are provided by system experts who know and work within the processes under assessment. In the case of the HAMMERS tool, the event trees used as the basis of the tool were initially populated with estimates using a team of experienced pharmacists and PHARMACY ASSOCIATES representing large national chains. The pharmacies where they worked were diverse in regards to geographic location, setting (urban/rural), prescription volumes, hours of operation, staffing patterns, and drive-through service. Meeting over several weeks, for each type of preventable adverse drug event prescribing error, data entry error, drug selection error, point of sale error the team provided frequency estimates for hundreds of events associated with: receipt of prescription, data entry and verification, filling the prescription, point of sale, and counseling. Reviewing each event, the team used published sources (Appendix B) and their own experiences to estimate error rates. They also factored in the relevant PERFORMANCE SHAPING FACTORS (PSFS), such as those in Table 1 in Appendix D. In very general terms, given a human performance limit for a single person operating in ideal conditions of (1/10,000), the modeling team often started with an error rate of (1/1,000) to account for the negative influence of a single PSF such as time pressure. Identification of additional PSFs (e.g., illegible prescriptions, look-alike product names, complex tasks, minimal training) was part of the group process. The number of PSFS and their degree of influence helped the team adjust its estimates upward or downward through an iterative process before deciding on a final probability of error. When using the HAMMERS tool, your frontline pharmacists and PHARMACY ASSOCIATES will supply many of Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

60 Appendix E How HAMMERS Works continued the estimates in response to questions in the tool that will generate a unique risk profile for your pharmacy. The chain of events leading to a specific error for example, a new prescription is entered into the wrong patient s profile is called a failure path. There are thousands of different failure paths leading to a specific error. When completed, the event trees define hundreds of failure paths, showing how each error is initiated and describing the recovery or CAPTURE OPPORTUNITIES for that error before it reaches the patient. The most pernicious failure path is called a single fault failure path because there are no, or very limited, opportunities to correct it before the error impacts the patient. An example would be at the point of sale, the wrong patient s bag is pulled and given to another patient. Unless that patient detects the error, the drugs are dispensed to the wrong person. This level of detail is unavailable from any other source. ST-PRA event trees predict the likelihood of each type of error, based on the risk probabilities associated with each combination of events leading to the undesirable outcome. The HAMMERS tool offers community pharmacies a unique opportunity to identify and quantify dispensing system vulnerabilities and how to go about reducing the risks that have the highest probability of reaching patients. Words or terminology in BLUE SMALL CAPS are defined in the Glossary ISMP

61 References References 1) Marx DA, Slonim AD. Assessing patient safety risk before the injury occurs: an introduction to socio-technical probabilistic risk assessment in healthcare. Qual Saf Health Care. 2003;12(suppl 2): ) Gertman D, Blackman H, Marble J, et al. The SPAR-H human reliability analysis method. Prepared for The Division of Risk Analysis and Applications, Office of Nuclear Regulatory Research, US Nuclear Regulatory Commission: August 2005; Washington, DC. NRC Job Code W ) Shelton CP. Human interface/human error. Pittsburgh, PA: Carnegie Mellon University; Accessed at June 10, ) System Reliability Center. Technique for human error rate prediction (THERP). Rome, NY: Alion Science and Technology; ) Garnerin P, Pellet-Meier B, Chopard P, et al. Measuring human-error probabilities in drug preparation: a pilot simulation study. Eur J Clin Pharmacol. 2007;63(8): ) Lewis M. THERP: Technique for human reliability analysis. Pittsburgh, PA: University of Pittsburgh; Accessed at May 15, ) The Institute of Petroleum. Human reliability analysis. London, England: Human factors no. 12 briefing notes; ) Gibson WH, Hicking B, Kirwan B. Feasibilty study into the collection of human error probability data. Brussels, Belgium: European Organisation for the Safety of Air Navigation, Eurocontrol Experimental Centre; EEC Note No. 02/06. 9) Nanji KC, Rothschild JM, Salzberg C, Keohane CA, et al. Errors associated with outpatient computerized prescribing systems. J Am Med Inform Assoc. 2011;18(6): ) Gandhi TK, Weingart SN, Seger AC, Borus J. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med. 2005;20(9): ) Kozer E, Scolnik D, Macpherson A, Keays T, et al. Variables associated with medication errors in pediatric emergency medicine. Pediatrics. 2002;110(4): ) Buurma H, de Smet PAGM, van den Hoff OP, Egberts ACG. Nature, frequency, and determinants of prescription modifications in Dutch community pharamacies. Br J Clin Pharmacol. 2001;52: ) Camp SC, Hailemeskel B, Rogers TL. Telephone prescription errors in two community pharmacies. Am J Health- Syst Pharm. 2003;60(6): ) Solberg LI, Hurley JS, Roberts MH, Nelson WW, et al. Measuring patient safety in ambulatory care: Potential for identifying medical group drug-drug interaction rates using claims data. Am J Manag Care. 2004;10(11 Pt. 1): ) Zhan C, Correa-de-Araujo R, Bierman AS, Sangl J, Miller MR, Wickizer SW, Stryer D. Suboptimal prescribing in elderly outpatients: Potentially harmful drug-drug and drug-disease combinations. J Am Geriatr Soc. 2005;53(2): ) McPhillips HA, Stille CJ, Smith D, Hecht J, et al. Potential medication dosing errors in outpatient pediatrics. J Pediatr. 2005;147(6): ) Friesner DL, Scott DM, Rathke AM, Peterson CD. Do remote community telepharmacies have higher medication error rates than traditional community pharmacies? Evidence from the North Dakota Telepharmacy Project. J Am Pharm Assoc. 2011;51(5): ISMP

62 References References continued 18) Franklin BD. Dispensing errors in community pharmacy: frequency, clinical significance and potential impact of authentication at the point of dispensing. Int J Pharm Pract. 2007;15(4). 19) Flynn EA, Barker KN. Effect of an automated dispensing system on errors in two pharmacies. J Am Pharm Assoc. 2006;46(5): ) Flynn EA, Barker KN, Carnahan BJ. National observational study of prescription dispensing accuracy and safety in 50 pharmacies. J Am Pharm Assoc. 2003;43(2): ) Grasha AF. Psychosocial factors, workload, and risk of medication errors. US Pharm. 2002;27(4):HS32-HS52. 22) Cohen MR, Smetzer JL, Westphal JE, Comden SC, et al. Using risk models to improve safety with dispensing highalert medications in community pharmacies. Accepted for publication in: J Am Pharm Assoc ) Ashcroft DM. Prospective study of the incidence, nature and causes of dispensing errors in community pharmacies. Pharmacoepidemiol Drug Safety. 2005;14(5): ) Grasha AF. Process and delayed verification errors in community pharmacy. Technical Report Number , December Cognitive Systems Performance Laboratory. Accessed at June 10, ) Malone DC, Abarca J, Skrepnek GH, Murphy JE, et al. Pharmacist workload and pharmacy characteristics associated with dispensing of potentially clinically important drug-drug interactions. Med Care. 2007;45(5): ) Flynn EA, Barker KN, Gibson JT, Pearson RE, et al. Impact of interruptions and distractions on dispensing errors in an ambulatory care pharmacy. Am J Health-Syst Pharm. 1999;56: ) Fogarty GJ, McKeon CM. Patient safety during medication administration: The influence of organizational and individual variables on unsafe work practices and medication errors. Ergonomics. 2006;49: ) Kimberlin CL, Jamison AN, Linden S, Winterstein AG. Patient counseling practices in US pharmacies: effects of having pharmacists hand the medication to the patient and state regulations on pharmacist counseling. J Am Pharm Assoc. 2011;51(4): ) Flynn EA, Barker KN, Berger BA, Lloyd KB, et al. Dispensing errors and counseling quality in 100 pharmacies. J Am Pharm Assoc. 2009;49: ) Nau DP, Erickson SR. Medication safety: patient s experiences, beliefs, and behaviors. J Am Pharm Assoc. 2005; 45(4): ) Svarstad BL, Bultman DC, Mount JK. Patient counseling provided in community pharmacies: effects of state regulation, pharmacist age, and busyness. J Am Pharm Assoc. 2004;44(1): ) Paterno MD, Maviglia SM, Gorman PN, Seger DL, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc. 2009;16(1): ) Isaac T, Weissman JS, Davis RB, Massagli M, et al. Overrides of medication alerts in ambulatory care. Arch Intern Med Feb 9;169(3): ) van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2): ) Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc. 2008;15(4): ISMP

63 References References continued 36) Kaiser Family Foundation: Prescription drug trends. September Accessed at May 15, ) Allan EL, Barker KN, Malloy MJ, Heller WM. Dispensing errors and counseling in community practice. American Pharmacy. 1995;NS35(12): ) Aspden P, Wolcott JA, Bootman JL, Cronenwett, LR, eds. Preventing Medication Errors: Quality Chasm Series. Washington, DC: The National Academies Press; ) Comden SC, Marx D, Murphy-Carley M, et al. Using probabilistic risk assessment to model medication system failures in long-term care facilities. In: Advances in Patient Safety: From Research to Implementation; vol. 2. Rockville, MD: Agency for Healthcare Research and Quality; February 2005: Accessed at January 18, ) Battles JB, Kanki BG. The use of socio-technical probabilistic risk assessment at AHRQ and NASA. Paper presented at: International Conference on Probabilistic Safety Assessment and Management; June 11 18, 2004; Berlin, Germany: Springer-Verlag. 41) Marx DA. Whack-a-Mole: The Price We Pay For Expecting Perfection. Plano, TX: By Your Side Studios; ) Reason J. Hazards, defences and losses. In: Reason J. Managing the risks of organizational accidents. Burlington, VT: Ashgate Publishing Company; 2000: ) Stamatelatos M, Apostolakis G, Dezfuli H, et al. Probabilistic risk assessment procedures guide for NASA managers and practitioners. Version 1.1. Prepared for the Office of Safety and Mission Assurance, NASA Headquarters; August 2002; Washington, DC. Accessed at June 3, ) Marx DA, Westphal JE. Socio-technical probabilistic risk assessment: it s application to aviation maintenance. Int J Aviat Psychol. 2008;18(1): ) Sues RH, Chen HC, Oswald EA, et al. Integrating internal events in an external probabilistic risk assessment: Tornado PRA case study. Reliab Eng Syst Saf. 1993;40: ISMP