COMFIRMATION AS A KEY FOR PATIENT SAFETY: A NETWORK ANALYSIS OF INCIDENT REPORTS

Transcription

1 COMFIRMATION AS A KEY FOR PATIENT SAFETY: A NETWORK ANALYSIS OF INCIDENT REPORTS Yingzi Jin 1), Satoru Yamamoto 2), Yutaka Matsuo 1), Ichiro Sakata 2), Masanori Akiyama 2) 1) University of Tokyo, Institute of Engineering Innovation 2) University of Tokyo, Policy Alternatives Research Institute ABSTRACT According to the results of the statistical analysis on incident reports released by the Japanese Ministry of Health, Labour and Welfare, drug-related medical accidents account for more than half of all medical accidents. With respect to the causes of medical accidents, those caused by human carelessness, such as incorrect drugs or incorrect drug dosages, account for more than 40% of all medical accidents. Therefore, establishing a method to prevent the occurrence of accidents due to human carelessness is a highly contributing factor in achieving safe, worry-free medical care. Accordingly, the purpose of this research is to construct a knowledge base that is useful for examining accident prevention measures by specifying a drug group closely related to the accident as well as the causes of the accident through the structuring of bibliographical information. Natural language processing and network analysis are used as research methods. The network analysis method facilitates analysis on various relationships that are not only direct relationships but also indirect relationships, which are at least two steps away from the former. The incident reports in Japan are used as the subjects of this research. First, the frequently appearing characteristic words were extracted according to the three categories of the accident s background, details, and solutions described in the report, and then networks using these extracted words were created. Next, using the pharmaceutical dictionary and our own rules for dissolving the discrepancy in descriptions, drug information was extracted from the accident report, and a drug network was created. From the above analyses it has become clear that inadequate information on drug labels, physician s instructions, and patient misunderstanding are the more important factors in medical accidents. Moreover, drugs and the combinations of drugs that frequently cause accidents have been revealed. The POAS (Point of Act System) is known as a cutting-edge medical information system that prevents confirmation errors made by humans. Finally, using the above results, the degree of effectiveness of POAS in the prevention of drug-related accidents was calculated. Of all the drugs that were returned through POAS, the ones that had actually caused medical accidents accounted for 31.6%. In addition, with regard to the ten most frequently returned drugs through POAS, the Pearson s Correlation between the number of drugs returned and the number of accident cases was high at 0.508, thus confirming the effectiveness of POAS implementation. From this point forward, in order to achieve an even more effective analysis, the following factors are important in incident reports: (i) unification of descriptions pertaining to drug(s) and prevention of discrepancy in descriptions; and (ii) encouraging more descriptions pertaining to the confirmation details. 1. INTRODUCTION There has been an increase in the awareness that the use of information pertaining to medical accidents is important when implementing medical safety measures. The medical safety mechanism of WHO aims to prevent accidents by reusing incident reports through the introduction of IT technology. Harvard University is engaged in the standardization for the collection of medical accident reports and accident information in the risk management consortium. In England, the National Health Service conducts the medical accident/incident report collection project. Even in Japan, the Ministry of Health, Labour and Welfare began the project to Collect Medical Near-Miss/Adverse Event Information in Through this project, the Ministry conducts analyses based on the collected incident reports and releases their results. According to the research conducted by the Japan Council for Quality Health Care, it was revealed that of approximately 1,067 incident reports, majority of them were drug-related incidents (see Table 1). With regard to the details of such accidents, incorrect drugs and incorrect drug dosages accounted for 40% of all the accidents. For the causes of these medical errors, descriptions such as I did not confirm with the physician and I did not firmly confirm the name of the drugs written on the prescription are seen in the reports, suggesting that human carelessness is the cause of many errors.

2 However, all the analyses conducted till date are relatively simple. It can be assumed that by conducting an analysis while paying attention to the semantic information of the language contained in the report, knowledge that is effective in accident prevention can be extracted. Accordingly, the purpose of this research is to construct a knowledge base that is useful for the examination of accident prevention measures, by specifying a drug group closely related to the accident and the causes of the accident through the structuring of bibliographical information. Regarding the subjects of research, the focus was on cases related to drugs having high incidence rates. Table 1 Categories and the Number of Incident Reports Category # Cases Category # Cases Extravasation 15 Combination 24 Incorrect speed 76 Incorrect method 50 Incorrect patient 71 Incorrect drug 219 Forgot to give or 30 Drug 31 take drug hypersensitivity Incorrect drug 219 Incorrect dosage 223 Dosage and usage 29 Others Methodology 2. METHODOLOGY & DATA In this research, natural language processing was first conducted on the incident reports. Keywords that emerge characteristically were then extracted for each category of background/causes, details, and solutions, using morphological analysis, the tfidf method, and the Ngram method. Subsequently, the semantic tendency of the incident report was investigated in order to create a network of words by calculating the co-occurrence information of the words using the Jaccard coefficient. The tendency of the drug-related accident was then specified by creating a drug network which specified the drug names and the combination of the drugs described in the incident reports using the pharmaceutical dictionary. The co-occurrence information between the drugs required for specifying the combination was calculated using the Jaccard coefficient. Moreover, the drug categories that tend to cause accidents or those whose concomitant use tends to cause accidents were clarified by organizing the drugs into their corresponding drug categories. Lastly, the effectiveness of introducing the Point of Act System (POAS), the cutting-edge medical information system, for the prevention of drug-related accidents was calculated through the investigation of the correlations between drugs with high rates of return and those with large numbers of return cases under POAS Data sets Of all incident reports provided by the Project to Collect Medical Near-Miss/Adverse Event Information, drug-related data that had been published during the period between 2005 and 2010 were used in this research. In order to achieve an accurate analysis, of all the cases provided, only 1,067 documents that included the abstract, background, and solutions were used. The free composition style was taken for each case, with the abstract, background, and solutions being approximately 300 characters long. Moreover, each case was classified into one of two groups drugs and accidents. As shown in Table 1, there are twelve categories. For POAS data, the Act data from October of each year during the five-year period from 2002 to 2006 was used. Of all the 473,337 cases in the Act data, the number of drugs returned and the rate of return recorded as type 2 and type 3 Acts were used. As pharmaceutical dictionary, the glossaries for drugs for medical professionals and the general public provided by the Japan Pharmaceutical Information Center (JAPIC) were used. 3. RESULT 3.1 Keyword Extraction When working on a document, the frequently used method in natural language processing is that each word from the document (Part of Speech) is extracted first and the importance of the frequency of the word in the document is then calculated. The significance of these words determined through this method is used for the classification or ranking of the document. First, the sentences written in natural language were divided into morpheme lines, and the part of the speech of each word was identified. Mecab [1] was used as a morphological analysis engine. In the morphological analysis, a word that has an independent meaning, such as prescribe, form, double, and check, are handled as independent morphemes. It is therefore necessary to connect several morphemes based on the frequency of the continuous occurrence of the word in order to obtain compound words such as prescription, double check, and brought drug. In other words, compound words in which word B emerges immediately after word A is determined as AB (2 grams). If AB emerges more than at a certain frequency in the copus (more than five times in this research), it is handled as a candidate for a characteristic word. Eventually, compound words that connect up to six individual words (6 grams) are handled. Next, the significance of the word is calculated using the tfidf formula [2] as shown below. N tfidf i = tf i log( ) (1) df i

3 Here, tf i represents the occurrence of i and df represents the number of documents in which i appears (i.e., the number of cases) and N represents the number of all cases. The top ten keywords that appear in the incident report such as background/causes, details, and solutions are shown in Table 2. Under the category of accident background, the words lack, confirmation, inadequacy, drugs, and instruction rank high. Moreover, the fact that the word nurse ranks high suggests that there are many accidents related to nurses. Also, the words confirmation, drugs, and double check rank high under the category of solutions. Tables 3 and 4 show the keywords obtained from the reports of the accidents caused by incorrect drugs and drug dosages. With regard to incorrect drugs, the words similar, standard, dispensation, and inspection rank high. With regard to incorrect drug dosages, the words instruction, unit, and standard rank high. Table 2 Keywords in Case Reports Top Background/Causes Details Solutions 1 Lack Patient Confirmation 2 Confirmation Administration Drugs 3 Inadequate Prescription Thorough 4 Drugs Instruction Occasion 5 Instruction Infusion Instruction 6 Prescription Nurse Caution 7 Not present Dispensation Double check 8 Patient Ward Brought drug 9 Description Physician Ensure 10 Nurse Injection Prescription Table 3 Keywords in Case Reports (Incorrect Drugs) Top Background/Causes Details Solutions 1 Similarity Dispensation Drugs 2 Drugs Prescription Confirmation 3 Standard Patient Thorough 4 Prescription Ward Dispensation 5 Confirmation Nurse Label 6 Inadequate Injection Caution 7 Lack Error Medicine 8 Medicine Instruction Standard 9 Dispensation Administration Inspection 10 Inspection Drugs Occasion Table 4 Keywords in Case Reports (Incorrect Drug Dosages) Top Background/Causes Details Solutions 1 Instruction Prescription Thorough 2 Unit Administration Confirmation 3 Prescription Instruction Instruction 4 Lack Unit Prescription 5 Confirmation Appreciation Prescription 6 Check Oral administration Drugs 7 Not present Dispensation Brought drug 8 Standard Nurse Double check 9 Brought drug Ward Input 10 Dispensation Physician Caution 3.1 Network of Characteristic Words One word such as lack, confirmation, or drugs alone cannot express the tendency of the accident. In this research, the co-occurrence networks of the words were created by connecting the words that co-occurred with each other at a high frequency. The degree of cooccurrence is calculated using the Jaccard coefficient [3]. Here, if the co-occurrence frequency is expressed as A B when word A and word B appear in the same accident case and the number of cases in which either word A or B appears is expressed as A B the ratio of the Jaccard coefficient can be obtained. This ratio is used as an index to indicate the tendency of co-occurrence of the words A and B. Jaccard coefficient A B A B = (2) Figure 1 depicts the networks of words created using the accident reports related to incorrect drugs. Each node represents a word, and an edge represents the intensity of the co-occurrence between the words. The stronger the relationship between the words, the shorter the length of the edge. First of all, in viewing the network for background/causes (see Figure 1 (a)) it is clear that the network is created around the word confirmation, and one can see that the cause of many accidents is the fact that the confirmation on drugs, label, prescription, and physician s instruction lack or is inadequate. Connecting the words that co-occur frequently allows us to understand what tends to become inadequate. In the network of accident details (see Figure 1 (b)) many different words appear at once, indicating the presence of diverse accident details. Viewing the network for solutions (see Figure 1 (c)), as with the network for background/causes, it is created around the word confirmation. Furthermore, when using the MCODE [4] method on the network for solutions, a clique shown in Figure 1(d) can be depicted. A clique is a group of nodes that are mutually connected in the form of direct coupling, i.e., a group containing the edges created with the combinations of all nodes. For instance, one can see that in clique 1 confirmation of prescription is mostly required as a solution for the accidents related to similarity and drugs, and that in clique 3 the solutions of thorough check for the patient is mostly required. Both cliques indicate that it is important to reduce human carelessness. 3.3 Drug Networks The accident tendency that depends on drugs more specifically the drugs that most frequently cause accidents or the combinations of drugs that most frequently cause accidents was investigated. Drug glossaries for medical professionals and the general public provided by JAPIC were used to specify the drugs appearing in the reports.

4 Figure 1 Networks of the Words in Incident Reports (a) Network regarding Background/Causes (b) Network regarding Details (c) Network regarding Solutions (d) Clique of the Words Extracted from the Network regarding Solutions However, in the reports there were many descriptions using the drugs superclass names. For example, for 10 mg of Dormicum injection it is described as after the placement of Dormicum, Proternol and Dormicum were administered. Therefore, matching was conducted after converting the drug names to superclass names using our own rules for drug name conversions (see Table 5). Table 5 Rules for Drug Name Conversions Rule Drug Name => Superclass Name Excluding dosage, * Aspora Capsule 5, Aspora Capsule 10 => specifications, unit Aspora Capsule * Atarax Tablet 10 mg, Atarax Tablet 25 mg, => Atarax Excluding formulation * Artane Powder 1%, Artane Tablet (2 mg) => Artane * Akineton Tablet 1 mg, Akineton Injection 5 mg, Akineton Granule 1% => Akineton Excluding supplementary explanations provided in the parentheses such as,[ ] and ( ) *Aspirin (Kenei) Aspirin (Bayer), Aspirin (Hishiyama) => Aspirin * [Deleted Product]Elental P=> Elental P * Anaesthesine (Hishiyama) [Interim Measures]=> Anaesthesine Figure 2 depicts the drug networks, in which each node represents a drug and in each node the name of the drug is described. The size of the node represents the number of accident reports containing the name of the drug. The edge represents the presence of the co-occurrence relationship. The intensity of the co-occurrence relationship when two drugs appear in the same accident is represented by the length of the edge. The co-occurrence intensity can be calculated using the following formulae: Jaccard(x,y) = x y / x + y. In other words, this is the rate of accidents caused by drugs x and y simultaneously among all the accidents described in the reports. The bigger the node, the more frequent the drugs will cause accidents. Also, the more the edges, the more frequent the drugs will cause accidents when used with other drugs. For example, drugs such as saline (95 cases), sodium chloride (28 cases), and glucose (21 cases) frequently appear in accidents, thus suggesting that those drugs frequently cause accidents when used with other drugs. Next, the names of the drugs were organized according to the categories specified by JAPIC to obtain the drug category networks (see Figure 3). Each node represents a drug category, and the size of each node reflects the total number of accident reports in which the drugs belonging to the concerned category appear. The edge indicates the intensity of the co-occurrence among the categories. The ratio of the co-occurrence of drugs belonging to the same category was calculated using the Jaccard coefficient. For example, one can see that the categories of blood substitute (164 cases), adrenal hormone drugs (58 cases), drugs for psychoneurosis (53 cases), and antipyretic analgesics (51 cases) frequently cause accidents by themselves or when used with other drugs.

5 One can also see that the concomitant use of drugs belonging to the category of antacid and drugs belonging to the category of laxative, enema frequently causes accidents. Figure 2 Network of Drugs Figure 3 Network of Drug Categories deemed as a type of act, and the subject of the management is the POA. In POAS, all act processes, from the instructions of the physician for the injection, drug dispensation by the pharmacist, to the nurse s coinjection and (final) implementation of the injection are recorded. An act in which all the processes have been completely performed is classified under type 1. An act that was discontinued before the co-injection of the drug and an act that was discontinued after the co-injection of the drug are classified under types 2 and 3, respectively. Akiyama et al. have proposed a measure to reduce wasted drugs in types 2 and 3 due to product returns by adjusting the time of the drug bring co-injected based on this information [5]. There are many causes of act types 2 and 3, in which drugs are returned due to the discontinuation of orders, such as changes in the condition of the patients and the replacement of drugs. In any case, if no alarm call is given and the process is not discontinued then, this raises the possibility of an accident. In POAS, if a nurse attempts to use a drug that is different from the one ordered by the physician, an alarm call will be given to her/him requesting that he/she reconfirm the drug, thereby preventing accidents caused by human carelessness. In Japan, four hospitals (International Medical Center of Japan, Morioka Red Cross Hospital, Kyoto Red Cross Hospital, and Japanese Red Cross Kochi Hospital) have already introduced POAS, and have reduced the number of incidents to zero [5]. Furthermore, because all act information, i.e., total population can be collected in POAS, it is possible to calculate the percentages of the drugs that are returned (i.e., the rate of return) for each drug appearing in the prescription given by a physician. Using the aforementioned analysis results, the correlations between drugs that are frequently returned or those with high rates of return under POAS, and the drugs that tend to appear in real accidents, can be calculated. 4.2 Correlation between POAS and Incident Reports 4. PROPOSAL FOR PREVENTION OF MEDICIAL ACCIDENTS THROUGH POAS 4.1. Point of Act System (POAS) POAS is a system for managing the point (Point) of a medical action (Act). In this system, the main focus is placed upon the 6Ws and 1H information (when, where, who, whom, why, what, and how), and such information is collectively managed. Accurately recording the implementation information of a medical action at the point where the action occurs, through the use of bar codes and PDA (Personal Digital Assistant), facilitates the progress management of the process. In the conventional ordering system, information was managed for each order. However, in POAS an order is The overlap between the drugs for which alarm calls were frequently given and those that actually caused accidents was obtained. As shown in Figure 4, of all the drugs that actually caused accidents, 193 evoked alarm calls more than once (types 2 and 3), 242 appeared in actual accidents more than once, and 61 overlapped. Statistically, 31.6% (i.e., 61/ %) of the returned drugs under POAS caused an accident, so it can be said that by introducing POAS, such accidents have been prevented. In addition, it is obvious that when a drug accident or a drug dosage error has occurred under POAS, 25.2% of all accidents can be prevented by giving an alarm call prior to the procedure. These drugs include those that have high rates of return and those that tend to cause accidents as well as those that cause accidents with other drugs. Regarding these 61 drugs, i.e., the overlap

6 between the POAS and incidents, the Pearson s Correlation between the number of returned drugs under POAS and the number of incidents is In particular, regarding the ten drugs most frequently returned, the Pearson s Correlation between the number of returned drugs and the number of incidents is The names of these top ten drugs are listed in Table 6, and one can see that many of them are medically important drugs. When considering only the number of cases, one cannot say that drugs with high rates of return always cause accidents. Figure 5 shows the distribution of the number of incidents for each range of rate of return and the number of returns. It is obvious that the number of returns (115) and that of incidents (62) are the greatest in the drugs whose rates of return range from 0.1 to 0.2. Consequently, in this range of rate of return, 53.9% of the drugs returned may cause accidents, and it can be assumed that these accidents can be prevented by giving alarm calls for those drugs through POAS prior to the procedure. 5. CONCLUSION In this paper, the characteristic words were extracted by analyzing incident reports, and the co-occurrence networks of the characteristic words were created as well as an analysis focusing on drugs was conducted. As a result, the language networks with the hub of the word confirmation, thereby revealing that inadequate confirmations on the drug labels, instructions of a physician and patient were very significant causes of accidents. Moreover, it has become clear which drugs and combinations of drugs frequently appear in incident reports. These results suggest the effectiveness of introducing the network analysis method. POAS (Point of Act System) is a medical information system that can prevent confirmation errors due to human carelessness and is currently being implemented at four hospitals. Thus, using the above analysis results, the degree of effectiveness of this system in terms preventing accidents related to drugs was calculated. Of all the drugs that were returned through POAS, the ones that had actually caused medical accidents accounted for 31.6%. In addition, regarding the ten most frequently returned drugs through POAS, the Pearson s Correlation between the number of drugs returned and the number of accident cases was high at 0.508, thus confirming the effectiveness of implementing POAS. Moreover, it has been shown that accidents can be more effectively analyzed by focusing on the drugs and the combinations of the drugs that have been deemed important through this research. We want to propose the collection of more descriptions of the word confirmation, which has been recognized as extremely important for accident prevention in incident reports because doing so will enable a more effective analysis. In addition, it was necessary to use the glossaries Figure4 Diagram of Drug Overlap Table 6 Top Ten Drugs with High Rates of Return Under POAS Drug Name Return Rate # Returns # Cases Droleptan Metilon Perdipine Prograf Dobutrex Sandimmun Navelbine Vasolan Lepetan Anapeine Figure 5. Number of Incidents for Each Range of Rates of Return and the Number of returns of POAS analysis. In addition, it was necessary to use the glossaries provided by JAPIC as well as our rules for the conversion of names of drugs. For facilitating a more accurate analysis, we want to propose an improvement of notational rules for the names of drugs and dosages in incident reports 6. REFFERENCES [1] [2] G. Salton and M. McGill, Introduction to Modern Information Retrieval, McGraw-Hill Book Company, [3] C.D. Manning and H. Schutze, Foundations of statistical natural language processing, The MIT Press, 2002 [4] G.D. Bader, and C.W. Hogue, An automated method for finding molecular complexes in large protein interaction networks, BMC Bioinformatics, 4:2, 2003 [5] M. Akiyama, Risk Management and Measuring Productivity with POAS-Point of Act System -- A Medical Information System as ERP (Enterprise Resource Planning) for Hospital Management, Method Inf Med.2007:46: , 2007