*University of Health Sciences/The Chicago Medical School **Illinois Institute of Technology (312) ABSTRACT INTRODUCTION.

Transcription

1 MAINTAINING A KNOWLEDGE BASE USING THE MEDAS KNOWLEDGE ENGINEERING TOOLS Frank Naeymi-Rad*, Martha Evens**, Timothy Koschmann* Chui-Mei Lee* Rao Y.C Gudipati *, Theresa Kepic*, Eric Rackow*, Max Harry Weil* *University of Health Sciences/The Chicago Medical School **Illinois Institute of Technology (312) ABSTRACT This paper describes the process by which a medical expert creates a new knowledge base for MEDAS, the Medical Emergency Decision Assistance System. It follows the expert physician step by step as a new disorder is entered along with its relevant symptoms. As the expanded knowledge base is tested, inconsistencies are detected, and corrections are made, showing at each step the available tools and giving an example of their use. INTRODUCTION This paper describes a set of Knowledge Engineering Tools developed to make it easy for physicians to add medical expertise to the knowledge base of MEDAS, the Medical Emergency Decision Assistance System. MEDAS is the outgrowth of almost ten years of research on pattern recognition and expert systems carried on originally at the University of Southern California and continued for the last two years at the Chicago Medical School. It differs from most other medical expert systems in applying advanced pattern recognition concepts to the problems of diagnosis. The system currently contains information about eighty-seven lifethreatening diseases that show up most frequently in emergency rooms and critical care facilities. The system provides differential diagnoses based on approximately 780 features (symptoms, physical signs and results of objective [laboratory, imaging] tests). A year ago while expanding MEDAS, the details of which are described below, it became obvious to us that a physician attempting to enter a new disease or a new test was faced with the almost impossible task of tracing through lists of meaningless numbers. What is more, we had no way of detecting inconsistencies and no facilities for making corrections. The MEDAS TOOLBOX was created using Accent-it (Relational Data Base) to provide the expert physician with a comfortable interface for entering information into the system, detecting errors, and correcting them. We show here how a medical expert uses the TOOLBOX to create a new knowledge base. The importance of knowledge engineering tools is stressed by Buchanan and Shortliffe 3. Without tools the expert has to explain his expert knowledge to the knowledge engineer and leave it to that intermediary to encode what he knows. This estranges the expert from the system and diffuses responsibility for validation, moreover it increases the error rate and compromises the reliability of the system. We have tried tools that do for MEDAS some, at least, of what Teiresias 4 does for MYCIN, tools that help the physician transfer knowledge directly to the system without the need of an intermediary, and also tools which help identify inconsistencies and errors. The MEDAS System MEDAS is designed to provide the clinician with assistance in determining diagnosis, ordering tests, and providing treatment. The strategy used in diagnosis is based on a modified version of Bayes' theorem that allows the system to handle patients with more than one disorder. 5 The system develops disorder patterns in terms of collections of features. A feature is any piece of clinical data, age, sex, race, complaints, signs, symptoms, findings from the physical examination, results of tests and diagnostic medical procedures. Computations are based on estimates of the probability that a given symptom will or will not appear in a patient with an identifiable disease or syndrome. Thus the kind of knowledge that the system needs to acquire from the expert is information about disorders, features, and estimated probabilities. To track down errors and inconsistencies in the data base the expert needs information from the system in return about the ways in which features interact and also comparisons between feature patterns for different disorders. The system also needs to acquire treatment protocols from the expert. 6 At the moment these are simply a set of actions to be performed and can be handled with a simple text editor. We plan to increase the sophistication of this part of the system. The addition of a relational data base to MEDAS has made the development of knowledge engineering tools much simpler. 7

2 A year ago the twelve unformatted FORTRAN files used by the then-current version of MEDAS were replaced by a commercial Database Management System on the DEC-20 to make a new system called MEDAS-R. As a result modification of the knowledge base became much easier. In the new system it is also possible to keep track of the source of all information entered into the knowledge base, along with the data and time of the entry. The new system contains two levels, a knowledge maintenance level and a production level. This design makes it possible to certify new disorder patterns before entering them into the production system. Entering a New Disorder Suppose an expert cardiologist wants to enter a disorder, Pulmonary Thromboembolism, into the knowledge base. The physician looks at the TOOLBOX Screen (Fig. 1) and selects the Knowledge Base Entry option. This brings up the Knowledge Base Entry Screen (Fig. 2) and the physician chooses to Enter/Update Disorder. The Disorder Data Entry Screen (Fig. 3) appears with the first unused disorder number already filled in, but the physician is required to type in the disorder name, the category, the prevalence rate, the severity code, as well as a four line description, which is later used by the explanation and treatment Systems. The physician is now ready to enter the disorder pattern for Pulmonary Thromboembolism. The disorder pattern for a disease lists all the features relevant to the disorder and the associated conditional probabilities. MEDAS must store both p(f/d+), the probability that the feature is present given that the patient has the disorder, and p(f/d-), the probability that the feature is present given that the patient does not have the disorder for use by the diagnostic consultant. 2 The Disorder Pattern Entry Screen (Fig. 4A) allows the physician to enter estimates of these probabilities. The system then calculates the ratio between p(f/d+) and p(f/d-) which serves as a measure of the contribution of this feature. The Disorder Pattern Screen lists features in the order of the feature number. A sort option permits the features in a disorder pattern to be listed in a number of different ways. If the physician wants to check whether a particular feature is present, he/she may ask for the features in alphabetical order as seen in Figure 4B. To check instead whether the probability estimates are consistent, the physician may ask for a screen sorted by feature probability ratios, as seen in Figure 4C. Now that the disorder pattern has been entered, the physician can use the Knowledge Base Test Option in the TOOLBOX to try it out. The Test Option runs a modified version of the MEDAS system using the newly updated version of the Knowledge Base. The physician can enter a hypothetical case or even use a patient record from the MEDAS library of test patients. Cases can be saved and rerun when further changes are made to the Knowledge Base. The following is a hypothetical case. Our patient is a 60 year old, white gentleman with a history of Ischemic Heart Disease (I.H.D.) and Congestive Heart Failure (CHF), admitted to the hospital with a sudden onset of chest discomfort, dyspnea, diaphoresis and non-productive cough. He also has a history of smoking as well as a strong family history of I.H.D. The patient's physical examination indicated a temperature of 98.8 F, regular pulse rate of 116/min., a blood pressure of 160/110, and a respiratory rate of 28/min. His skin is cold and pale. He is sweating. Auscultation of his heart revealed an S4 gallop rhythm. His lungs were essentially clear to auscultation. The patient's electrocardiogram showed ST segment depression in anterolateral leads as well as T wave inversion in the same leads. Arterial blood gases showed a PCO2 of 28 mmhg and PO2 of 68 mmhg. Serum CKMB enzymes were measured and revealed 2 units/liter. The Ventilation/Perfusion (V/P) scan showed multiple segmental perfusion defects with mismatched ventilation. Impedance Plethysmography of the lower extremities showed evidence of deep vein thrombosis. These findings are diagnostic of pulmonary thromboembolism due to deep vein thrombosis. To our expert's dismay, the first test case run (using MEDAS-R) is diagnosed by the system as having a Myocardial Infarction instead of a Pulmonary Thromboembolism (Figures 5A,5B,5C). To determine the reason for the "wrong" diagnosis, the expert can call up a Disorder Differentiation Report (Fig. 6). This report compares Myocardial Infarction and Pulmonary Thromboembolism in detail showing the contribution of each feature. If the difference between the probability ratios of a given feature is small, then the feature is not useful for discriminating between the two disorders. If the discriminating features are not sufficiently robust, the Knowledge Base author should add one or more new features to the disorder pattern. The physician researches the options and identifies that Impedance Plethysmography (IPG) makes an important contribution in the diagnosis of Pulmonary Thromboembolism. 8 A positive IPG in the setting of the V/P lung scan, an embolism has a very high probability

3 of establishing the diagnosis of Pulmonary Thromboembolism and almost equals the diagnostic specificities of Pulmonary angiography, accordingly this is a more invasive test with its added risks, discomfort and cost. In this case, we need to add new features to the Pulmonary Thromboembolism disorder pattern. Entering a New Feature The first step in entering a new feature is to call up the Feature Data Entry Screen (Fig. 7). The system provides the next unused feature number, but the expert must enter the rest of the information. First comes the feature names: V/P lung scan, Mismatched Ventilation, and IPG. The feature category here is "Radiology." Other possible categories are complaint, S/O (setting of), H/O (history of), physical examination, and laboratory tests. The display key indicates where the feature is to be displayed on the screen when MEDAS is being used as a diagnostic consultant. Features including age, sex, and race are entered when the patient is admitted and cannot be changed; these features are classified as "admission-only features". When a feature is marked as "assumed negative", the system assumes that the feature is assumed to be absent (negative) unless the user consulting the system specifies that it is present (positive). The description is used by the explanation system. The cost code is used for measuring the cost associated with collecting this information. The current system is using numbers from one to five, this range could be changed for more elaborate measurement of cost and quality control. In this implementation, for example, an invasive and possibly dangerous test like a pulmonary angiogram has a cost code of 5, while a non-invasive test like IPG will have a cost code of 2. Our expert is also interested in the feature Dyspnea. This feature is already in the system. For what other diseases is this a relevant feature? What other disorder patterns does it belong to? The expert can find this information in the Feature Pattern Report for Dyspnea (Fig. 8). The system can produce such a report for every feature in the system. Since there are more than 700 features, this information is not stored in the Knowledge Base. Instead, it is computed whenever the medical expert calls for it. Whenever a new feature is entered it is very important for the expert to also consider the way it interacts with other features and enter this information into the system as well. To do this the physician calls up the Feature-Feature Interaction Entry Screen (Fig. 9). All related features should be entered on this screen along with the nature of the interaction. To give a simple example, whenever the feature "female" is marked as present (+), the related feature "male" must be marked as absent (-). Conversely, when female is marked as absent (-), "male" must be marked as present (+). In addition, the S/O abortion must be marked as absent (-). IPG, therefore, with regards to our hypothetical case, interacts with the past treatment of Deep Venous Thrombosis (DVT), as well as venous insufficiency. It is at this time then, that the physician can recall the same patient and enter (mark) the newly created feature. This in turn would make it possible to compute the probabilities with these modifications as shown in Figures 5D and 5E. Comment We have described a set of knowledge engineering tools for the pattern-based system MEDAS, that help the physician to add new diseases and symptoms, and detect inconsistencies in the knowledge base. The physician can also insert or correct probability estimates as the medical literature brings new information and broadens the physician's knowledge. Currently we are using the MEDAS-R in conjunction with EMYCIN in our 4 week senior elective, which presents to our student the use of computers in medical diagnostics. The new tools provide the students time to work with Disorder and Feature interactions and not be bogged down with the technical aspects of man-machine interface. We have many plans for the future of MEDAS. The appearance of high performance low-priced microcomputers has made the development of a micro version of MEDAS a high priority, with the aim of making the system cheap enough so that every hospital emergency room can afford one. We are studying the internal strucutre of our treatment protocols with the aim of developing an entity-relationship model and then a relational database for treatment information to work with MEDAS. We hope to compare the performance of MEDAS using probability estimates derived from large public domain database like the National Stroke Registry with its performance using the subjective estimates of physicians as now.

4 Acknowledgement ACCENT-R is a trademark of National Information Systems, Cupertino, CA. References [1] Koschmann, T., Evens, M.; Naeymi-Rad, F., Gudipati, R., Lee, C.M., Weil, M.H. "Knowledge Engineering Tools for a Bayesian Diagnostic Consultant." Submitted to Ninth Annual Symposium on Computer Applications in Medical Care, [2] Ben-Bassat, M., Carlson, R.W., Pun, V.K., Davenport, M.D., Schnver, J.A., Lahf, M., Smith, R., Portigal, L.D., Lipnick, E.H., and Weil, M.H. Pattern-Based Interactive Diagnosis of Multiple Disorders: The MEDAS System," PAMI-2, No. 2, March, 1980, [3] Buchanan B., and Shortliffe, E. (Eds.). Rule-Based Expert Systems, Addison-Wesley, Reading, MA, [4] Davis, R., and Lenat, D. Knowledge-Based Systems in Artificial Intelligence, Mc- Graw-Hill, New York, [5] Ben-Bassat, M. "Multimembership and Multiperspective Classification: Introduction, Applications, and a Bayesian Model," IEEE Transactions on Systems, Man, and Cybernetics SMC-10, 1980, [6] Ben-Bassat, M., Carlson, R.W. Puri V.K., and Weil, M.H. "A Hieraichicai Modular Design for Treatment Protocols," Meth. Inform Med., Vol. 19, No. 2, April, 1980, [7] Koschmann, T., Solomon, D.. Naeymi-Rad, F., Evens, M., Weil M.H., and Rackow, E.C. "Relational Storage Techniques as Applied to a Medical Expert System," Proc Conference on Intelligent Systems and Machines, Rochester, Michigan April, 1984 [8] Hull, R.D., et.al. "Pulmonary Angiography Ventilation Lung Scanning and Venography for Clinically Suspected Pulmonary Embolism with Abnormal Perfusion Lung Scan," Annals of Internal Medicine, Vol. 98, No. 6, June, 1983,

5 Figures Mar. 4, 1985 <<MEDAS KNOWLEDGE BASE TOOBOX>> Screen No. 9:32:35 AM Developed by UHS/CMS Computer Center MEDMOO 1 -- Look at Help and Background Information 2 -- Knowledge Base Entry KnowledgeBase Report Knowledge Base Convert. S -- Knowledge Base Test (MEDAS R) Treatment Entry Treatment Reports Exit Select one of the above. > _ FIGURE #l Mar. 4, 1985 <<MEDAS KNOWLEDGE BASE TOOBOX>> Screen No. 9:32:35 AM Developed by UHS/CMS Computer Center MEDM Enter/Update Disorder Enter/Update Fedfure Enter/UpUdte Probability, 4 -- Enter/UpddLe Fedture Interd~tion Exit. Select one ol the above. > _ FIGURE #2 Mar. 4, 1985 <<DISORDER DATAENTRY>> Screen No. 9:32:35 AM Developed by UHS/CMS Computer Center MEDS10 Disorder # (or STOP): 12 1) Category #: I CNAME. RESPIRATORY 2) Intl disease code: 415,1 3) Disorder name: PULMONARY THRDM80EM80LISM 4) Prevalence rate:.020 S) Severity code: 4 6) Descr 1 : EMB0LISM OF THE PULMONARY ARTERIAL 7) Descr 2 : VESSELS RESULTING FRDM VENOUS 8) Descr 3 : THROMBOSIS WITH ORWlTHOUT PULMONARY 9) Descr 4 : INFARCTION. Do you have any changes? (# / N) >> FIGURE #3

6 FIGURE #4A Disorder Pattern Entry Screen Showing Features in F# Order. FIGURE #4B Disorder Pattern Entry Screen Showing the Features in Alphabetical Order, by Feature Name

7 FIGURE #4C Disorder Pattern Entry Screen Showing the Features in Ratio Order FIGURE #5A Diagnostic Consultant Screen from MEDAS FIGURE #5B Justification for the Posterior Probability for Acute Myocardial Infarction

8 FIGURE #5C Justification for the Posterior Probability of Pulmonary Thromboembolism FIGURE#5D The Result of the Diagnostic Consultant Screen After a New Disorder is Added for Pulmonary Thromboembolism FIGURE #5E An Explanatlon for Pulmonary Thromboembolism Diagnosis

9 FIGURE #6 This Report has Three Sections: 1) Features that are Found in Common Between the Two Disorders 2) Features in the First Disorder, and 3) Features in the Second Disorder FIGURE #7

10 FIGURE #8 Feature Pattern Report for DYSPNEA FIGURE #9 Feature-Feature Interactlon Entry Screen