1 The Intelligent Interface for On-Line Electronic Medical Records using Temporal Data Mining Susan E. Spenceley James R. Warren University of South Australia, Warrendi Road, The Levels, South Australia 5095, Australia. Abstract This paper presents work that has been conducted towards predicting user input requirements with view to making an intelligent interface to support data input. The principles are considered within the context of an on-line electronic medical record system where it is particularly valuable to have intelligent support. The intelligent interface involves the incremental data mining of existing records. The information mined from the data is used to predict current input requirements to improve, among other things, the efficiency of data entered for that user. The use of temporal information in predicting requirements is explored using a case specific and a case independent approach which respectively involve maintaining histories of past treatments and abstracting intervals of problem duration to predict medication. The results show that the case specific approach is very useful although predictions are limited exactly to what has already been encountered in the patient history. The case independent approach is more useful and compares favorably with a simple crosssectional model. Further work is suggested to investigate incremental temporal data mining. 1. Introduction This paper explores how data mining may be used to model user input requirements subsequently using the model in an intelligent interface to support data entry. This is especially relevant in on-line record systems and is explored in the context of an electronic medical record system. The paper is significant in the incremental nature of the proposed data mining technique (since data mining is used to model the requirements of a particular user as University of South Australia, Warrendi Road, The Levels, South Australia5095, Australia. they develop over time) and also in the extraction of temporal information from the medical data itself, using the domain specific characteristics of various disease and medication progression. The application of data mining techniques to support data entry in the medical domain is also particularly important as the medical world has hardly begun to utilize technology. Enabling efficient, accurate creation of medical records is just one way that medicine can benefit from technology. This paper explores two approaches to identifying and using temporal information in medical records to support the development of the treatment plan. The relationship between problem diagnosis and medication is used to predict a relevant drug therapy. Previous work has utilized past patterns of user interaction to derive a predictive data entry model , ,  in a cross-sectional way. This work considers temporal data mining using both a case specific and case independent approach. The case specific approach uses an individual patient history to determine what the input requirements might be during another visit of the same patient. The case independent approach involves abstracting problem intervals from individual visits and predicting the medication from this interval based data. This paper first considers the way that data mining techniques have been used to build models that assist with predicting data entry requirements (2) particularly mentioning a non-temporal statistical analysis used to predict data entry requirements in echocardiography. Following this the temporal information in general purpose clinical data is considered (3) and experiments described to describe how temporal information within the data itself has been used to derive a temporal model (4). Subsequently the question of intelligent interfaces are considered (5) and the particular way that the intelligent
2 interface might be realized in an electronic medical record system (6) before concluding (7) with some suggestions for further work that will augment the temporal data analysis. 2. Data-mined models to anticipate input requirements Data mining includes a variety of techniques. Some of these techniques have already been applied to the anticipation of user input requirements. In particular rule induction, artificial neural networks and some statistical models have been utilized. However none of these models utilize the temporal component of the data as is considered in this paper and none suggest using incremental data mining as the input examples are built up in the database. Hermens and Schlimmer  demonstrated how rule induction may be used to assist on-line form filling. The rules are automatically induced from the data and relate different items on the form. From these rules it is possible to predict the user s probable preferences in input. Test results demonstrated that nearly 90% of the entries on the form were correctly predicted and there was an 87% saving in keystroke effort. The artificial neural network has also been used to help the user complete a task when using the Unix operating system. Typically there are many different ways to complete the task and some set of optimal commands. Eberts et al  demonstrated that the artificial neural network can be used to associate system commands with user tasks. After having learnt the optimal associations user help could be provided to make interactions more efficient. This model also went beyond simply predicting what the input requirements were and extended to anticipating what the optimal set of might be. list is in alphabetical order. The contents of the intelligent portion will change according to context. In terms of data entry effort Canfield compared the intelligent menu, the alphabetical menu and the intelligent split determining that the intelligent menu was actually 5.5 times faster in terms of data entry than the alphabetical menu while the intelligent split was only 4.5 times faster. However users preferred the intelligent split menu (perhaps because of the consistency of the alphabetical portion combined with the variability of the top portion adapting according to context). This result supports that found by Sears and Schneiderman  who investigated user preference between alphabetical menus and frequency lists. They found that users preferred a combination in the intelligent split menu. Previous work by the authors has extended the investigations made by Canfield by broadening the investigation to the general practice medical domain ,  considering both the anticipation of treatment from diagnosis and diagnosis from symptoms. The work also considered a variety of statistical models  where the predictive power of conditional probabilities, multiple linear regression and discriminant analysis were compared in the context of predicting a treatment from a diagnosis. The model was employed in a recognition process where the predicted items were placed in the hotlist of an intelligent split menu. Figure 1 illustrates the intelligentsplit menu. Figure 1. The intelligent split menu with hotlist and alphabetical list Conventional statistical models have also been used to help predict input items. Canfield  has considered how frequencies derived from associations in medical data can be used to prime intelligent menus in the echocardiography domain. Canfield was particularly concerned with how the model predictions would be utilized to measure the improvement in input effort. The work explored a number of different types of menu including the intelligent split, alphabetical list and categorical lists. The intelligent split is organized so that the top few items are ordered according to potential relevance in the context while the remaining portion of the
3 Time is an important dimension to consider particularly in a system that is essentially building up longitudinal records. A particularly good example of such a system comes with medical records (where the patient is typically followed through for many years). This work will consider how temporal information might be used to help predict input requirements particularly focusing upon temporal information in the medical domain. 3. Temporal information in clinical data The existence and use of temporal information has been noted in many areas and medicine is one particular domain where such information is particularly useful. Temporal modeling has been explored to support both medical expert systems  and to assist in electronic medical record systems . It has chiefly been used in the management of data and its display rather than in clinical data entry. Additionally the type of data typically tends to be results from specific laboratory measurements rather than the general variety of data that is encountered in general practice medicine. Data from general practice typically consists of a number of qualitative measurements for diagnoses, symptoms and treatments for each visit of a patient rather than specific readings of laboratory tests. The temporal component of general practice medicine poses a particular challenge since the data might reflect a series of independent unrelated visits (with different symptoms, diagnoses and treatments) perhaps many months apart or be related to an on-going condition, perhaps extending over years, or possibly over days. As far as temporal modeling is concerned the interval is an important model of time contrasting to the point. The interval extends between and perhaps over time points. As with the point the interval might be relative (e.g. to the first visit or first diagnosis of problem) or absolute (e.g. measurement is absolute between particular days in the year). The interval is an important piece of information particularly in general practice medicine where not only the set of symptoms, diagnoses and treatments at a given visit are of interest but also those in the history. In particular a treatment might be prescribed at one visit which still continues when the next visit occurs; however, it might not specifically be mentioned again because it is not being prescribed at that visit. The problem might even be persisting although the patient has gone for another unrelated reason and this is the reason that is recorded. Considering the patient visits as isolated events potentially looses the most persistent problems within a patient s experience and the most frequently prescribed (and represcribed) drugs for individuals. The temporal abstraction of time-oriented clinical data has been considered in a variety of medical contexts. Temporal abstraction is the task of abstracting high-level, interval-based concepts from time-stamped data . In the context of clinical data from general practice records abstraction would involve deriving an interval for a particular problem or treatment or symptom. The interval might be relative (e.g. between visits 3 and 5) or absolute (e.g. over 5 months). This paper proposes to utilize the temporal information within the clinical data by making use of a case specific and a case independent data mining analysis. The case specific analysis will simply utilize the patient s history in order to determine appropriate treatment patterns. The case-wise frequency of medications will be the most significant factor in determining future input requirements. The case independent analysis will make a temporal abstraction of problem intervals upon which predictions of medication are based. This is independent of any particular medical case. 4. Method and results 4.1 The data A database of 3085 records collected over a 1 year period was obtained from a general practice in South Australia which used an electronic form of record note keeping. The note keeping system accepts textual notes for a given patient including a locally used problem code and a treatment. The anonymity of the records was respected and information extracted from the record solely for the purpose of modeling the relationship between problem diagnosis and treatment. Of the various types of treatment that are possible the prescription of medication was actually modeled. There were 352 different problem codes utilized in the records out of some 500 codes available in the scheme employed and some 466 different drug keywords identified in the treatment field. Some of these drug keywords were generic names and some were brandnames. A manual mapping of brandname to generic name was made, correcting for typing errors, resulting in 370 generic drug names. The task was to explore the relationships in existence between the 352 problems and the 370 generic drugs.
4 4.2 The method of analysis Data was divided into training and test sets. The training set was used to build a predictive model of drugs from problem codes. The test set was effectively unseen data that could be used to evaluate how well the model generalized and predicted the drug prescriptions of new problem combinations. Two different methods of deriving a model from the training data were considered being the case specific and case-independent analysis. Table 1. Training and test set composition Number of patient visits (n) No. of cases in the test set No. of cases in the training set n = 2 visits n = 3 visits n = 4 visits n = 5 visits n = 6 visits n = 7 visits n = 8 visits n = 9 visits n = 10 visits n = 11 visits n = 12 visits n = 13 visits n = 14 visits n = 15 visits n = 16 visits Testing involved measuring certain aspects of performance for the model. A hit-rate was computed involving a ratio of the number of drugs that were actually predicted and required over all those that should have been predicted. Hit-rates were determined using up to 30 drug predictions of a model. With 15 predictions under consideration the model effectively has up to 15 chances to make an accurate prediction. A range is considered rather than just the first drug because there are occasions when there should be several candidate drugs and ideally the model will predict them. While the topmost predictions might be valuable if the model were used to generate an intelligent menu, even later predictions might be valuable in disambiguating vocal or handwritten input. The database of 3085 cases from 800 patients was divided up into 15 training and test sets. Each training set consisted of records from patients who had n visits (from n=2 to n=16). The most recent visit is always excluded from the training set to be part of the test set. The frequency of number of patients in each set and the actual size of that training set is presented in table Case independent temporal analysis Case independent temporal analysis aims to build a general model from the data. There are a number of ways that this might be done. The approach taken here utilizes a form of temporal abstraction to infer a range of problem duration from a set of visits. Temporal abstraction involved identifying those problems that occurred more than once in a patient history and making an interval by noting the first and last occurrence of a problem. The inferred range of problem duration is used to modify the co-occurrences between problem and drugs. The problem is assumed to be present in the whole interval and the cooccurrences with drugs consequently altered. The modified co-occurrences between problems and drugs are then used to derive a temporally abstracted probability model Figure 2. Difference between performance for temporally abstracted and non-temporally abstracted training sets Hit-rate difference n=2 1to6 hotlist n=4 n=6 n=8 n=10 7to15 hotlist n=12 n=14 n=16 patients with visits n=2 to n=16 The temporally abstracted probability model was compared with a non-abstracted probability model for the same training and test data. The difference in hit-rates was computed for patients with n=2 to n=16 visits. This was done for upto 12 model predictions. A negative difference between the hit-rates suggests that the temporally abstracted probabilities produced better results. A difference of zero occurs when n = 2 visits for a patient
5 since the historic data contains just 1 case from the patient and there can be no temporal abstraction over just 1 case. The average difference in hit-rate for predictions 1 to 6 was computed and compared with the average difference in hit-rate for predictions 7 to 15. Figure 2 plots the difference in hit-rate between the models for each of the training sets (n=2 visit to n=16 visits). Temporal abstraction does not make an improvement when the top few model predictions are considered (i.e. the top 6 predictions) however, there is evidence of some improvement when the second best predictions are considered (i.e. 7 to 15). In particular this occurs when temporal abstraction is made for patients who have more than one visit per month. For patients with a large number of visits there is even an improvement in hit-rate for the top model predictions. The actual scale of the improvements are in the order of 5%. It is possible that patients with many visits within the year typically have persistent problems (where inferring the interval is particularly valid). 4.4 Case specific temporal analysis The second experiment in examining the temporal component of the data considered case specific temporal analysis. The approach uses patient history as the means by which the drug predictions are made. The frequency with which a drug has been prescribed to a patient in the past is used to predict what should be prescribed at a given visit. In particular patients who had 8 visits are considered and for each a simple patient history list is maintained of the treatments that they had been prescribed for the previous 7 visits. A comparison was made directly with the hit-rates that were achieved for the case independent temporal model in the temporally abstracted probabilities and both temporal models compared with the cross-sectional probability model. Figure 3 compares the actual hit-rates achieved from these three different methods for hotlist sizes ranging from 1 to 30. The results show that individual patient history gives the best results when the top ranked predictions of the hotlist are considered. However, it would not be very useful for the first visit of a patient and also it can never predict drugs that have not already been prescribed for a particular patient (hence the eventual leveling off of the hit-rate). In contrast the probability models demonstrate the ability to predict beyond what has already been encountered and so would be useful for first visit patients and demonstrate improved performance as more model predictions are considered. Figure 3. Comparison of hit-rates for a patient specific model and probability model using nontemporal and temporally abstracted data. Hit-Rate probability temporal history Intelligent interfaces Size of Hot-list In this work the intelligent interface is defined as an interface in the most general sense between the user and the computer system encompassing conventional windows-based keyboard and mouse workstations, penbased notebook computing, voice recognition, handwriting interpretation and other non-conventional forms of interaction. The interface might be necessary for a number of tasks ranging from entering information to an on-line database, posing a query into a knowledge-base or selecting menu options to navigate through a program. Some features of intelligent interfaces include the following : the intelligent interface is designed to account for differences between the input requirements of users, anticipating those requirements and making data entry more accurate and efficient. An interface may be tailored to a particular task, particular users or groups of users. the intelligence of the interface is gained from a model that is automatically derived from system usage that is encountered. The model is a predictive model of what the input requirements will be in a given context. the model may be rederived over time. the model may be exploited in a variety of ways to assist input including context sensitive menu generation, keyword completion during typing,
6 disambiguation in on-line handwriting and voice recognition, instantiation of default entries and provision of recommendations for particular input combinations to support the user with a given task. The intelligent interface shares some similarities with the adaptive interface but remains distinct. Benyon  introduces the idea of an adaptive interface in its fullest. It is essentially an interface that is able to adapt according to the user s level of competency with a system enabling there to be a range of different modes of system operation ranging from novice user to expert. To achieve the adaptive interface Benyon suggests that there should be an explicit user model, task model and interface model. The adaptive interface is concerned with the various tasks that different types of users might want to perform and with adapting the interface so that the user is able to perform that task given their current level of system knowledge. In particular the intelligent interface is concerned with a broader range of interaction, with users input requirements rather than level of competency, with a single model of those requirements (rather than an explicit model of user, task and interaction) that relates input items for a given task, for a given user. Both the intelligent interface and the adaptive interface are designed to be incrementally re-derived, to change or adapt over time and both might be regarded as learning. Maes discusses the Intelligent Agent  describing how the Personal Assistant is a type of agent that can learn to emulate user behavior. It uses both explicit instruction and inference. This type of agent that is particularly concerned with user behavior shares some similarities with the intelligent interface in the sense that emulating user behavior requires that patterns of user input are known. The intelligent agent also automatically derives part of this information. However the intelligent agent is a more general concept and does not share the emphasis on maximizing the efficiency of user input once the input requirements are known. The intelligent interface is still a general concept that may find use in a variety of systems, particularly those that involve on-line record creation. In this work it is explored in the context of electronic medical record systems. 6. The intelligent interface in an electronic medical record system Data input to an electronic medical record should be efficient. The physician is typically removed from creating the clinical electronic record because data entry is perceived to be a burden and a clerk employed instead. There has been some work that seeks to completely automate the process of attaining an electronic record. In particular using off-line interpretation of the paper-based record via handwriting recognition  and speech recognition from dictated summaries. These approaches miss the opportunity of supporting the physician in practice. Ideally the physician will create the record being supported in many ways by electronic resources including the following : At the highest level the physician will be supported in practice. For example it will be possible to receive warnings about drug interactions or patient specific allergies. It may be possible to access other on-line resources, decision support systems, analysis and data display packages that all provide assistance and support with decisions and evaluations made during a consultation. The physician will also be assisted with the task of making an accurate complete record. There is a movement towards coding medical concepts with a variety of coding schemes such as READ  and SNOMED  that typically involve thousands of codes. Recently code browsers have been made that assist with the derivation of a code from a textual description . Often these code browsers are used by a clerk to interpret the physician s paper-based record but such third party interpretation is clearly less than ideal. Electronic resources like this may assist the physician in directly translating their practice into a formalized record. Finally, with the intelligent interface the physician would be supported at the lowest level with the data entry task having a system that minimized the amount of time spent physically entering data (because of the predictive model of input requirements). The interface might help with data entry assisting both the recognition and generation components of the task. In some instances recognition of a required input item is an easier cognitive task than generation of that item and the interface might present a list of most likely options from which one is selected. In other instances the generation of the required input item is an easier task and the interface might assist with keyword completion of an item once keying has started, or with identification of a spoken / written word. In this particular application there is an important consideration of how a computer is used by the physician. A study exploring the effect of a physician using a computer within the consultation , concluded that
7 patient disclosure was influenced by the visible presence of the keyboard and monitor. A less obtrusive use of computers is obviously required perhaps via pen-based computing or voice recognition. The PEN-Ivory project  is a particular pen-based system for creating medical records. The project was also particularly concerned with data entry efficiency and considered user efficiency in menu selection tasks. Aspects considered include the positional constancy of menu items, user uncertainty and system anticipation. Canfield  has also evaluated the efficiency with which menu items might be selected during the creation of a record within the echocardiography domain looking at various types of menu. Data input might take the form of generation or recognition. The generation process involves generating an input. This is a cognitively different task to recognizing an input. Generation involves the physician knowing the concept code, drug name, problem diagnosis and entering that information typically by a keying, writing or dictation process. The recognition process involves recognizing the input that is required. Recognition involves detecting that the required input is present perhaps selected from a menu, perhaps confirming an instantiated default entry. A predictive model should be able to assist with both forms of input. Without necessarily constraining the interface to take any particular shape or form a prototype was constructed within a menu-based windows system. The prototype realized the predictions of a datamined model demonstrating the use of three separate menus constructed by temporal analysis, cross-sectional probabilities and alphabetical ordering. The interface was informally evaluated by the general practice from which the data originated with favorable results. Figure 4 illustrates the screen for entering medication. A simulation experiment in the style of Canfield demonstrated that there was a two fold improvement in terms of mouse clicks using an anticipative model . 7. Conclusions This paper has considered the task and important problem of making data entry more efficient particularly in the context of on-line record systems with electronic medical record systems being the chosen information system. The Figure 4. Example of the utilization of temporally mined information in a menu-based interface.
8 interface may encompass many forms of interaction such as pen-based computing, voice recognition, handwriting analysis or windows-based computing. The intelligent interface emphasizes making data entry efficient in the various contexts in which input might occur. In particular that data entry is efficient in a given context for a given user, rather than for a user with a given level of competency (as happens with the adaptive interface). Importantly the intelligence of the interface is gained from a data mined model that is incrementally re-derived over time. The model may be exploited to assist with input in both recognition tasks (e.g. menu selection) and generation tasks (e.g. keyword completion). This work also identified that the temporal aspects of data are important to consider particularly in applications such as on-line medical records where there is a longitudinal component to the data. The temporal information content of the data was investigated in two main ways being case specific and case independent approaches. The case specific approach involved the utilization of simple patient history to predict (for individual patients) what the possible drug treatments might be. This technique gives the best predictive power measured by hit-rate for the top model predictions although is not particularly suitable for new patients who have no history because it cannot generalize beyond what has already been encountered. The case independent approach utilized the idea of temporally abstracting the data to infer a range of problem duration based upon isolated occurrences of problems within a patient history. This range of problem duration was then used to derive a probability model that was modified according to the information introduced by the abstraction. The approach is more general than the case specific approach and compares favorably with the probability model with some slight improvement in patients who have long visit sequences (that suggest a particularly persistent problem where the inference of a continuing interval is valuable). Finally further work is suggested that will continue to explore the temporal information content of the data, particularly focusing upon extending temporal abstraction to make inferences over absolute measures of time (rather than visits), to infer ranges of drug treatment and the integration of the results from case specific and case independent approaches. The resulting model will ideally contain greater predictive power than a simple crosssectional model of input requirements and shall be of value in the intelligent interface. Particularly in an intelligent interface for creating electronic medical records. 8. References  Spenceley SE, Warren JR, Mudali SK, Kirkwood I, Intelligent Data Entry for Physicians by Machine Learning of an Anticipative Task Model, in V.L. Narasimhan and L.C. Jain eds, Proceedings of the Fourth Australian and New Zealand Intelligent Information System Conference (ANZIIS-96), Piscataway, New Jersey, IEEE 64-67,  Mudali SK, Warren JR, Spenceley SE, An Adaptive Two-Tier Menu Approach to Support On-Line Entry of Diagnoses, Proceedings of the Sixth Conference in Artificial Intelligence in Medicine Europe. Springer Verlag, Lecture notes in Artificial Intelligence, 1997 (in press).  Spenceley, S.E., Warren, J.R., and Mudali, S.K Toward the Intelligent Physicians Interface : Use of Diagnosis Codes to Anticipate Drug Therapy. School of Computer and Information Sciences, University of South Australia, Technical Report CIS n  Benyon D. and Murray D., Experience with adaptive interfaces, The Computer Journal, Vol 31, No.5, 1988, pp  Maes, P. Agents that reduce work and information overload. Communications of the ACM, 37 (July), 31-40;  Eberts R, Villegas L, Phillips C, Eberts C, Using neural net modeling for user assistance in HCI tasks, pp 59-77, International Journal of Human-Computer Interaction,  RCC Read Clinical Coding Scheme, Computer- Aided Medical Systems, (CAMS), Leicester.  College of American Pathologists SNOMED III, Systematized Nomenclature of Medicine Version 3, International, Illinois.  Walker, D A new browser for the Read clinical coding system. Healthcare computing,  Greatbatch D, Heath C, Campion P and Luff P : How do desk-top computers affect the doctor-patient interaction?, Family Practice 12(1):32-6, 1995.