User Interface Optimization for an Electronic Medical Record System

Transcription

1 MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, The authors. All rights reserved. User Interface Optimization for an Electronic Medical Record System Kai Zheng a, Rema Padman b, Michael P. Johnson b a School of Public Health and School of Information, University of Michigan, U.S.A. b H. John Heinz III School of Public Policy and Management, Carnegie Mellon University, U.S.A. Abstract Many information technology-enabled healthcare applications have failed because their interfaces are difficult to use. Unfortunately, little attention has been paid in the health informatics community to designing effective user interfaces that are acceptable to healthcare professionals. This paper illustrates a method for improving application interface usability by applying sequential pattern analysis to analyze temporal event sequences recorded in an electronic medical record system. Such event sequences, or clickstreams, reflect clinicians navigation patterns in their everyday interactions with the computer system. The identified patterns have been used by software developers to calibrate the user interface of the system, so that the within-application workflow is better aligned with clinicians mental model of medical problem-solving. Such inferred patterns may also help to modify clinicians suboptimal practice behavior components, as manifested through their actual usage of this point-of-care electronic system. Keywords: user-centered design; user interface design; sequential pattern analysis; human-computer interaction; usability assessment; data display Introduction Medical practice is a complex process. Large amount of data must be accessed, assembled, and analyzed at the point of care to inform proper medical decision-making. In the era of paper-based patient records, clinicians flip through stacks of paper charts to look for desired information. The use of electronic systems has greatly facilitated health data retrieval. However, it has also introduced new dimensions of problems. Two paper documents, for instance, can be laid out side by side for cross reference, while on a computer screen it is usually impractical to have two windows visible at the same time. How to preserve the easy look-and-feel of paper charts is a real challenge for software developers. In addition, poorly designed application navigation flow may also escalate learning effort, decrease productivity, and increase user errors [1, 2]. Lack of good user interfaces has been long recognized as a major impediment to the acceptance and routine use of clinical informatics applications [3]. Unfortunately, very few research studies have looked at design principles for building intuitive and effective healthcare user interfaces (UI); even fewer have validated the usability of existing UI design in realistic clinical settings. Consequently, systems are created ad hoc, users are dissatisfied, and often systems are abandoned [2]. The present study was motivated by these facts. The computer system in question, the Clinical Reminder System (CRS), is a lite electronic medical record system (EMR) that collects, stores, and manages a wide range of patient and clinical data [4, 5]. In addition to its regular EMR functionalities, CRS is also intended to improve quality of care by providing clinicians just-in-time alerts and advisories using evidence-based guidelines. Since 2002, CRS has been deployed in an outpatient clinic at an urban hospital, and used by clinicians to treat patients in real time. While user, task, and representational analysis were performed during the software design phase with constant feedback by participating clinicians, its UI design was still critiqued after being routinely used in clinicians everyday practice. In a user satisfaction survey following a 10-month field trial, users complained that the application s early user interface, shown in Figure 1, provided little guidance as to a desired workflow [4, 5]. As a result, user acceptance was not satisfactory, and the utilization rate of the system remained low [4, 5]. Although this UI reflected the best knowledge of developers and preferences of the client organization, the standard Windows-based layout was reported as not aligned with our common practice styles. The horizontally arranged tabs, for example, did not reflect the preferred order of clinical information access. As a result, users expended substantial energy unnecessarily to adapt their practice to a UI design that they considered uncomfortable. Selected for best paper award. 1058

2 Figure 1 - An early user interface To solve the identified UI flaws, the system was reengineered into a full web-based application. A screenshot of the new web interface is shown in Figure 2. Unique features of the web-enabled application provide tremendous promise for maximally preserving the look-and-feel of traditional paper charts. In the new design, for example, different features conveying different clinical information elements are no longer arranged in a tabular form, instead, they are displayed in the same workspace that can be easily navigated by mouse scroll wheels, simulating paper-flipping behavior. A navigation menu is also provided on an adjacent frame to enable fast switches across different features. However, it is not known whether this new design is consistent with, or represents an improvement upon, clinicians typical workflow. The study reported in this paper was therefore conducted to identify the preferred sequential order in which different features of the system are accessed. To learn clinicians navigation behavior, this study uses a sequential pattern analysis method to analyze actual usage recorded in the computer logs that contain time stamped events. Actual usage data, unlike many software usability experiments, represent users interaction with a system under real working conditions, rather than on contrived laboratory exercises. Methods Sequential pattern analysis Sequential pattern analysis discovers hidden and recurring patterns within large sequences of events. It has been applied in a wide variety of domains such as web person specialization and page recommendation [6], HCI usability testing [7], and genetic sequence analysis [8]. In this study, a consecutive sequential pattern algorithm is employed to analyze the event sequences recorded in CRS. This algorithm detects consecutively occurring events that Figure 2 - New user interface to be evaluated appear across different sessions. Such patterns, that represent adjacent feature accesses frequently occurring next to each other and in a given sequential order, are of particular interest to inform UI redesign. Let s denote an even sequence by <e 1, e 2,, e n >, where e j, j 1 n, is an event that occurs at the j th position in s. The consecutive sequential pattern algorithm finds a sequence p <p t, p t+1,, p t+l > that is a subset of s, which is also part of, or supported by, other sequences. The support for p is defined as the fraction of total sequences that supports p. When a sequence satisfies a certain minimum 1059

3 support threshold, it is named a Sequential Pattern. The largest length sequential pattern that is not part of any other patterns is called a Maximal Sequential Pattern. The objective of the sequential pattern analysis is to find all such maximal sequential patterns. When the minimum support is a constant for any given length, the most efficient algorithm starts with calculating support for all possible sequences composed of two consecutive events. When a sequence does not satisfy the minimum support, it is removed from further computation; otherwise, it is treated as a candidate sequence to compute support for larger length sequences. The algorithm stops when no larger length sequences based on a current candidate would satisfy the minimum support. The current candidate sequence is then chosen as a maximal sequential pattern. Study site and data collection In this study, 10 months of usage data were electronically collected from October 1, 2005 to August 1, 2006 and analyzed. These usage data were generated from the most recent web-enabled version of CRS. The system implementation was accomplished in the summer of 2005 and substantial training was provided afterwards. The main CRS user population during the study period was composed of 40 first-, second-, and third year internal medicine residents. Residents who used the system for fewer than 5 patient encounters are excluded from the analysis. It is likely that such users interactions with CRS do not reflect mature application usage. 30 active resident users were thus identified, whose system usage was recorded in 973 unique patient encounters. same feature. For example the segment MMM, prescribing multiple medications consecutively, is collapsed into one single event M. In this study only across-feature navigation is of interest, that is, jumps across different features. Figure 3 shows the distribution of event sequence length after the collapsing operation. The sequences composed of 4 or less events are excluded from further data analysis because they provide little information in regard to sequential navigational patterns. This operation results in the loss of 6 additional users whose recorded sequence lengths are all below 5. After these data preparation procedures 473 event sequences are retained, generated by 24 distinct resident users. Distribution of number of sequences owned by each user is depicted in Figure 4. Several sample event sequences are shown below: HMOMYXAM GHXVHADADHA HGYXADAOMYSX OMRHFYXYXADADA HXOPMOMOMOMODADAM HSXDADADADADAMOMOMOMOMO S e q u e n c e L e n g t h Data analysis and results Data preparation Data preparation procedures were performed prior to the analysis. All events and their affiliated attributes, such as session ID and time stamp, were first collected from scattered data tables. The event type was then mapped based on a labeling schema, which is composed of distinct letter symbols. Table 1 lists all 17 main features 1 that the CRS application provides, ordered alphabetically by their labeling symbols 2. The screenshot shown in Figure 2 illustrates the on-screen positions of each of the 17 major features. Event sequences were then constructed. HMMMYAD, for instance, is a 7-length sequence composed of 7 events that occurred within a patient encounter, ordered chronologically by their time stamps. The resulting event sequences are further consolidated by collapsing repeating access to a Figure 3 - Distribution of event sequence length Figure 4 - Usage distribution among users Frequency of feature access Table 1 shows the aggregated proportion of feature accesses 3. These proportions roughly represent how frequently each application feature was used. As shown in Table 1, among the 17 major features Assessment and Plan, Diagnosis, and Medication were most heavily used. Note that while Encounter Memo appears on top of screen, it was seldom accessed. 1 Feature that must be displayed in a certain position for legal reasons, such as patient s demographics always appearing on top of an encounter page, is excluded from the consideration of this study. Also excluded are non-actable or not yet activated features, for example Reason for the Visit that is entered by nurses when a patient calls to make an appointment. 2 A symbol letter is usually the first letter of a feature unless there is a conflict. 3 Repeating access to a same feature is counted only once

4 Table 1 - Main features and overall frequency of access Label Feature Proportion (%) A Assessment and Plan B Retaking BP.34 D Diagnosis E Medication Side Effects.22 F Family History 1.24 G Allergies 1.88 H History of Present Illness 7.26 L Laboratory Test 3.58 M Medication O Order P Procedure.38 R Encounter Memo.44 S Social History 2.85 T Office Test.62 V Vaccine.83 X Physical Examination 6.69 Y Review of Systems 4.43 Table 2 shows the results of the sequential pattern analysis. All maximal sequential patterns included in the table satisfy a minimum support threshold of 15%. These patterns are sorted by the level of support they received. Table 2 - Maximal sequential pattern discovered Maximal Sequential Pattern Level of Support (%) ADAD DADA XADA OMOM MOMO YXAD HS OL 18.6 OMY 16.7 LO HO Some interesting sequential patterns emerge from Table 2. ADAD, appearing in 51.16% of all encounters, is the most salient pattern discovered, followed by a similar and partially overlapped pattern DADA, with 43.97% support. It indicates that the users of CRS frequently switched between the features Assessment and Plan and Diagnosis. Similarly, users frequently switched between Order and Medication, with 35.1% support for OMOM and 27.06% support for MOMO; and Order and Laboratory Test, with 18.6% support for OMOM and 15.64% support for MOMO. A further examination found that A precedes D more often (89.18%) when a user entered the AD...AD or DA...DA segment. Similarly, O was usually accessed before M (72.57%), and before L (71.58%). Supported by 40.17% of all encounters, ADA is preceded by X - Physical Examination, and YXAD appears in 21.78% of time. This indicates that Physical Examination Assessment and Plan Diagnosis is a frequently traversed path, which is often preceded by accessing Review of Systems. Further, OMY occurs in 16.7% of all sequences, indicating that OM - Order and Medication were often used before Y - Review of Systems. HS - History of Present Illness then Social History and HS - History of Present Illness then Order, are two other consecutive patterns with slightly smaller support, 19.03% and 15.01%, respectively. An ad hoc within-sequence analysis was further conducted to detect sequence segment recurring within an encounter session. Results are shown in Table 3. The Probability of Repeat in Table 3 exhibits the probability of a two-length event segment recurring within a sequence. DA or AD - Diagnosis and Assessment and Plan, OM or MO - Order and Medication, and OL or LO - Order and Laboratory Test, are three frequently repeating segments thus identified, which also confirm the cross sequence patterns of DADA, ADAD, OMOM, MOMO, OLOL, and LOLO. Because items in these reappearing sequence segments were usually accessed next to each other, they are hereby referred to as Bundled Action. Table 3 - Recurring patterns within encounters Sequential Pattern Probability of Repeat (%) AD MO OL DA OM LO The repeating access to bundled actions, however, blurs the boundary of jumps from a series of bundled action accesses to other features. For example the reappearing AD with varying length in the sequence HADAD...ADADXY impairs the analytical power for discovering whether there exists a pattern H-AD-Y that may help reveal interesting patterns at an overall level. Similar to collapsing repeating access to the same feature, repeating access to the same bundled action is further collapsed to count as one single occurrence. For example the HDAD...ADADAXY sequence is converted into HDAY to form a new, higher level sequence. A second pass sequential pattern analysis was then conducted to analyze the event sequences obtained after this collapsing operation. ADO - Assessment and Plan to Diagnosis to Order is the only additional sequential pattern thus identified, supported by 15.64% of all encounters. This pattern indicates that after a user finished working on Assessment and Plan and Diagnosis, he or she would switch to the Order section immediately to prescribe orders of new medications or laboratory tests. Discussion Based on the findings from analyzing actual usage data with sequential pattern analysis, several UI design principles can be arrived at: Encounter Memo should be properly relocated. This feature is less frequently used while occupying the most salient position in the current design; 1061

5 Assessment and Plan, Diagnosis, and Medication are the most frequently accessed features. They should be placed in the most salient positions on a computer screen; Assessment and Plan and Diagnosis, Order and Medication, and Order and Laboratory Test are bundled actions. They are usually accessed next to each other and often used multiple times within an encounter session. Navigation aids such as hyperlink shortcuts should be provided to facilitate these frequent feature switches; Review of Systems, Physician Examination, Assessment and Plan, and Diagnosis should be presented adjacent to each other in this sequential order. Accesses to these four features often appear as a series of events occurring sequentially. These design principles have been used in redesigning the existing user interface of CRS. Since the basic EMR functionalities that CRS provides are universal, these design principles may also be applicable to other electronic medical record systems. Conclusions Improving the UI design of an electronic medical record system can be successfully attained by analyzing the actual usage data recorded during its everyday use. The sequential patterns identified in this paper led to a set of design principles used in redesigning the application s user interface. These design principles mainly propose that different clinical information elements should be presented in the sequential order in which they are usually accessed, which reflects clinicians mental model of medical problem-solving during patient encounters. This study has a few limitations. First, actual usage data must be collected from a working system. Its current design, inevitably, may exert an influence on users own working style. Second, the findings are derived from testing a single system with certain unique features. While the method and the results provide general insights into designing user interfaces for other types of health applications, they may not be used without careful customization. Finally, the user population of this study was mainly composed of internal medicine residents. The derived design pattern reflecting their practice style may not be generalizable to other clinical specialties. References [1] Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, Onderdonk A, Wybenga D, Winkelman J, Brennan TA, Komaroff AL, and Tanasijevic M. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999; 106(2): [2] Tang PC and Patel VL. Major issues in user interface design for health professional workstations: summary and recommendations. Int J Biomed Comput 1994; 34(1-4): [3] Johnson CM, Johnson TR, and Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005; 38(1): [4] Zheng K, Padman R, Johnson MP, Engberg J, and Diamond HS. An adoption study of a clinical reminder system in ambulatory care using a developmental trajectory approach. Medinfo. 2004; 11(Pt 2): [5] Zheng K, Padman R, Johnson MP, and Diamond HS. Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. Int J Med Inform 2005; 74(7-8): [6] Montgomery AL, Li SB, Srinivasan K, and Liechty JC. Modeling online browsing and path analysis using clickstream data. Marketing Science 2004; 23(4): [7] Siochi AC and Ehrich RW. Computer analysis of user interfaces based on repetition in transcripts of user sessions, ACM Transactions on Information Systems 1991; 9(4): [8] Eres R, Landau GM, and Parida L. Permutation pattern discovery in biosequences. J Comput Biol 2004; 11(6): Address for correspondence Kai Zheng, kzheng@umich.edu, M3531 SPH II, 109 S. Observatory, Ann Arbor, , U.S.A. 1062

6