Evaluating the Usability of the OpenVista Electronic Health Record EHR

Transcription

1 ABSTRACT Evaluating the Usability of the OpenVista Electronic ealth Record ER Eliz arkowitz School of Biomedical Informatics, The University of Texas ealth Science Center at ouston 8181 Fannin Street #612 ouston, TX 77054, USA In the midst of pressing demands for viable Electronic ealth Record (ER) technology, edsphere OpenVista offers a Graphical User Interface (GUI) system that strives to provide such technology across a variety of healthcare organizations and settings. While the overall structure and function of OpenVista may improve overall performance, in terms of cost reduction, clinical performance, and patient care outcomes, there is still much room for improvement in OpenVista s user interface. Via the application of user interface design methods and techniques, recommendations for the overall improvement of the OpenVista Patient Summary screen were made and a low-fidelity prototype was created. The goal of the new patient summary screen was to create a user environment that will optimize user performance via reduction of time spent acquiring data, improving the ease of data acquisition, and clarity of data. The two patient summary screens were compared via a L analysis and the low-fidelity prototype screen was found to increase ease of use and efficiency. Specifically, OpenVista required, on average, 60% more mental operations and 1% more time per task than the prototype. Consequently, while ERs are quite valuable, poor design can lead to extensive usability issues amongst both novice and expert users. While OpenVista is quite prevalent amongst many hospitals utilizing ERs, there is still much room for improvement. eywords Design, uman Factors, OpenVista, Usability, Electronic ealth Record (ER), eystroke-level odel (L) 1. INTRODUCTION 1.1 OpenVista: Electronic ealth Records (ERs) are viewed by many as useful tools for improving and supporting the provision of high quality healthcare. [1] While there is currently no standard ER at the moment, the ER aims to digitally record health information about specific patients or populations. OpenVista, one of the most prevalent ER systems currently in use, derives from the Veterans ealth Information Systems Technology Architecture; an ER used and developed by the United States Department of Veterans Affairs (VA). [2] In the midst of pressing demands for viable Electronic ealth Record (ER) technology, edsphere OpenVista offers a Graphical User Interface (GUI) system that touts use across the continuum of acute, ambulatory, and longterm care environments as well as in multi-facility, multispecialty healthcare organizations. [] Despite the fact that the VA and many other organizations successfully utilize OpenVista, the user interface leaves much to be desired in terms of usability and design. 1.2 Design Guidelines for a GUI: A GUI allows a user to interact with an application via direct manipulation. The most important aspect of healthcare applications revolves around usability. Usability, as far as GUI design is concerned, encompasses ease of use, a low learning curve, and preventing user error as much as possible. Indeed, in order to ensure an intuitive system that adheres to professional standards, applications utilized in the healthcare field must take into consideration the diverse backgrounds and tasks of clinicians while employing usability standards. [4] In addition to providing the required functionality, design guidelines should be adhered to in order to increase the overall usability of the interface. Additionally, heuristic evaluation is often used in order to determine potential issues with the user interface design. While a plethora of usability heuristics exist, the majority focus on similar issues such as consistency, task match, appropriate visual presentation, user control, memoryload reduction, error handling, and guidance and support. [5] 1. OpenVista Design Principles Violations: While the current structure and function of OpenVista may improve overall performance, in terms of cost reduction, clinical performance, and patient care outcomes, there is still much room for improvement in the OpenVista user interface. Indeed, the overall structure of the GUI violates many of the aforementioned design principles. There have been numerous papers discussing usability violations of OpenVista and comparing OpenVista to other ERs. owever, there is a distinct dearth of literature focused on the improvement of OpenVista. Accordingly, rather than providing solutions to the problems of OpenVista, individuals are simply told that alternative ERs are either better or worse for clinicians. [6-7] Due to the aforementioned, the focus of this analysis will be on OpenVista s Patient Summary Screen (Figure 1). The patient summary screen contains a plethora of information that a clinician can browse through when treating a patient. Unfortunately, however, the layout and design of this screen leaves much to be desired. Furthermore, improved design would not only improve the aesthetic and appearance, but would also improve the efficiency and usability of the ER. 2. ETODS 2.1 euristic Evaluation and Severity Ratings: euristic Evaluation (E) allows an evaluator to examine the interface and make note of usability issues. In order to determine potential user problems, a set of heuristics is used to judge the usability of the interface itself. One E method utilized by many to determine the usability of a GUI is

2 Figure 1: Sample OpenVista Patient Summary Screen Nielsen s ten usability heuristics. Developed by Jakob Nielsen and Rolf olich, Neilsen s usability heuristics (Table 1) are widely used in order to pinpoint issues with GUIs. [8] In addition to determining design flaws, the evaluator may be asked to rate the severity of each issue. Nielsen defined the severity of a usability issue as a combination of the following factors: frequency, impact, and persistence. [9] If the frequency, impact, and persistence of a usability problem is high, the severity of the usability issue will also be quite high. In order to weigh the severity of usability issues against one another, Nielsen devised a severity rating scale. The severity of a usability issue can be rated on a 0-4 scale (Table 2), with 0 indicating no usability problem and 4 indicating a usability catastrophe. [10] Estimating the severity of usability issues offers a number of benefits. Primarily, severity ratings can aid the system producer in the allocation of resources. For example, if the severity ratings indicate major usability issues, it would be necessary that these problems be fixed prior to release. Conversely, if there are only a limited number of minor and/or aesthetic 1 Visibility of system status 2 atch between system and the real world User control and freedom 4 Consistency and standards 5 Error prevention 6 Recognition rather than recall 7 Flexibility and efficiency of use 8 Aesthetic and minimalist design 9 elp users recognize, diagnose, and recover from errors 10 elp and documentation usability issues, the producer might decide to release the system without further changes. While conducting a E and assigning severity ratings does not, in itself, provide interface solutions, it does allow for specific usability problems to be pinpointed. Once specified, individuals can propose solutions that solve the usability issues and reduce the severity rating to a sufficient state. In short, E allows producers to identify potential usability flaws. 2.2 Prototyping: A prototype is a draft version of a concept, application, or idea that can range from a set of drawings to a fully functional model. [11] Prototyping is not only cost-effective, but can provide useful insight regarding application design. Furthermore, prototyping allows the design team to quickly make changes and alter the design prior to the release of the product. Accordingly, prototyping has been known to improve the productivity of a department s applications development effort. [12] 0 Not a usability problem at all Cosmetic problem only: Need not be fixed unless extra time is available on project inor usability problem: Fixing this should be given low priority ajor usability problem: Important to fix, so should be given high priority Usability catastrophe: Imperative to fix this before product can be released Table 1: Nielsen s 10 Usability euristics [8] Table 2: Severity Rating Scale [10]

3 Usability Problem Description Layout and position of the patient summary boxes is inconsistent and this leads to poor visibility of problems, vitals, and admissions information, which, in turn leads to inefficiency. There is an inconsistency between metric and U.S. measurements for weight, height, and temperature. The user should be able to quickly/efficiently switch between these systems depending on preference. The fonts and capitalization are all the same which leads to no clear level of hierarchy. Differentiating the fonts could improve the clarity of the patient summary screen. The Vitals section on the patient screen could be improved by creating visual models to represent whether the vitals are in normal or abnormal ranges. The patient summary screen violates the less is more principle. inimize the information and limit what the user sees to the essential aspects. Buttons at the top of the patient summary screen do not appear as buttons upon first glance. euristics Violated Consistency Visibility Consistency Flexibility ean Severity Rating.25 Consistency atch Efficiency Aesthetic & inimalist Design atch 2.5 It is unclear that the titles of each sub-box can be ordered when selected. atch 2.5 Table : Usability Problems, euristics Violated, and the Assigned Severity Rating 2 There are a variety of different prototyping techniques that can be utilized. Generally speaking, there are three main categories of prototypes: high-fidelity, medium-fidelity, and low-fidelity. [1] A high-fidelity prototype could be a fully functioning model of the product, a medium-fidelity prototype could be a limited functionality, mockup of the model, and a low-fidelity prototype could be a non functional mockup or storyboard representation. While low-fidelity prototypes have limitations in terms of usability assessments, they can be helpful in visualizing alternative design solutions, which, in turn, can provoke innovation and improvement. [14] There has been much debate as to whether using high-fidelity prototypes ultimately results in a final product with a higher level of quality than utilizing low-fidelity prototypes. One study attempted to discover if using a high-fidelity, computer based prototype was more effective than using a low-fidelity, paper prototype. The results showed that there was no reliable difference in the number or type of usability issues discovered using the low- and high-fidelity prototypes. [15] After the results of the aforementioned study were released, a number of follow up studies were conducted in order to determine if the results were limited to specific domains. Primarily, Catani and Biers used low- and high-fidelity prototypes to perform library search tasks and found that there was no reliable difference in the usability issues discovered. In fact, there was a high commonality in the usability issues uncovered using the different prototypes. [16] In 2002, the idea was put to the test again when Walker et. al. had participants carry out banking tasks using low- and high-fidelity prototypes. Like the prior studies, it was found that there was no reliable difference in the number and type of usability issues uncovered. [17] Clearly, there are many benefits to low-fidelity prototyping. Due to the fact that low-fidelity prototyping is quite costeffective, many design teams utilize this type of prototyping early in the design stages. Furthermore, it is often unnecessary to use a fully functional prototype when designing an application. Perhaps most importantly, as previously noted, there are no significant differences in determining usability issues with low- and high-fidelity prototypes. 2. GOS/L Analysis: When considering usability, it is imperative that one takes into account both the physical and mental actions undertaken by the user. GOS analysis (Goals, Operations, ethods, and Selection rules) is one of the best-known models of cognitive task analysis. [18] The GOS analysis aims to speak to the coverage of the functionality of a system and provide estimates of task performance time. [19] The eystroke-level odel (L), proposed by Card and oran, is a simplified version of GOS wherein the execution time of a task is estimated by listing sequence operators and assigning a time to each operator. The original L had six classes of operators: for pressing a key, P for pointing to a location on screen with the mouse, for moving hands to home position on the keyboard or to the mouse, for mentally preparing to perform an action, and R for system response where the user waits for the system. [20] For the purposes of this project, the L analysis was used to determine both physical and mental actions undertaken by a user who wishes to garner information from the patient summary screen. The actions undertaken by the user were chronicled step by step and the time associated with each action was recorded.. RESULTS.1 E and Severity Rating Results: Based upon Nielsen s usability guidelines, a E of the OpenVista patient summary screen was conducted and severity ratings were assigned to each issue (Table ). Four individuals, having backgrounds in both informatics and medicine, conducted the E. While none of the usability problems were catastrophic in nature, rectifying the problems would enhance the user experience and the overall efficiency of the system.

4 Figure 2: Prototype of Improved Patient Summary Screen any of the usability issues centered on consistency/standards and visibility; i.e., in the case of the patient summary screen, there was little adherence to platform conventions. For example, there appeared to be a lack of hierarchy amongst the different areas of the patient summary screen. By clearly delineating the difference between headings and input, clinicians would struggle less to acquire necessary information. The visibility of the patient summary screen was also limited. Primarily, the layout and position of the information boxes is inconsistent and leads to poor visibility of problems, vitals, and admissions information. The lack of consistency and visibility can lead to both inefficiency and medical errors. Furthermore, it was also found that there were issues regarding concept matching between the system and the real world. A representation is a mapping between a represented world (that which is to be represented) and a representing world (that which does the representing). [21] As far as the patient summary screen is concerned, it is imperative that functions of the GUI match the internal representation of the user; i.e., buttons should appear to be buttons and there should be no question regarding their function..2. Prototyping Results: As previously discussed, many studies demonstrated that there was no reliable difference in the usability issues discovered when using lowand high-fidelity prototypes. Accordingly, rather than create a high-fidelity prototype of the OpenVista patient summary screen, I chose to create a low-fidelity, paper mock-up (Figure 2). In order to create a viable prototype, it was essential to understand the outcome of the E and associated severity ratings (Table ). Via the utilization of the E a prototype was created that fixed the usability issues found in the initial OpenVista patient summary screen. In order to test the usability of the new prototype in comparison to the typical OpenVista patient summary screen, a L analysis for both screens was undertaken.. L Analysis: For this L analysis, industry standard estimated times for each operation are used. [22] As seen in Tables 4 and 5, the L analysis was used determine the internal and external actions required by the user for a number of common actions. In each scenario, the low-fidelity prototype of the patient summary screen required fewer actions of the user than the current OpenVista patient summary screen. Specifically, on average, the prototype decreased the time a user spends on a single task by 1% (Figure ). Such a huge decrease in time spent would not only lead to more timely care, but would also be more efficient and cost-effective. Furthermore, when the mental operations were compared, it was found that OpenVista required 60% more internal, cognitive operations from the user than the prototype (Figure 4). Due to the fact that the mental load of the user was decreased, the usability of the prototype patient summary screen far exceeded the usability of the OpenVista patient summary screen.

5 4. CONCLUSION In this study the current OpenVista patient summary screen was evaluated via Nielsen s usability heuristics. After determining the various usability issues, the issues were given a severity score based on Nielsen s severity rating scale. Based upon the outcomes of both the E and severity rankings, a low-fidelity prototype was developed. The aim of the prototype was to improve upon the current OpenVista patient summary screen by addressing the discovered usability problems. Both patient summary screens were then evaluated via a L analysis. ere, different user actions were tested in order to analyze the amount of time an expert user would spend on each task. Based on the results of the L analysis, it was ethod for determining most recent appointment. Description Operation Internal or External Time (Sec) Locate Appointment Information Point to Appointment P heading [field] External 1.10 Click on Appointment heading Recognize that appointments can be [recognize] ordered Click on Appointment heading to order options Locate most recent appointment 5.50 ethod for determining patient temperature. Locate Vitals Information Locate Patient Temperature Determine if temperature is in English/etric [determine] measurement Determine if temperature is within the normal or abnormal range [recognize] 5.20 ethod for determining active problems. Locate Active Problems ove pointer to scroll P bar [scroll bar] External 1.10 Scroll down to view all active problems 2.90 Table 4: L analysis of the current OpenVista patient summary screen (Figure 1). found that expert users must invest more time and perform more mental operations to garner information from the current OpenVista patient summary screen. Further directions for this project include creating a functional prototype and comparing the functional usability of OpenVista and the prototype. 5. CONCLUSION This study demonstrates that while ERs are quite valuable, poor design can lead to extensive usability issues amongst both novice and expert users. While OpenVista is quite prevalent amongst many hospitals utilizing ERs, there is still much room for improvement. By upgrading the user interface, the OpenVista ER could improve usability leading to an overall improvement in user efficiency. 6. REFERENCES [1] Poon EG, Wright A, Simon SR, Jenter CA, aushal R, Volk LA, et al. Relationship Between Use of Electronic health Record Features and ealth Care Quality: Results of a Statewide Survey. ed Care. 2010;48(): ethod for determining most recent appointment. Description Operation Internal or External Time (Sec) Locate Appointment Information Point to order appointments Click arrow to order appointments Locate most recent appointment P [field] External ethod for determining patient temperature. Locate Vitals Information ove pointer to P English or etric [button] Button External 1.10 Click on English or etric Button Locate Temperature 4.10 ethod for determining active problems Locate active problems 1.60 Table 5: L analysis of the prototype for an updated patient summary screen (Figure 2).

6 Time (Seconds) Time Spent Performing Tasks Operation 1 Operation 2 Operation Average Tasks OpenVista Prototype Figure. Time required by the user when undertaking operations in the OpenVista and Prototype screens. ental Operations Required Amount Operation 1 Operation 2 Operation Average Tasks OpenVista Prototype Figure 4: ental operations required by the user when undertaking operations in the OpenVista and Prototype screens. [2] Brown S, Lincoln J, Groen PJ, olodner R. VistA U.S. Department of Veterans Affairs national-scale IS. Int J ed Inf. 200;69: [] System Overview. Open Source ER eaningful Use Leader - edsphere Systems OpenVista Available at: January 21, [4] Johnson C, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. 2005;8(1): [5] vannberg ET, Law EL, Larusdottir. euristic evaluation: comparing ways of finding and reporting usability problems. Interacting with Computers. 2007;19(2): [6] Saitwal, Feng X, Walji, Patel V, Zhang J. Assessing performance of an electronic health record (ER) using cognitive task analysis. Int J ed Inform ay 6 [epub ahead of print]. [7] Zhang Z, Walji F, Patel VL, Gimbel RW, Zhang J. Functional analysis of interfaces in U.S. military electronic health record system using UFuRT framework. AIA Annu Symp Proc Nov 14; 2009:70-4. [8] Neilsen J, olich R. euristic evaluation of user interfaces. Proceedings of CI 90; 1990 April 1-5; Seattle, WA , [9] Nielsen J. Usability inspection methods. New York: John Wiley & Sons; [10]Nielsen J. Severity ratings for usability problems.use-it.com Available at: arch 1, [11]U.S. Department of ealth & uman Services. Develop a Prototype - Design. Usability.gov Available at: arch 18, [12]raushaar J, Shirland LE. A prototyping method for applications development by end users and information systems specialists. IS Quarterly. 1985; 9(): [1] Sauer J, Selbel, Ruttinger B. The influence of user expertise and prototype fidelity in usability tests. Applied Ergonomics. 2010;41(1): [14] Bailey B. Paper prototypes work as well as software prototypes. Usability.gov Available at: arch 18, [15] Virzi RA, aris D, Sokolov JL. Usability problem identification using both low and high-fidelity prototypes. Proceedings of CI 96; 1996; Vancouver, BC, Can [16] Catani B, Biers DW. Usability evaluation and prototype fidelity: users and usability professionals. Proceedings of the uman Factors and Ergonomics Society 42 nd Annual eeting; 1998 Oct 5-9; Chicago, IL [17]Walker, Takayama L, Landay JA. igh-fidelity or low-fidelity, paper or computer? Choosing attributes when testing web prototypes. Proceedings of the uman Factors and Ergonomics Society 46 th Annual eeting; , [18]Card S, oran TP, Nowell A. The psychology of humancomputer interaction. Lawrence Erbaum, 198. [19]John BE, ieras DE. Using GOS for user interface design and evaluation: which technique? AC Transactions of Computer-uman Interaction (TOCI). 1996;(4): [20]Card, S, oran TP, Newell A. The keystroke-level model for user performance time with interactive systems. Communications of the AC. 1980;2(7): [21]Zhang J. Representations of health concepts: a cognitive perspective. J Biomed Inform. 2002; 5(1): [22]ieras DE. Using the keystroke-level model to estimate execution times. (Unpublished report, 199). Available at: Accessed April 4, 2010.