Usability Engineering 1 Running head: USABILITY ENGINEERING, COGNITIVE SCIENCE, AND HEALTHCARE INFORMATION SYSTEMS Usability Engineering, Cognitive Science and Healthcare Information Systems Yong IL Choi Duke University School of Nursing Nursing 409 Dr. Constance Johnson February 26 2008
Usability Engineering 2 Introduction Usability engineering is a methodology that applies cognitive science theory to software design and development to achieve a user-centric interface. The goal of usability engineering is to develop a system that helps the end user perform tasks accurately, quickly and with a minimum amount of effort. In addition there are other groups, the purchasers of the systems and IT staffs responsible for installing, distributing and maintaining the systems, which have different usability concerns than the end-users. With the NHII deadline for electronic health record (EHR) implementation coming up in 2014, health care institutions face significant challenges in meeting the deadline and the usability requirements of their stakeholders. Usability Engineering Usability engineering is the application of engineering and cognitive science theory to building usability into an information system. The goals of usability engineering are to reduce errors, increase user efficiency, increase user satisfaction, decrease training time, decrease maintenance time, and save money (Johnson, Johnson et al., 2005). Kushniruk and Patel (2004) describe usability engineering as an application for designing software with an emphasis on continual input from the end users throughout the development phase of the information system. Using an iterative process to evaluate a health care system during its development ensures that end user considerations are dynamically considered throughout design and development rather than an IT designer s static prior notion of the user requirements (Kushniruk and Patel, 2004). As in all engineering disciplines, usability engineering incorporates analysis, evaluation and review/testing in all phases of the project. During the design phase, analysis and evaluation are the primary activities, followed by evaluation and testing during the development phase. Developers and end-users do the evaluation and testing of the systems usability.
Usability Engineering 3 The goal of evaluation and testing is to identify problems and then implement solutions. Important goals of usability include the following (Nielsen and Mack, 1994): 1. Learnability: the system is easy to learn how to use, thus reducing the training time. 2. Efficiency: the system helps users achieve a high level of productivity. 3. Memorability: the system implements features in a way that is easy to remember. 4. Errors: the system design helps prevent, detect, and recover from errors easily. 5. Satisfaction: users find the system pleasant to work with. Cognitive science Cognitive science combines research from psychology, computer science, cognitive anthropology, philosophy and linguistics to explain how humans and machines interact and evolve (Horsky, Kaufman et al., 2003). Applying cognitive science to usability engineering facilitates assessments of the healthcare information systems for its learnability, memorability, efficiency, and human-computer interaction problems (Kushniruk and Patel, 2004). To understand how cognitive science affects usability engineering, an illustration of modified cognitive architecture ACT-R/PM by Byrne (Shortliffe 2006, p. 142) is shown below. The cognitive architecture attempts to explain how the human brain organizes information and knowledge to produce intelligent behavior. Vision module Procedural memory Declarative memory Cognitive layer Motor module Speech module Auditory module Perceptual-motor layer Environment Figure 1. Illustration of a cognitive architecture ACT-R/PM. (Source: Diagram modified from Byrne [2003] (Shortliffe 2006, p. 142))
Usability Engineering 4 The architecture consists of two layers, a cognitive layer and a perceptual-motor layer. The cognitive layer contains two long-term memory (LTM) modules; a procedural memory to dictate what and how to execute and a declarative memory that retains concepts. The perceptualmotor layer interacts with the environment and stores information in the declarative memory via our sight and sound senses. Procedural memory executes functions via the perceptual-motor layer using vision, motor, speech, and hearing. This perceptual-motor layer is what is also known as working memory (WM). Long-term memory stores information and knowledge for the long term and working memory stores information that is required for immediate cognitive activities, such as reading and understanding documentation (Shortliffe and Cimino, 2006). Cognitive science theory explains to designers how a user s working memory is affected by interface design. Understanding the strengths and limitations of human cognition allows designers to create interfaces that enhance people s skills rather than stressing them. Usability evaluation methods based on cognitive science can assess a range of performance measures such as error rates, execution times, and learning curves (Shortliffe and Cimino, 2006, p. 143). User cognitive overload can be detected by interview, surveys/questionnaires, and selfreporting by the users or testers. Then, the problems can be evaluated and fixed during development of the information system. A well-designed information system with an intuitive interface can reduce the demands on a user s working memory. A busy interface with difficult to find objects or no contextual help can create cognitive overload. Understanding the properties of human cognition can help the development team to build user-friendly information systems. Usability Evaluation and Testing Usability testing and evaluation are activities performed by actual users or representative users to identify problems in the system. Procedures for testing and evaluation can vary
Usability Engineering 5 depending on the complexity of the system. According to Charlton (2002), there are 4 components to the testing of human factors: a situation with the environment where the users or the system is placed; individuals who will be using the system and the level of their experience, skills, and cognitive states; tasks to achieve using the system; and the outcome of the test to determine its effects on the users, such as satisfaction or disappointment. A few examples of usability evaluation methods are: a cognitive walkthrough that helps to identify user learning problems, a GOMS or keystroke level model that can help predict user performance, a heuristic evaluation, or a think aloud protocol. One method for evaluating system usability, heuristic evaluation is known to be an economical approach. Heuristic evaluation to identify potential user problems in an interface usually employs 3-5 experts in a laboratory setting for a short amount of time. (Nielsen and Mack, 1994). Initially advocated by Nielsen in the 1990s as discount usability engineering, heuristic evaluation can reveal 80-90% of usability problems (Nielsen and Mack, 1994). The heuristic evaluation method is a good method for a development team to use before actual endusers evaluate the system. Another evaluation method, the think aloud protocol takes its cues from cognitive psychology and puts an emphasis on understanding the cognitive processes of computer users as they interact with user interfaces (Patel, 1998). With think aloud, users are asked to say out loud what they are saying or thinking to themselves while they work with the system. The evaluators do not aid the users in their tasks but observe and make notes of the issues that users encounter. The gathered information is evaluated and the findings are applied to the refinement of the interface (Patel, 1998). This method of testing can detect 80% of usability problems with
Usability Engineering 6 only 4-5 testers and is considered a cost effective option (Patel, 1998). Think aloud is a good method to use with actual end-user participants when organizations evaluate and test a system. The iterative approach to software and system development has a lot of support in the usability engineering field (Nielsen and Mack, 1994; Kushniruk et al., 2004). Applied to an entire development cycle or to the evaluation and testing of a vendor s product in a larger system, the iterative approach helps manage the scale of a large project and can help ensure that problems get identified before they become too large to fix. Cost-benefit of Usability Engineering Applying usability engineering during the design and development phase can help reduce the life cycle costs of a system. It is easy to think that an iterative process of design, development, evaluation and test is time consuming and overly expensive, but the later in development a problem is found the more it costs to fix. It costs 10 times more to fix a problem during development than during the design stage and 100 times more after the development stage (Nielsen and Mack 1994). Furthermore, the lifetime costs associated with maintenance and updates of the system can be 2-5 times more than the original development costs (Shortliffe and Cimino, 2006, p.242). Usability engineering is a systematic approach to making software easier to use for the end users, but it can also affect other stakeholders of the healthcare information system. For instance, if a system is hard to learn how to use or is easy to make a mistake with, then the enduser s organization s IT staff will need to allocate more time for responding to end-user questions and errors. A large number of change requests to the vendor can lead to additional updates to the system. These updates will require the organization s IT staff to spend more
Usability Engineering 7 resources installing and testing the system. If the usability issues are serious enough the organization will have to repeat the system acquisition process, which can be quite expensive. Conclusion The usability of a system is not limited in its effects to the end-users of the system. The organizations business team and IT staff have their own usability concerns and costs. Treating usability as a major function that must be incorporated during the earliest stages of design and development of an information system will go a long way to meeting the needs of the different groups.
Usability Engineering 8 References Charlton, S.G., O Brien, T. G., eds. Handbook of human factors testing and evaluation. 2 nd Edition. Lawrence Erlbaum Associates, Inc; 2002. Horsky, J., D. R. Kaufman, et al. (2003). "A framework for analyzing the cognitive complexity of computer-assisted clinical ordering." Journal of Biomedical Informatics 36(1-2): 4-22. Johnson, C. M., T. R. Johnson, et al. (2005). "A user-centered framework for redesigning health care interfaces." Journal of Biomedical Informatics 38(1): 75-87. Nielsen, J., Mack, R.L., eds. Usability inspection methods. Wiley; 1994. Kushniruk, A. W. and V. L. Patel (2004). "Cognitive and usability engineering methods for the evaluation of clinical information systems." Journal of Biomedical Informatics 37(1): 56-76. Patel, V., Kushniruk, AW (1998). Interface design for health care environments: the role of cognitive science. AMIA Symp: 29-37. Shortliffe, E.H. and J.J. Cimino, eds. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. Third Edition. Springer; 2006.