An Iterative Usability Evaluation Procedure for Interactive Online Courses by Laurie P. Dringus ABSTRACT The Internet and World Wide Web (W3) have afforded distance learners simple links to access information. Faculty can easily bring their courses online to students by building menu systems using hypertext markup language (HTML). Since online course design must include attention to effective user interface design, usability evaluation becomes a major component of the design process. This article focuses on the importance of providing iterative usability evaluation of online courses and strategies to ensure effective online course interface presentation. Some of the goals for evaluating effective online course presentation are: ease of use, aesthetic appeal, clear feedback, error handling, controls for parallel and serial group communication, and transparency of the user interface. In addition, a procedure is described for ensuring iterative usability evaluation throughout the online course lifecycle that will help instructors routinely inspect and improve the interfaces of their online courses. Introduction The interface or the presentation structure of an online course should provide clear organization of course information. For example, a menu structure could centralize course syllabi, lecture slides, and class announcements. The interface should present a structure for presenting student activities that require communication exchange and interactivity among students. The interface should enable efficient navigation for students to extract, manipulate, and share new information, and it should enable students to organize and manage their course activities. The design of an online course interface should be simple and transparent. Dialogue design should support basic usability heuristics such as ease of use, ease of learning the system, aesthetic appeal, clear system feedback and error messages to allow quick recovery from system or user input errors (Shneiderman, 1992, Nielsen, 1993). Essentially, the interface, if designed effectively, should enable students to work independently, increase their competence of being in the online environment, and give students control over their own learning experience. The opportunity to easily design graphical interfaces using hypertext and multimedia has resulted in an explosion of innovative applications surfacing on the Internet (Hughes, 1994). What seems to occur as educators enthusiastically bring online courses to the World Wide Web is that emphasis is placed on the aesthetic appeal of the course home page, its submenus and links, as well as the infusion of a variety of media such as hypertext and/or hypermedia, without a systematic regard to evaluating the usability of the interface from the students perceptions. How do students (i.e., navigators of the interface) perceive the online course interface as an extensible learning environment? Assuming that improvements and changes to the course interface presentation will evolve over time, it is critical that usability evaluation becomes an iterative part of the online course design process. This article discusses usability 10 SPRING 95
evaluation as an essential component for evaluating distance education environments. A procedure is described for ensuring iterative usability evaluation throughout the online course lifecycle that will help instructors routinely inspect and improve their online course interfaces. Usability Evaluation of Online Courses Educators need to continually assess their assumptions about distance education, namely that distance education provides convenience, more learner control, and better quality than conventional education (Verduin and Clark, 1991). Routine summative or formative evaluations on distance education environments will further validate and perhaps increase an understanding of these assumptions about distance education. For example, routine student evaluations enable students to provide feedback in regard to their perceptions about course content, the degree to which course objectives are met, and instructor performance. Student perceptions about the online course presentation or interface also should be included in online course evaluation. Would distance learners perceive content of online courses differently based on the visual presentation of the course? For example, if students rated overall course content favorably, but rated course structure as below average, will the instructor attribute the low course structure rating to poor planning or will the instructor attribute the low rating to a navigation problem in the online course interface? These are subtle and important issues that, unfortunately, are not typically evaluated by students or faculty. The dynamics of visual presentation and physical environment in an online course environment are dramatically different from the dynamics that traditionally occur in a face-to-face learning environment. How does an instructor know that the course interface provides full potential for students to access and utilize the information that is being presented? What process does the instructor need to follow to assess the course interface in terms of the interface providing students with a rich and effective learning environment? One method used to assess these issues is performing a usability evaluation of the interface with students. Usability evaluation can be performed formatively or summatively with simple or complex methods and is described in detail by Nielsen (1993). User evaluation studies are becoming more prevalent in the software industry. One method, heuristic evaluation, is a process of looking for what is good or bad about an interface (Nielsen, 1993). Nielsen proposed that the goal of heuristic evaluation is to find the usability problems in a user interface design and treat the evaluation process as a normal part of the iterative design process. Generally, heuristic evaluation is performed by having each individual inspect the interface alone under controlled conditions. It is common practice in usability evaluation to combine inspection methods with subjective evaluation procedures such as the use of standardized questionnaires to gauge users subjective satisfaction of the interface. When a composite number of subjects have completed their inspections and evaluations, the data is then compiled into a formal record for future reference (Nielsen, 1993). Design Heuristics Applied to Online Course Presentation The emphasis on usability evaluation is related to the work of Shneiderman (1992) and Nielsen (1993). Principles for good design are being used extensively by user interface designers as a basis for systematic inspection of user interfaces. Usability heuristics summarized by Shneiderman (1992, pp. 72-73) and Nielsen (1993, p. 20) can be applied to evaluating online course interfaces. They include: Strive for consistency. Terminology, commands, and consistent sequences of actions should be required in similar situations. Online course menus should maintain a consistent flow for navigation. Command actions for nav- SPRING 95 11
igating back and forth from submenus should be similar and recognizable from screen to screen. Minimize user memory load. Simple displays, consolidated multiple screens, and reliance on recognition memory can reduce short-term memory load. Online course hyperlinks to local and global locations should be easily located and easy to navigate. For menus, use meaningful letters or words for menu items that they represent. Provide roadmaps for simple and complex information links. Provide informative feedback. Modest to substantial system feedback is necessary for a range of actions. If online course hyperlinks are currently not operational, the system should appropriately inform users that the link is not currently available and they can exit or return to a previous link. Provide clearly marked exits. A user should be able to easily exit from a current system state either by choice or via necessity. Provide consistent display of appropriate exit commands on online course screens. Provide shortcuts. Reduce the number of interactions and increase the pace of interaction as the user desires. Organize course links and hyperlinks into logically similar groups. Provide an index of links that could reduce the need for navigation of multiple menu layers where links are originally positioned. Prevent errors. Try to prevent serious system errors from occurring. Carefully design and test the online course interface for navigation efficiency and the degree to which escape and undo functions are recognizable. Be sure that error messages are clearly written, displayed, and suggest a solution. Provide help and documentation. Short and concise information that leads the user to complete the task may be necessary via screen display or hardcopy. If users cannot intuitively follow the course interface or portions of it, then provide them with online help screens and hardcopy documentation to effectively learn and use the system. Do not make assumptions about students navigating your course interface based on what you know about their computer experience level. Other important considerations that can be applied to online course presentation include: Effect transparency of the interface. Online course interfaces should promote visibility of information accesses and course activities without interfering with the student s desire to get the job done. Provide controls for parallel and serial group communication. Online course interfaces should provide instructors and students with links to tools that allow them to communicate under the following interaction scenarios: student-to-teacher, teacher-to-student, and student-to-student. The interface should assist the instructor in promoting a balance between individual and group experiences by providing online mechanisms that are needed to support individual needs (e.g., structured assignment activities) and group needs (e.g. parallel communication, idea generation and sharing, and project management). Provide ease of use and ease of learning the system. Avoid an overemphasis on complex hyperlinks and multiple menu pages. Provide simple menus that are organized by breadth instead of depth. Breadth is the number of menu items, depth is the number of menu layers. Provide online and hardcopy tutorials for first time use. Achieve aesthetic appeal of the interface. Follow usability guidelines by user interface experts (Shneiderman, 1992, Nielsen, 1993) for designing text or graphical screens. Do not crowd online course interfaces with overuse of fancy fonts and multimedia links. Do not mistake increased functionality for aesthetic appeal. 12 SPRING 95
These usability heuristics and considerations and their direct application to online course interface presentation provide instructors with some sample guidelines for designing course interfaces. These guidelines can be converted into measurable goals that can be evaluated when following a procedure for iterative usability evaluation of online courses. Procedure For Iterative Usability Evaluation of Online Courses The following suggested procedure presents ideas for conducting formative usability evaluations. However, portions of the procedure can be extracted for summative usability evaluations. Instructors will need to determine the point in the online course design and delivery when assessment is appropriate and the type of assessment (formative or summative) to incorporate. The suggested procedure is purposely brief; the extent of a formal procedure can be complex. This section contains a simple snapshot of some ideas of how usability evaluation can be integrated into the online course lifecycle. The procedure for assessing usability is meant to be iterative throughout the stages of development, evaluation, and testing. Development 1. Develop the course interface with an understanding of the diversity of students and user experience levels. Shneiderman(1992) emphasized the importance of assessing user profiles in terms of age, gender, physical abilities, education, training, motivation, goals, and personality. Also, conduct an assessment of computer experience. How long has the student used computers? What types of software applications is the student familiar with? How long has the student been exposed to your institution s distance education environment? 2. Perform a task analysis (Nielsen, 1993) of the actions and tasks students will need to do and want to accomplish with the online interface. Seek out the goals of the system and the steps necessary to perform them. Analyze and prioritize these tasks into a list of functions that the interface should include. 3. Design the course interface by effecting the principles and heuristics of good design as outlined by Shneiderman (1992) and Nielsen(1993). Evaluation 4. Recruit five to ten students to evaluate your course interface based on the usability heuristics presented in the section of this paper Design Heuristics Applied to Online Course Presentation. Provide students with a survey that asks them to rate on a 1 to 10 scale the extent to which these principles were met. In addition, provide survey items or interview questions that ask students for their overall perceptions of the course in terms of ease of use, ease of learning, aesthetic appeal, error handling, etc. Here are some sample questions to prompt students reactions about the interface: What errors typically occurred as you worked through the course interface? How did you discover and correct errors? Describe notable successes and failures. What kinds of problems did you have with this interface? What are your ideas for improvement? What things would you like to see most changed or least changed to the interface? 5. Integrate questions relating to the organization of information, navigation of course menus, and perceptions of communicating with the instructor and other students into student course surveys. Share the data with your administrators and faculty peers. This procedure is especially good for summative course evaluations. Testing 6. For formative or summative evaluations: Conduct a usability test of your course interface. Establish user test goals and a set of tasks that five to ten students can complete at a designated time and location. Provide a general description of each task to be performed, the steps in each task, and, perhaps, distinguish inter- SPRING 95 13
action with other tasks. A simple task list could be presented as follows: Locate the course syllabus submenu Read today s course announcements Click on the Conferences hyperlink Select and review at least two course lecture files Review all submenus from the home page Send your instructor an e-mail message from the Activities menu There are different strategies for structuring the test for your students. All students can either work together at the same time, a pair of students can be tested at one time, or one student can be tested at one time. During testing, if possible, observe each student completing the usability test and write down their notable actions and reactions to the course interface as they complete the tasks. Look for trouble areas in navigation, error recovery, exiting submenus, selecting hyperlinks, etc. Some common measures for usability testing of interfaces include: subjective comments, task completion time, speed of navigation through the system, problems with titles and screen formats, the number of errors the user made, and the number or extent of requests for help either from online or documentation sources. 7. Use the data collected from students to make direct changes to your course interface. A few iterations of making changes to the course interface, as well as user evaluation and testing, may occur during the course lifecycle. The process may also continue for subsequent course terms until the instructor is confident that the online course interface is fully functional from a usability standpoint. Summary This paper has addressed the importance of providing iterative usability evaluation of online courses, heuristics to ensure effective online course interface presentation, and a procedure for implementing iterative usability evaluation throughout the online course lifecycle. As more innovative applications surface on the Internet, educators need to systematically evaluate the usability of their online course interfaces from their students perspectives. By following a routine process for iterative usability evaluation of online course interfaces, instructors will improve their course interfaces based on meaningful data and, in turn, strengthen the distance learner s opportunity for having a rich and an effective learning environment. References Hughes, K. (1994). Entering the World- Wide Web: A guide to cyberspace. (Version 6.1) Palo Alto, CA: Enterprise Integration Technologies. Nielsen, J. (1993). Usability Engineering. San Diego, CA: Academic Press. Shneiderman, B. (1992). Designing the User Interface: Strategies for Effective Human-Computer Interaction. Second Edition. Reading, MA: Addison-Wesley. Verduin, J.R. and Clark, T. (1991). Distance Education: The Foundations of Effective Practice. San Francisco, CA: Jossey-Bass. About the Author Laurie P. Dringus, Ph.D. is an Assistant Professor in the School of Computer and Information Sciences at Nova Southeastern University, Ft. Lauderdale, Florida. She earned her Ph.D. in Information Systems from Nova Southeastern University in 1991. She teaches graduate courses in human-computer interaction and decision support systems. Several of her courses are accessible via the Internet. Her research interests focus on design and evaluation of human-computer interfaces, usability engineering and evaluation, computer-mediated communications technology and user performance, group support systems, and collaborative learning in distance education environments. E-mail Address: laurie@alpha.acast.nova.edu. 14 SPRING 95