Student Perceptions on the Importance of Distance Learning Module Design Dimensions

Student Perceptions on the Importance of Distance Learning Module Design Dimensions Cristina Pomales-Garcia, Yili Liu and Ángel D. Lopez cristina.pomales@upr.edu,yililiu@umich.edu, angel.lopez8@upr.edu Abstract - A controlled experimental study was conducted to better understand how guidelines in screen design could be applied to the design of Web-Based Distance Learning (WBDL) environments, where multimedia such as audio, video, figures and text are displayed simultaneously on the screen. Six design dimensions were studied including aesthetics, clarity, excitement, organization, simplicity and structure. Using content analysis methodology, a total of 215 descriptions of the design dimensions were evaluated and 37 design attributes were identified from the descriptions. The results of the study provide a comprehensive list of the attributes associated with each design dimension and the relative level of importance between the design dimensions. The results showed that Clarity is the most important dimension while excitement is apparently the least important of all. Further, the characteristics of text and how it is used in a Web module appear to be a design attribute that influences all the design dimensions under study. The results suggest that even though attractiveness might not be considered one of the important dimensions based on the rankings, as modules are made Attractive following the results of design attributes in this study, then desirable qualities of the other design dimensions are achieved. Index Terms Distance Learning, Design Dimensions, Engineering Education, Web Modules. INTRODUCTION Distance learning courses are being designed by instructors or instructional designers using one or more of these digital media and technological advances to meet the needs of a diverse and demanding student population. Many designers of distance learning environments are often using common sense, intuitive judgments and educated guesses instead of empirically based guidelines [1]-[3]. Guidelines and models exist for screen design [1] [4]- [7] multimedia environments [8] and Web-based distance learning (WBDL) instruction [9]-[12]. In the area of screen design, aesthetics guidelines are used to help designers create attractive screens [13] although the design theory is often rudimentary and oversimplified [14]. [15] identified content elements that must be accounted for in the overall design of Web-based courses, but do not address how the information is presented, excluding aesthetic factors as important variables in the design process. According to [16] successful educational environments should incorporate learner elements along with Website elements to support information presented. The research literature suggests that several design dimensions or aspects of design should be taken into consideration when designing Web screens. These dimensions generally include simplicity, aesthetics or visual attractiveness, organization, clarity, excitement and structure. These dimensions mentioned above are very broad and include a variety of attributes that can have multiple interpretations in design. OBJECTIVES This study proposes to evaluate the relative importance of six design dimensions and understand how these dimensions are described by two group of stakeholders students versus researchers, and achieve further understanding of how guidelines in screen design and multimedia could be applied to the design of WBDL environments where multimedia such as audio, video, figures and text are displayed simultaneously on the screen. To our knowledge, no article in the literature analyzes all 6 dimensions that are studied in the current paper from the standpoint of the stakeholders of interest. In this article, the following questions are addressed: 1. How are dimensions defined by our sample of students? 2. What is the relative importance of each dimension when designing WBDL modules? To answer these questions our study was designed and conducted with the following steps: (1) Determine the different attributes used to define the design dimensions of interest by applying content analysis technique on selected papers in the literature and the responses of student participants of a laboratory experiment; (2) Summarize characteristics given for each design dimension using attributes; (3) Present the order of importance of the dimensions of design in WBDL modules according to the ranking given by the study participants. LITERATURE REVIEW In order to define the attributes of the design dimensions of interest (Aesthetics, Clarity, Excitement, Organization, Simplicity and Structure) a journal database was reviewed using the following keywords Design attributes, Distance learning problems, Interface Design, Web Distance Learning and Aesthetic Design. Only those journal M1G-1

articles related to the design of web modules or distance learning designs were considered suitable for analysis. A total of 15 journal articles were analyzed that are related to the design attributes of WBDL modules. Each paper was carefully analyzed by doing a content analysis, searching in the context for the dimensions of interest (Clarity, Simplicity, Organization, Structure, Excitement and Aesthetic Appeal/Attractiveness) and the words used to describe each dimension. If in the reviewed documents one of the dimensions under study was mentioned with at least one design attribute, then it was considered in our study. The six dimensions of interest are described below with their respective attributes as described in the literature. Aesthetic Appeal / Attractiveness has been defined in the literature through different design attributes such as balance and symmetry [13], and through the use of color and object placement [17]. Incorporating functional elements [18] as well as clear and quick messages is considered to be attractive [5] thus motivating the intended user. Clarity has been described in the literature by the concept of grouping objects and the use of interfaces that helps to understand information projected clearly [5], for example the use of pull down menus [19]. Clarity as a design dimension appears to strongly influence user reaction [20] and directly impacts user time to navigate an interface. Excitement as a design dimension is difficult to study as it is seldom used in the literature as an important design characteristic. Designers have suggested that by improving visual design of menus, icons and text, some excitement can be generated in the user [18, 21] as well as motivating the user by creating content that is humorous and fun to explore [18]. Other designers suggest that the appeal of the module [17], interactivity [18] and content flow [21] are design aspect that impact module excitement. Organization as a design dimension makes a Website functional and serviceable [18]. The literature suggests that the use of line borders to direct the viewer s eyes to the required direction of information scanning, can enhance organization [25]. The dimension of Simplicity has been widely studied. Color has been described as an attribute that influences appeal as well as ease of access to information [17]. Through simple and reliable design structure, functional stability can be achieved in a layout design [4]. Galitz [5] suggested that the most desirable screen features are simplicity, clarity and understandability, thus defining simplicity as a combination of elements that results in ease in comprehending the meaning of a pattern. From a design perspective [25] have suggested that simplicity can be achieved by designs with narrow text (45 characters), double spacing and organized paragraphs to create a feeling of ease, spaciousness and simple manageable text. From another perspective [14] defined simplicity as the extent to which component parts are minimized and the relationships between the parts are simplified. Ngo s [14] concept of minimizing the components of a screen is similar to [5] s idea of counting the total number of fields in a screen and choosing screens with minimal number of fields, [25] feeling of spaciousness, [4] keeping the page structure simple and uncluttered, and [7] use of white space to guide the eye and help users understand the grouping of information. As described in the literature, the dimension of Structure is very broad and has different ways of being described. Grabinger and Amadeo [25] proposed that structure can be described by text designs that indicate a hierarchical and systematic arrangement of subject material with isolated headings, spaced paragraphs and directive cues (underline, italics, and bold type). Stewart et al. [24] developed and validated an instrument for student evaluation of the quality of Web-based instruction in which appearance and structure of Web-pages was found to be an important dimension related to quality. Hong and Kim [22] experimentally compared the design of Web modules to the design of architectural structures, showing that robustness is related to reliability and security and that the navigation influences the utility of a Web module. These three authors present different perspectives on structure which reflect the diversity of perspectives found in design. METHODS To collect student perception and definition of WBDL module design dimensions and their importance, a group of students participated in an experiment in which they were exposed to a WBDL environment and responded to questions raised by the researcher [11]. I. Participants Sixteen engineering students between the ages of 18-22 years participated voluntarily in the experiment (8 females and 8 males). Participants were paid $15 each for 1.5 hours of their time. II. Module Design Thirty four images, simulating WBDL modules, were created using PowerPoint. These images were created with different information delivery modalities, color of text and figures in the content of the slides, and different slide text font size [11]. Information delivery was simulated through the use of video, audio only or text of the course material, found on the left side of the display. Course material was simulated using slides with bulleted text in all black fonts or with selected words in color or bold font. Slides with figures were represented by a graph in either grayscale or color, using the same set of colors used for colored text. III. Experimental Procedure In a well-lit laboratory, participants sat at a desk facing four 17-inch LCD monitors at 1024 x 768 pixel resolution. PowerPoint software was used to display the simulated WBDL modules, one at a time in slide-show or full screen mode. During the experiment, data were collected by manually recording verbal responses to standard questions about the images viewed [11]. Each participant was instructed to avoid thinking too much about their responses, M1G-2

and to take into account the different types of attributes of design of the Web modules when answering the research questions. After the experiment, each participant was asked to rank a list of six design dimensions in order of importance. The dimensions were clarity, organization, structure, visual/aesthetical attractiveness, simplicity and excitement. Then, they were asked to explain the meaning of each dimension in the context of the Web-modules seen in the experiment. IV. Data Analysis Methods Each definition or description given by the participants to explain the meaning of each design dimension was carefully analyzed by two researchers, using content analysis methodology. Each researcher individually assigned a primary attribute that was most related with the definition and a secondary attribute that supported the description given by the participants. After the individual assignment, the two researchers compared notes and decided as a team to select the final attribute that would be used to summarize each participant s description. The attributes selected by the investigators are then compared to the findings in the literature. A total of 215 definitions were provided by the study participants for the dimensions under study (Aesthetics, Clarity, Excitement, Organization, Simplicity and Structure). From those descriptions, 215 primary and secondary attributes were assigned by each researcher individually. Based on the primary and secondary attributes assigned to each dimension there was a high level of agreement (84.19%) between the researchers. Finally the rankings given to the 6 design dimension by the 16 participants were analyzed based on mean scores, to determine relative level of importance between the design dimensions. Conjoint analysis was used to determine the predicted rank of the design dimensions based on participant ranking results. RESULTS I. Agreement with literature Of the 215 descriptions collected for the attributes of design, 37 attributes summarized all the responses (shown in Table I). 21 (*) of these 37 attributes (57%) were in agreement between the different stakeholders. Agreement is defined as at least one participant and one research article describing a design dimension with the same attribute. These attributes (in alphabetical order) used by participants or evident in the literature are listed in Table I. It is interesting to point out that some of the attributes of design given by the participants and in the literature to describe each dimension are related to and in many cases similar to the design dimensions. For example, the simplicity dimension is described using the attributes clear and simple, which are also related to the clarity dimension. TABLE I LIST OF 37 DESIGN ATTRIBUTES Appeal/catching(*) Navigation(*) Appearance(*) Order Balance Organized(*) Borders Parallelism Bullets(*) Quality Clear(*) Readability/legibility(*) Color(*) Robustness Comfort/pleasing(*) Search/Find Content (writing quality, flow) Simple(*) Font(*) Spacing(*) Grouping(*) Stability Headings Structure(*) Humor Symmetry Informative Text(*) Interactive(*) Understand, comprehension(*) Interfaces Usability Layout(*) Visibility Less information/plain(*) Visuals(*) Linearity(*) II. Relative Importance (dimension ranking) For each dimension the average ranking between all participants was computed. Table II shows a summary of the rankings given by the participants. The responses in grey highlights the relative ranking of each dimension based on number of participants who gave the same ranking. Following the ranking scale, the dimension with the lowest value (closest to one) has the highest ranking and vice versa. The result of this analysis showed that the majority of the participants ranked clarity as the most important dimension related to the design of WBDL modules. The second most important dimension was organization and then simplicity. The dimension of simplicity had a high range of difference between the rankings given by the participants. The dimension ranked in fourth place was structure, also with a high range of difference between the rankings. The fifth dimension was aesthetics, and the sixth excitement. With respect to the other five dimensions evaluated, excitement seems to be not as important to the design of WBDL modules. The low range of difference between the rankings of participants for the dimension of excitement suggests that most participants agreed in their ranking. Conjoint analysis of the rankings for the 6 dimensions, in Table III, shows that the dimensions were ranked in the following order (most important first): clarity, organization, structure, simplicity, attractiveness, and excitement. This ranking order is very similar to the results obtained by the Conjoint Analysis shown in Table II. The main difference between the two methods is that the dimensions of simplicity and structure are in different order. M1G-3

TABLE II DIMENSION RANKING RESULTS Ranking Attract. Clarity Excit. Org. Simp. Struct. 1 1 8 0 2 2 3 2 1 3 0 7 3 2 3 3 2 0 3 4 4 4 3 3 1 2 3 4 5 5 0 4 2 3 2 6 3 0 11 0 1 1 Ranking Sum 67 32 90 43 53 51 Avg. 4.2 2 5.6 2.7 3.3 3.2 Std. Dev. 1.47 1.21 0.62 1.25 1.49 1.51 Variance 2.16 1.46 0.38 1.56 2.23 2.29 TABLE III UTILITIES TABLE BASED ON NON-METRIC CONJOINT ANALYSIS OF RANK DATA Variable Utility Predicted Rank (r 2 =0.5) Intercept 3.5 -- Clarity 1.2524 4.75 Organization 0.9299 4.4 Structure 0.4767 3.9 Simplicity 0.3448 3.8 Attractiveness -0.4928 3.0 Excitement -2.5064 0.99 DISCUSSION The different design dimensions of WBDL modules are related to a wide array of attributes of design. In this study, a total of 37 attributes were identified that describe the six different design dimensions of interest. According to the results in this study, both groups of stakeholders (students and the research literature) agree in associating the use of text and its different characteristics such as font, spacing, and color, with many of the design dimensions studied. On the other hand, none of the participants but several articles in the literature [13] [17] [22] [23] agree that simplicity, organization, structure and attractiveness are all medium for transmitting information. The participants in this study ranked the dimension of clarity as the most important dimension to consider in the design of WBDL modules. In the context of learning, clarity is an important dimension as the information presented needs to be clear, simple, organized, accompanied with visuals, understandable and comprehensible, and grouped appropriately. Visuals, as another important attribute of design, seems to impact most of the studied design dimension, except for structure. Various researchers [13], [17], [21], [22] and the participants in our study, all agree that the use of visuals in a module is a very important attribute for defining the dimension of aesthetic appeal/attractiveness. Understanding that engineering students strongly prefer information to be presented in visual form, the use of visuals can make a Web module not only more attractive but clear, simple, organized and exciting. This finding is important and should be considered to create attractive learning interfaces for engineering courses. Although excitement was considered by our sample as one of the least important design dimensions the content of the module (writing quality and flow) can have an impact on it. Also, the Structure dimension is controlled with all attributes related to the writing and content of the module that separate each element related to writing, picture, graphs and videos. In this study it surely seem that organization is very similar with structure because both are related to the writing content, but what makes differs between them is that the positioning of pictures, graphs, video and information influences the structure and not organization. Grouping of text, pictures and videos, and having a pleasant structure, which can be altered with colors or visuals, the dimension of organization can be met. According to [4], [5], [7], [14] the dimension of Simplicity is accomplished with the amount of information presented and the ease of retaining the information, finding information and ease of visuals for the eyes. Simplicity was also related to all the elements of writing and visuals of a Web module that make a learning experience easier and pleasant. Although Clarity and Simplicity are both related to the writing and visuals of Web modules, Clarity is different from simplicity because it is related to the understanding and organization of the content. On the other hand, at times clarity triggers simplicity and vice versa. This study concluded that there is a difference between the dimensions of structure and organization, and simplicity and clarity. Also it seemed that many participants as well as researchers defined how important the dimension of attractiveness was to the design of a Web module. By controlling the dimensions of organization, structure, simplicity and clarity the attractiveness/aesthetic appeal in a WBDL can be directly influenced through the following design attributes: Font, Bullets, Spacing, Color, Visuals, Navigation, Clear, Organized, Layout, Grouping, Order, Appearance, Visibility, Interactive, Quality, Humor, Borders, Headings, Symmetry, Balance, Text, Interfaces, Robustness, Parallelism, Simple, Search/Find, Usability, Stability, Informative, Linearity, Structure, Less information/plain, Appeal/catching, Comfort/pleasing, Readability/legibility, Understand, comprehension, Content (Writing quality, flow). The design attributes mentioned above are critical in the design of enhanced educational environments. Although the consideration of these attributes would impact any educational environment, the results are based on an experimental study where only engineering students participated. This study provided evidence to support the importance of appearance in design of WBDL environments and M1G-4

understand how guidelines in screen design and multimedia could be applied to the design of WBDL environments. Attractiveness was described by both stakeholders using the attributes of color and visuals, which are also important when creating a WBDL module that is considered clear, simple, organized and exciting. Even though attractiveness might not be considered one of the important dimensions based on the rankings given by the students who evaluated the module, as modules are made Attractive following the results of design attributes in this study, then desirable qualities of the other design dimensions are achieved. The authors understand that there is a possibility to miss several important design attributes as there are a wide array of design characteristics and attributes that can be used to describe the dimensions under study. Relating the views of engineering students and researchers provide evidence to support that there is a common language between the two, but there are also differences on how these two groups define attributes related to design dimensions. Most of the attributes explained in this study are very useful in the definition of the dimension studied, and future research could examine additional design dimensions and attributes. REFERENCES [1] Hannafin, M.J. and S. Hooper. An integrated framework for CBI screen design and layout, Computers in Human Behavior, 5, 1989, 155-165. [2] Liu, Y. Engineering Aesthetics and Aesthetic Ergonomics: Theoretical Foundations and a Dual-Process Methodology, Ergonomics, 46, 2003, 1273-1292. [3] Liu, Y., The Aesthetic and the Ethic Dimensions of Human Factors and Design, Ergonomics, 46, 2003, 1293-1305. [4] Darlington, K., Effective Website developments: Tools and techniques, Pearson Education Limited, England, 2005. [5] Galitz, W.O., User-interface screen design, QED Information Sciences, Inc, 1993. [6] Grabinger, R. S., Screen layout design: Research into the overall appearance of the screen, Computers in Human Behavior, 5,1989, 175-183. [7] Nielsen, J. Designing Web usability: the practice of simplicity, Indianapolis, IN, USA: New Riders, 2000. [8] Mayer, R., Multimedia Learning, Cambridge University Press, 2001. [9] Miller, S. M., and K. L. Miller, Theoretical and practical considerations in the design of Web-based instruction, in Instructional and Cognitive Impacts of Web-Based Education, ed. B. Abbey, Hershey, PA: Idea Group Publishing, 2000, pp.156-177. [10] Willis, L.L. and Lockee, B.B., A pragmatic instructional design model for distance learning, International Journal of Instructional Media, 30, 4, 2003, 9-17. [11] Pomales-Garcia, C., and Y., Liu., Web-Based Distance Learning Technology: Interface Design Variables and their Effects, International Journal of Instructional Technology and Distance Learning, 3, 5, 2006. [12] Pomales-Garcia, C., and Y. Liu, Web-based distance learning technology: The impacts of Web module length and format, American Journal of Distance Education, 2007. [13] Bauerly, M., and Y. Liu., Computational modeling and experimental investigation of effects of compositional elements on interface and design aesthetics, International Journal of Human-Computer Studies, 2006, 64, 8, 670 682. [14] Ngo, D. C. L., Measuring the aesthetic elements of screen designs, Displays, 22, 2001, 73-78. [15] Berge, Z.L., Collins, M., and Dougherty, K., Design Guidelines for Web-based Courses, In B. Abbey (Ed.), Instructional and Cognitive Impacts of Web-Based Education, Hershey, PA, USA.: Idea Group Publishing, 2000, pp. 32-40. [16] Rogers, P.L. Layers of Navigation for Hypermedia Environments: Designing Instructional Web Sites, In B. Abbey (Ed.), Instructional and Cognitive Impacts of Web-Based Education, Hershey, PA, USA.: Idea Group Publishing, 2000, pp. 217-226. [17] Udsen, L., and A. Jørgensen. The aesthetic turn: unraveling recent aesthetic approaches to human-computer interaction, Digital Creativity, 16, 4, 2005, 205-216. [18] Zhang, P., and G.M. von Dran, Satisfiers and Dissatisfiers: a Two- Factor Model for Website Design and Evaluation, Journal of the American Society for Information Science, 51, 14, 2000, 1253-1268. [19] Chiu, C., C. Chuang, and H. Hsiao. An interface design for a structured computer-mediated communication tool for elementary school students: pull-down versus explicit menus, British Journal of Educational Technology, 37, 2, 2006, 303-306. [20] Champness, B.G. and M. De Alberdi, Measuring subjective reactions to teletext page design. Alternate Media Center, New York University. 1981. [21] Brown, D., and E. Cameron, Designing the interface. International Review of Law, Computers & Technology, 19, 1, 2005, 65-81. [22] Hong, S., and J. Kim, Architectural criteria for Website evaluation - conceptual framework and empirical validation, Behaviour & Information Technology, 23, 5, 2004, 337-357. [23] Min, R., T. Yu, G. Spenkelink, and H. Vos, A comparison of parallelism in interface designs for computer-based learning environments, Journal of Computer Assisted Learning, 20, 5, 2004, 360-367. [24] Stewart, I., E. Hong, and N. Strudler, Development and Validation of an Instrument for Student Evaluation of the Quality of Web-Based Instruction, The American Journal of Distance Education, 18, 3, 2004, 131-150. [25] Grabinger, R.S. and D. Amadeo, CRT Text Layout: Perceptions of Viewers, Computers in Human Behavior, 4, 1988, 189-205. AUTHOR INFORMATION Cristina Pomales-Garcia, Assistant Professor of Industrial Engineering, University of Puerto Rico Mayagüez, cristina.pomales@upr.edu. Yili Liu, Professor of Industrial and Operations Engineering, University of Michigan, Ann Arbor. Angel D. Lopez, Undergraduate Student, Industrial Engineering, University of Puerto Rico Mayagüez. M1G-5