Usability of the Academic Websites of Jordan's Universities An Evaluation Study

Transcription

1 1 Usability of the Academic Websites of Jordan's Universities An Evaluation Study Suleiman H. Mustafa Loai F. Al-Zoua bi Faculty of Information Technology, Yarmouk University, Irbid, JORDAN Abstract The aim of this research study was to evaluate the websites of Jordan's universities from the usability perspective. Two online automated tools, namely: html toolbox and web page analyze were used along with a questionnaire directed towards users of these websites. Tools were used to measure the websites internal attributes which can not be perceived by users, such as html code error, download time, and size of html page. The questionnaire was developed and designed based on 23 usability criteria divided into 5 categories. Each category deals with one usability aspect. The results showed that the overall usability level of the studied Websites is acceptable. However, there are some weaknesses in some aspects of the design, interface, and performances. Suggestions are provided in the study to enhance the usability of these websites. Keywords Website Usability, Website Evaluation, University Websites. Jordanian Universities. 1. INTRODUCTION The Web is playing a main role in diverse application domains such as business, education, industry and entertainment. As a result, there are increasing concerns about the ways in which websites are developed and the degree of quality delivered. Developing a website should be passed through several design guidelines to ensure that the website can achieve the purposes and goals intended to be accomplished. Additionally, an organization s website is a gateway to its information, products and services. As such, it should ideally be a reflection of the needs of the clients it serves. Unfortunately, website design is often driven by technology, organizational structure or business objectives, rather than by users needs. However, in recent years website owners and developers have gradually begun to acknowledge and address the issue of usability [1]. Usability is considered as one of the major factors that determine the successfulness of a website. As defined by ISO , it is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [2]. Several evaluation methods have been proposed to assess the usability of websites in order to suggest enhancements to the design of websites. Some methods are address experts, while others are directed toward users. Such evaluation methods include Heuristics used by a group of experts with reference to established guidelines or design principles [3, 4, 5, 6, 7], Prototyping based on developing a mock site that can be shown to users before the real site is launched. Users can also participate in focus user groups or controlled laboratory sessions to provide usability feedback [7], Checklists of usability items [8], Cognitive walkthroughs involving one or a group of evaluators inspecting a user interface by going through a set of tasks to evaluate website understandability and ease of learning [9, 10], Questionnaires for extracting, recording, and collecting information to measure the user satisfaction with website usability [11, 12]. The usability of any website can be evaluated and determined using usability evaluation methods and techniques.

2 2 Generally, any website should meet the needs of its various stakeholders [13]. Educational Website users are mainly concerned with the following two major questions: 1. Can I find the information I am looking for in my website easily? 2. Can I find the information in timely manner? The Websites of Jordan's universities aim at providing up-to-date information and services to students, instructors, academic administrators and other users in efficient ways. There are, at present, twenty three universities in Jordan, each having its own website. Generally, usability studies of Arabic Websites and in particular academic Websites are almost nonexistent. The research reported in this paper was intended to point out the strengths and weaknesses of the usability aspects in the design of these academic websites, including interface, performance and effectiveness, content and organization, and the educational purposes for these websites. The results are supposed to draw the attention of webmasters to overcome the limitations of these websites and improve their efficiency. 2. RELATED WORK Evaluations of the usability of websites have been conducted over the years and for many domains. For example, Akoglu [14] developed a special tool for assessing the usability of architectural department websites in the University of Istanbul. The tool consisted of two parts; the first part can be accessed by users where they visit and answer the questions about the website, and the second part was constructed for the use of administrators where he/she can manage the content of the usability evaluation. The evaluation was based on two environments: traditional laboratory environment and Internet environment. Kirakowski et al. [15,16] evaluated the user satisfaction with usability of 5 websites based on a questionnaire method. The authors developed a new questionnaire ( named WAMMI) for the evaluation. For more information about WAMMI the reader is referred to ( The questionnaire showed that the evaluation of user satisfaction contributes to the successful development of websites. The research by Chiew and Salim [17] focused on developing a web-based tool (called WEBUSE) which consists of 24-questions for evaluating the usability of websites. The report generated by the tool indicates how good is the website with respect to usability. The researchers claim that WEBUSE is suitable for the evaluation of all types of websites and for any domain. The tool can assist webmasters to improve their websites based on the response provided by the visitors of the intended websites. In another research conducted by Harms and Schweibenz [18], the researchers used a combination of two methods of usability engineering for the web, namely the heuristics method and the laboratory-test method with actual users involved in evaluating the usability of the Saarland Museum's Website. The authors claim that this combination of methods provides good results, but it is costly in terms of time and resources. They also pointed out that one of the major drawbacks of the heuristics is that it cannot be applied successfully by novice evaluators since they need some background knowledge in Web design and evaluation. The results obtained from the evaluation led to the redesign of the Saarland Museum s Website. With regard to academic websites, the research conducted by a group from the University of Maryland [3] performed a usability evaluation for the Master of Information Management (MIM) (part of the College of Information Studies at the University of Maryland, College Park) program website to know whether their website meets the needs of its users. The methods used focused on conducting user testing tasks and performed heuristics evaluation for these tasks. A set of recommendations were provided to guide future redesign of the website. The researchers proposed a new heuristic evaluation technique based on modifications of two widely used heuristics evaluation techniques [4] for the evaluation process. As a result, the researchers suggested that the MIM website does not meet the needs of many of its users. Although the website does a good job of presenting information to prospective students, it does not fulfill the needs of current students and faculty. A similar case study was conducted by the Website Usability Testing Center at Wisconsin-Stout University [19] to evaluate the usability of their University s website The researchers used qualitative testing criteria such as navigation times through subject evaluations to assess the usability of the site. The study showed that the website suffers from several problems including website specific jargon, unorganized link patterns, confusing search engines, and poorly

3 3 emphasized information. Based on the results of the evaluation the authors proposed a number of recommendations in order to improve and unify the university website. A study by Gullikson [20] looked at assessing how the information architecture affected user navigation of an academic institution s Website. Participants answered a series of six questions using the site under a 3-minute time constraint for each question. User interaction with the site was recorded, and the subjects were asked to elaborate on the decisions they made during a playback of the testing session. The study tracked the number of answers found correctly, the pages viewed in the process, and the time between mouse clicks. Corry [21] conducted a usability evaluation of an existing Midwestern University Website. An analysis was conducted to restructure the information contained in the current Website; a prototype was developed and tested against the existing site. Usability was based on the ability of subjects (such as students, parents, and faculty) to quickly and accurately locates answers to a set of questions. While the study worked well, the metrics used to measure usability were limited to task completion time and the number of user errors. 3. QUESTIONNAIRE-BASED EVALUATION METHOD In this study, two website evaluation methods were used: the questionnaire and automated tools In order to select the appropriate website evaluation usability criteria for the questionnaire-based evaluation method, we identified and analyzed the usability aspects in terms of user satisfaction, readability, navigation and others related to the websites of Jordanian universities. Two forms were used to conduct our questionnaire: a paper form and an online form. The questionnaire was divided into two sections. The first section addressed the characteristics of participants, including: university name, internet and computer usage, age of participant, and the access frequency of participant university website. The second section included thirty questions that were used to evaluate the usability of Jordanian academic websites. The questions were classified into five categories (See Appendix 1): Content, organization, and readability (CAT 1), Navigation and links (CAT 2), User interface design (CAT 3), Performance and effectiveness (CAT 4), and Educational information (CAT 5). After an extensive study of related resources [16, 21, 22, 23, 24, 25], twenty three website usability evaluation criteria were identified as shown in Table 1. Each criterion was assessed, in terms of the five categories given above (as applicable), on a scale of five items as shown in Table 2. The responses were evaluated according to the merit points adopted by Chiew and Salim [17]. Table 1: The 23-Usability Evaluation Criteria No Usability Criteria 1 Display space 2 Scroll left and right 3 Accessibility 4 Distracting or irritating elements 5 Orphan page 6 Placement and content of site map 7 Information search 8 Link colors 9 Up-to-date information 10 Download time 11 Back button 12 Open new browser windows 13 Respond according to users expectations 14 Web advertising 15 Follow real world conventions 16 Hyperlink description 17 Consistent design 18 Use of color 19 Organization of information 20 Navigational aids 21 Registration information 22 Faculties information 23 Instructors information Table 2: Options for question and corresponding merits Option Merit Strongly Disagree (S.D) 0.00 Disagree (Dis) 0.25 Fair 0.50 Agree (Ag) 0.75 Strongly Agree (S.A.) 1.00 The response to each question is assigned a merit value. Merit points are then accumulated according to the five usability categories shown in Table 3. Mean value for

4 4 each category is considered as the usability index for that category. The usability index for a category, x, is defined as: x = [ (Merit for each question of the category)] / [number of questions] Table 3: Usability Categories and their Points Usability Category Point Level Content, Organization and Readability (CAT1) Navigation and Links (CAT2) User Interface Design (CAT3) Performance & Effectiveness (CAT4) Educational Information (CAT5) Overall Usability The usability level of each category is determined by the usability points. Table 4 shows the usability points and the corresponding usability level. The overall website usability index is the mean value of usability points for the five categories. Table 4: Usability Points and the Corresponding Usability Levels Points (x) Usability Level 0<=x<= <x<= <x<= <x<= <x<=1.0 Bad Poor Moderate Good Excellent 4. WEBSITE EVALUATION USING AUTOMATED TOOLS The first part of our evaluation methodology was intended to focus on how we can evaluate usability of the academic websites of Jordan's universities from the user s perspective. However, usability evaluation methods that are conducted by human intervention (users and experts) can assess only the external attributes of the website (such as readability of the contents of the website) rather than its internal attributes (such as textual duplicates of links embedded in images). External attributes depend on the website and its usage, while the internal attributes of the website only depend on how the website has been designed and developed [27]. These internal attributes of the website can be assessed and evaluated using automated tools. In this part of the evaluation, two automated online tools were used: Web Page Analyzer [28] and HTML Toolbox [29]). The internal attributes that can be measured include: Total number of HTML files on the page. The total number of objects on the page. The total number of images. Total size of images. Browsers compatibility. Total number of broken and bad links. Web Page Analyzer Web Page Analyzer is a free web-based tool provided by WebsiteOptimization [28]. It can calculate page size (Html page size, total image size, and total image number), composition, and download time for website. In this study, we measured the following attributes using this tool: Total number of html files Total html page size Total size of images Total number of images Total number of cascading style sheet files Total size of cascading style sheet files HTML Toolbox HTML Toolbox is available from NetMechanic Inc. It is an award winning product that identifies site problems and automatically repairs HTML code for the websites. This tool can assist in the evaluation process by measuring and identifying some of the internal attributes of a website. The internal attributes that were measured using this tool include: Download time HTML check & repair Browsers compatibility 5. RESULTS AND DISCUSSION 5.1 Questionnaire-Based Results A total of 252 users representing 9 different Jordanian universities have participated in the questionnaire-based evaluation method. 56.7% of the total number responded to the online questionnaire while 43.4% responded using the paper version of the questionnaire. Ages of most participants ranged between 18 and 25

5 5 years. More than 80% of the respondents were graduate and undergraduate students. As of gender, 67.9% of the respondents were males and 32.1% were females. Most of the participants had computer and internet experience for more than 4 years. For computer experience, 73% of the participants had 4 years or more and for internet experience 49.6% of the participants had 4 years or more. 2.8% of the participants had less than 1 year experience with computers and 4.8% of them had less than 1 year experience with internet. Table 5 presents a summary of the results obtained from the questionnaire-based evaluation method. Figure 1 shows the distribution of the responses over the five categories listed in Table 3. According to the 5-item scale, the results indicate that about 51% out of the total number of responses expressed satisfaction with the usability level provided by the websites covered in the study. In comparison, about 23% of the respondents were not satisfied and about 26% judged these websites as being "Fair" with respect to usability. Table 5: Results of the questionnaire (%) Categories Usability Scale Rate S.D Dis. Fair Ag S.A CAT CAT CAT CAT CAT Total ** For abbreviations see Table 2 and Table 3. graphically in Figure 2. The usability category "Content, Organization and Readability (CAT1)" exhibits the highest evaluation value, followed by the category "Navigation and Links (CAT2)", both being rated "Good" according to the scale used. The other three categories: "Educational Information (CAT5)", "User Interface Design (CAT3)", and "Performance & Effectiveness (CAT4)" were rated "Moderate". The overall usability mean value for the websites of Jordan's universities is 0.619, which is described as "Good" on the usability scale. Table 6: Usability Point and Usability Level for each Category Categories Usability Usability Point Level CAT Good CAT Good CAT Moderate CAT Moderate CAT Moderate Overall Good Usability Points ** For abbreviations see Table 2 and Table 3. Figure 2: Usability point for each category 5.2 Tool-Based Results In this part of the evaluation, two automated evaluation tools were used to assess website usability. The results obtained from HTML Toolbox are presented in table 7 and those of WebPage Analyzer are given in Table 8. Figure 1: Questionnaire-based results with respect to the 5 usability categories The usability level of each category, as determined by the usability points, is presented in Table 6 and depicted Table 7 shows that the average download time for all websites is second, and the average html check errors are 22, and the average browsers compatibility problems are 17. Each of the three evaluation measures is described bellow.

6 6 a. Load Time Page load time depends on several factors such as: the size of the HTML file and any frames it references, the number and size of the images, the use of HEIGHT and WIDTH attributes with image and table tags, the number of servers that must be contacted to download files and images, and the speed of the user's modem. For the websites covered in this study, the average download time was approximately 23 seconds on a 56k modem; which exceeds 15 seconds, the recommended acceptable level. This might be attributed the large number and size of images some of these websites contain. Table 7: Results obtained from HTML Toolbox Elements to be Measured University Website Load Time Html Browsers (Home Page) (56Kbps) Check compatibility Errors Problems Yarmouk problems JUST second 0 errors 2 problems Mu ta problems U. Jordan second 0 errors 2 problems P. Sumaya second 0 errors 28 problems Zaytoona problems Al-Balqaa problems Philadelphia problems Hashemite U problems Average Second b. Html Check Errors 22 errors 17 problems These errors are places where the web page does not follow the rules for proper HTML coding. These problems may cause the page to display incorrectly under different browsers. As Table 7 indicates, there were 22 html code errors in the websites covered in this study. Some of these problems represent places where HTML tags or attributes are used that does not follow the latest HTML standard and may not be supported by all browsers. c. Browsers compatibility Problems The measure, as assessed by the tool, shows how well the web page is displayed by different browsers. As Table 7 shows, 17 compatibility problems were found in the studied websites. This number represents problems which affect at least 10% of the website visitors. Table 8 summarizes the results obtained from WebPage Analyzer. A comment on each evaluation element follows. Table 8: the results of WebPage Analyzer of Universities Websites Elements to be Measured TH THS TIS TNI TCS F S University Website Yarmouk JUST Mu ta U. Jordan P. Sumaya Zaytoona TCS SS 3 364K 189K K 5 509K K 160K K 1 143K 139K K 33k K 242K K Al-Balqaa 4 454K 258K Philadelphi 1 83K 58K 5 1 1K a Hashemite 1 154K 64K 3 1 1K U. Average 2 218K K K THF: Total # of HTML Files THS: Total HTML Page Size (KB) TIS: Total Size of Images (KB) TNI: Total # of Images TCSS: Total # of CSS Files TCSSS: Total Size of CSS files (KB) Average Number of HTML Files: The average number of HTML files for the nine web pages (including the main HTML file) is 2 files which most browsers can multithread. Minimizing HTTP requests is a key for web site optimization. The tool used indicated that such a value is appropriate. Average Size of HTML Page: The average web page size is bytes, which will load in seconds on a 56Kbps modem as show in Table 7. Pages over 100K exceed most attention thresholds at 56Kbps, even with feedback. The tool issued a warning for such a value.

7 7 Average Size of Images: The average image size is bytes, which is over the acceptable threshold, which is 30K. The tool issued a warning for such a value. Average Number of Images: The average number of images is 12, which is not reasonable. Hence, the tool issued a warning for such a value. Average Number of CSS Files: The average number of external CSS files is 1. Because external CSS files must be in the HEAD of an HTML document, they must load first before any BODY content displays. Although they are cached, CSS files slow down the initial display of the web page. The tool indicated that such a value is appropriate. Average Size of CSS files: The average size of external CSS files is 3876 bytes, which is less than the threshold value (i.e., 4080 bytes). This will fit into three higher-speed TCP-IP packets. The tool indicated that such a value is appropriate. 6. CONCLUSION Jordanian universities websites are educational organizations that aim to provide the information and services to its stakeholders in efficient ways. To achieve their purposes, universities websites design should go through several design guidelines to ensure that users are more satisfied with the services provided by these websites. By evaluating the usability aspects of these websites we can improve the usability of these websites. In this study, we used two evaluation approaches: the evaluation based on questionnaire method and the evaluation based on online automated tools. The results obtained from both approaches showed that the usability of Jordan's university websites reasonably acceptable, based on the measures of evaluation used. The study covered various aspects of aspects of usability, some of which have been found not at the acceptable level of performance. It has been pointed out that some parameters of evaluation have satisfied when compared with the acceptable threshold values. It should be noted, however, that the results of this study should viewed with the fact that the design of some websites undergoes changes from time to time. REFERENCES 1. Pierce K. R. (2005). Web Site Usability Report for Harvard University", Capella University. 2. Jokela, T., Iivari, N., Matero, J. and Karukka, M. (2003). The Standard of User-Centered Design and the Standard Definition of Usability: Analyzing ISO against ISO , Oulu University, Nokia 3. Nielsen, J. (2001). How to Conduct a Heuristic Evaluation, (visited on 10/4/2007) 4. Nielsen J. and Molich R. (1990). Heuristic Evaluation of User Interfaces, Proceedings of CHI 90, Denemark 5. Muller, M. J., Matheson, L., Page, C. and Gallup, R. (1998). "Methods & tools: Participatory Heuristic Evaluation, interactions. 6. Michael D., Conard L. and Conard F. (1996). "A Heuristic Evaluation of a World Wide Web Prototype, interactions. 7. Kantner, L. and Rosenbaum S. (1997). Usability Studies of WWW Sites: Heuristic Evaluation vs. Laboratory Testing, Proceedings of the Fifteenth Annual International Conference on Computer Documentation. 8. Keevil B. (1998). Measuring the Usability Index of Your Web Site, Proceedings of the Sixteenth Annual International Conference on Computer Documentation, Canada. 9. Polson P., Lewis C., Rieman J. and Wharton C. (1992). Cognitive walkthroughs: a method for theorybased evaluation of user interfaces, International Journal of Man-Machine Studies. 10. Lewis, C., Polson P., Wharton C. and Rieman J., (1990). Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces, Proceedings of CHI 90, ACM. 11. Lewis J. (1995). IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use, International Journal on Human-Computer Interaction. 12. Perlman G. (1998). Web-Based User Interface Evaluation with Questionnaires, accessed in 20 th March Thompson T., Hourclé J. and Ma L. (2005). "Website Usability Evaluation", University of Maryland. 14. Akoglu C. (2002). Usability Evaluation: A Method for A Specific Field", Yildiz Technical University, Istanbul, Turkey. 15. Kirakowski, J. Claridge, N., and Whitehand, R. (1998). Human Centered Measures of Success in Web Site Design, Conference Proceedings: 4th Conference on Human Factors and the Web, USA. 16. Kirakowski J. (2000). Questionnaire in Usability Engineering: a List of Frequently Asked Questions (3 rd Ed.), Human Factors Research Group, Ireland. 17. Chiew K. T. and Salim S. S. (2003). WEBUSE: Website Usability Evaluation Tools, University of Malaya.

8 8 18. Harms, I. and Schweibenz, W. (2001). Evaluating the Usability of A Museum Website, University of Saarland Germany. 19. Smith, M., Rougier, B., Hamman, D., McKenzi, J., Johnston, B. and Maylath, B. (2001). Website Usability Evaluation of uwstout.edu, The University of Wisconsin-Stouta, Website Usability Testing Center. 20. Gullikson S., Blades R., Bragdon M., McKibbon S., Sparling M. and Toms G. (1999). "The Impact of Information Architecture on Academic Web Site Usability", the Electronic Library. 21. Corry D., Frick W. and Hansen L. (1997). "User- Centered Design and Usability Testing of a Web Site: An Illustrative Case Study", Educational Technology, Research & Development. 22. Paterno F. and Leporini B. (2004). Testing the Effects of Web Usability Criteria for Vision Impaired Users, ISTI-C.N.R, Italy. 23. Granic A., Glavinic V. and Stankov S. (2003). Usability Evaluation Methodology for Web-Based Educational Systems, Croatia. 24. Sinha R., Hearst M. and Ivory M. (2001). Content or Graphics? An Empirical Analysis of Criteria for Award-Wining Websites, University of California, Berkley. 25. Marsico M. and Levialdi S. (2003). Evaluating Web Sites: Exploiting User s Expectations, University of Roma, Italy. 26. Withrow J., Brinck T., and Speredelozzi A. (2001). "Comparative Usability Evaluation for an e- Government Portal", Diamond Bullet Design in Collaboration with the National Information Consortium, the National Information Consortium. 27. Brajnik G. (2000). Automatic web usability evaluation: what needs to be done?. In Proc. Human Factors and the Web, 6th Conference, Austin TX. 28. WebsiteOptomization Inc. (2007). Web Page Analyzer, (Visited on ) 29. NetMechanic Inc., (2007), HTML Toolbox, (Visited on ). 30. Nielson J. (2002). Top Ten Guidelines for Homepage Usability, (Visited on ). 31. Leavitt M. and Shneiderman B. (2006). Research- Based Web Design and Usability Guidelines, Secretary of Health and Human Services, Professor of Computer Science University of Maryland, U.S. Government Printing Office, Internet: bookstore.gpo.gov Appendix 1: a. Questions for evaluating content, organization and readability This website contains most of my interest material and topics and they are up-to-date.. I can easily find what I want at this website The content of this website is well organized. Reading content at this website is easy. I am comfortable and familiar with the language used. I need not scroll left and right when reading at this website. b. Questions for evaluating navigation and links I can easily know where I am at this website. This website provides useful cues and links for me to get the desired information. It is easy to move around at this website by using the links or back button of the browser. The links at this website are well maintained and updated. The website does not open too many new browser windows when I am moving around. Placement of links or menu is standard throughout the website and I can easily recognize them. c. Questions for evaluating user interface design This website s interface design is attractive. I am comfortable with the colors used at this website. This website contains no feature that irritates me such as scrolling or blinking text and looping animations. This website has a consistent feel and look. This website does not contain too many Web advertisements. The design of the website makes sense and it is easy to learn how to use it. d. Questions for evaluating performance and effectiveness I need not wait too long to download a file or open a page. I can easily distinguish between visited and notvisited links. I can access this website most of the time. This website responds to my actions as expected. It is efficient to use this website. This website always provides clear and useful messages when I do not know how to proceed. e. Questions for evaluating education purpose

9 I can easily access the registration page and I can easily register for semester. When I need to register, the website provides information about what the courses are offered and who is teaching the courses. This website is regularly updated in terms of personnel and course information in order to keep their information up-to-date I can easily contact with my instructors because this website provides information about instructors office location and hours, and addresses. This website suffers from problems during registration process for students. I know who I can contact for more information about anything in this website. 9