Usability Evaluation of Universities Websites

Transcription

1 1 Jabar M. A., 2Usman Abbas Usman, 3Sidi F. Department of Information System, upm.edu.my 2 Department of Information System, abbasusman4real@yahoo.com 3 Department of Computer Science, fatimah@ upm.edu.my Faculty of Computer Science and Information Technology Universiti Putra Malaysia 1 Abstract The goal of every academic websites is to provide valuable academic information to its users without any difficulties. However, there is a concern as to whether or not these websites are usable and meet the expectations of their intended users. This study measured the usability level of three university websites from the perspective of 351 students, using a questionnaire based on research model of measuring website usability based on factors of Content, Organization and Readability, Navigation and Links, User Interface Design, Performance and Effectiveness. The result identified the strengths and weaknesses associated with each Therefore this model can serve as guideline for evaluating website usability in order to know if a particular website has meet the need of its intended users and also assist the web designers in building more usable website. Keywords: Usability, University Websites, Usability Attributes 1. Introduction The university website is a means of communication with lecturers, students, faculties, alumni and guests. The web can be used to entice students, lecturers and attract funding from other places, increasing popularity of institutions all over the world [1]. Usability is one of the most important features of any user interface and measures how easy the interface is to use [2]. The website usability is an assessment on how successful a user is when finding information or performing some task in the website [3]. According to ISO part 11, usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [4]. If web site design is easy and simple, users are more likely to use it [5]. Most problems occur from the fact that designers construct a system from their own perspective. Evaluation of the usability of educational websites could be categorised into three with regards to the research techniques employed in identifying usability problems; evaluator-based methods; user-based (usertesting) methods; and tool-based methods. However, this study involves the second category of usability evaluation method which is user based testing and questionnaire based approach. The study adopted a questionnaire developed by [6]. The aim of this study is to measure the usability of educational websites from the viewpoint of students. The specific objectives for the research were to apply usability criteria for evaluating the usability of educational A total number of 351 students were asked to provide ratings for three Malaysian university websites and report qualitatively what they liked and disliked regarding the design of the tested This study chooses the instrument for evaluating websites usability called WEBUSE (standing for WEBsite USability Evaluation), developed by [6] and classified under four categories: 1. Content, organisation and readability; 2. Navigation and links; 3. User interface design and 4. Performance and effectiveness. 2. Literature Review Usability evaluation of websites has been conducted over the years by different researchers in different domains. For example, a study by [7] evaluated the usability of four Malaysian online news websites: The Star, The New Straits Times, Berita Harian, and Utusan Malaysia newspapers. The results of the analysis showed the good and the bad of the usability aspects of these websites in terms of the usability criteria can be used by website designers and developers to improve the The study by ([8] investigates usability of government websites in Uganda. Using the feature investigation International Journal of Information Processing and Management(IJIPM) Volume 5, Number 1, February

2 method, four Ugandan government websites were evaluated focusing on three perspectives which are design layout, navigation and legal policies. It was observed in the results that websites are partially usable in the perspective of both navigation and design layout but weak in terms of stating legal policies. Another study by [9] investigates the usability of the website of the Department of Marketing and Retail Management (DMRM) through the factors: effectiveness, efficiency and satisfaction from a perspective of marketing students. Similar methodology as in [6] and [10] was used in the study except that the method of heuristic evaluation was adopted for the DMRM s website instead of implementing it as a survey instrument targeted at students. The results revealed that, the overall feel and look of the DMRM website is consistent and therefore acceptable, DMRM website has poor navigation and content which reflects on the corporate website. Study conducted by [10] evaluates the usability of Jordan s universities websites from the perspective of usability only. The evaluation methods that were employed for this study are online automated tools and questionnaire method. Results indicated that both methods are reasonably acceptable when evaluating the usability of Jordanian universities However, weaknesses were identified in terms of the following aspects; performance, interface and design Abdul Aziz et al, in [11] investigate the usability of Malaysian university website. The authors concluded that still there are several issues that need to be addressed in these websites and also gave some suggestion on how to increase the usability of website. The study by Bairamzadeh & Bolhari, in [12], investigates factors affecting students' satisfaction of university website, and suggested that website innovation should be given more consideration in order to enhance students' insight of satisfaction, usefulness, trust. 3. Methodology The study adopted a questionnaire developed by [6]. A total number of 351 students were selected to answer questionnaires for evaluating websites usability called WEBUSE (standing for WEBsite USability Evaluation), based on Content, organisation and readability, navigation and links, user interface design and performance and effectiveness. The first section of the questionnaire addressed the demographic of the respondents, while the second section includes twenty four questions that were used to evaluate the usability of educational Open-ended questions on design features were also included Research Model Content, organization & readabilty Navigation & links Universities User interface design websites usability Usability Performance & effectiveness Figure 1. Universities Website Usability Model Figure 1depicted the research model which consists of dependent variable (usability) and independent variable : Content, organization & readability, navigation & links, user interface design and performance & effectiveness. These usability criteria can serve as a guideline and can also be used for measuring usability level of universities 11

3 3.2. Research Hypotheses The research predicts that there is significant difference in the four Usability categories across the three Universities websites and therefore, there is significant difference in usability across the categories of The hypotheses are clearly stated below: H0: There is no significant difference in Content, Organization and Readability across categories of H1: There is significant difference in Content, Organization and Readability across categories of H0: There is no significant difference in Navigation and Links across categories of Universities H2: There is significant difference in Navigation and Links across categories of H0: There is no significant difference in User Interface Design across categories of Universities H3: There is significant difference in User Interface Design across categories of H0: There is no significant difference in Performance and Effectiveness across categories of H4: There is significant difference in Performance and Effectiveness across categories of Universities H0: There is no significant difference in Usability across categories of H5: There is significant difference in Usability across categories of 3.3. Website Selection In order to select the top 3 Malaysian educational websites, one of the major international universities ranking websites was used; this was the 4International Colleges and Universities ( The list of Malaysian universities sorted by their web popularity provided by 4ICU (for the year 2013) was used to select the sample for this project. Three of these websites which had the highest ranking were then selected. This number was chosen to keep the study at a manageable size for the students and researcher. Table 1. Malaysian universities websites included in this study University University Website URL Universiti Teknologi Malaysia Universiti Malaya Universiti Putra Malaysia University Symbol UM Table 1, presents the Malaysian university websites involved in this study, together with their website URL and abbreviations used in the study. A pilot study was conducted among 30 undergraduate and postgraduate students randomly selected from seven (7) faculties in Universiti Putra Malaysia. The pilot study identified some ambiguities, the result was taken into consideration and minor corrections were made. Reliability test was conducted on the data that was obtained from the pilot study. Cronbach s Alpha Table 2. Reliability Statistics Cronbach s Alpha Based on Standardised Items N of Items 5 12

4 Table 2, depicted that the Cronbach s Alpha value of the questionnaire obtained from the pilot study is 0.914, this implies that the measurement instrument (questionnaire) adopted for this study are reliable. In order to select the university from which the respondents that participated in the study were chosen from, probability sampling called cluster sampling was applied. The three universities above were naturally grouped into three clusters. Cluster 1 (UM), cluster 2 () and cluster 3 (). Simple random sampling (SRS) was used to randomly choose one cluster out of three clusters and cluster 3 happens to be the chosen cluster. Cluster 3 was grouped into 2 clusters where cluster A consists of all the faculties and cluster B consists of all institutes in. Simple random sampling was again applied in which cluster A was chosen. Hence, the respondents that participated in the study were randomly selected from the 15 faculties in. The total population of students in is the sum total of undergraduates (18375) and postgraduates (9386) which gives In order to determine the required number of respondents (sample size), Slovin s formula (Ariola, 2006) was used. The population of students (27761) is approximately and has a corresponding sample size of 379. A total number of 456 questionnaires were distributed to the respondents out of which 351 participated in this study. 4. Results This section described the descriptive statistics of respondents, model adequacy checking and nonparametric tests. Statistical Package for Social Sciences (SPSS) version 20 was used for the data analysis Demographics The descriptive statistics of the respondents that participated in the study are displayed in Table 3. Factor Gender Category Internet Experience Table 3. Descriptive Statistics Category Male Female Undergraduate Postgraduate < 1year 1 5 years 6 10 years > 10 years Percentage A total number of 351 respondents participated in the study were 37% are male and 63% were female. 65.5% were undergraduate students and 34.5% were postgraduate students from various faculties in the university (). For internet experience, 2.6% of the respondents have less than 1 year, 30.2% have 1-5 years experience, 35.6% have 6-10 years experience and 31.6% have more than 10 years of internet experience Model Adequacy Checking In order to determine the statistical tests to be carried out on the data, we perform model adequacy check to determine whether the data are parametric or non-parametric. In order to determine whether variables are normally distributed, the significant value obtained from the test is compared with α. The significant-value obtained from the test for each variable is < α. It is therefore concluded that the variables are NOT normally distributed. The content, organization and readability together with user interface design have the significant-values of and respectively which is > α while navigation and links together with performance and effectiveness have the significant-values 0.32 and respectively which are < α. It was concluded that variables do not have homogeneous variances. Since the variables are not normally distributed and do not have homogeneous variables, nonparametric tests are employed in this study. Mann-Whitney is used but need to take into account the 13

5 problem of multiple testing. Three tests were conducted for each usability criteria and the overall usability, these tests are vs. UM, UM vs. and vs. therefore the significance level used for the Mann-Whitney tests throughout this study is 0.05/3 = The research hypotheses presented in section 3.3 were tested using kruskal-wallis test. Each hypothesis and corresponding test is elaborated below: Hypothesis 1 H0: There is no significant difference in Content, Organization and Readability across categories of H1: There is significant difference in Content, Organization and Readability across categories of A summary of the ranked data for each university websites in terms of Content, Organization and Readabilit, with its associated degree of freedom of 2 has an asymptotic significance of which is less than α, therefore we Reject H0 and accepts H1. To determine where the difference are, the statistics, we found out that the Content, Organization and Readability of UM website is significantly different from those of and websites, whereas Content, Organization and Readability of website has no significant difference from that of website. Hypothesis 2 H0: There is no significant difference in Navigation and Links across categories of Universities H2: There is significant difference in Navigation and Links across categories of The summary of the ranked data for each university websites in terms of Navigation and Links, with its associated degree of freedom of 2 has an asymptotic significance of which is less than α, therefore we Reject H0 and accepts H2. To determine where the difference are, Mann-Whitney tests were conducted and from the tests statistics, Navigation and Links of UM website is significantly different from those of and websites, whereas Navigation and Links of website has no significant difference from that of website. Hypothesis 3 H0: There is no significant difference in User Interface Design across categories of Universities H3: There is significant difference in User Interface Design across categories of The summary of the ranked data for each university websites in terms of User Interface Design, with its associated degree of freedom of 2 has an asymptotic significance of which is less than α, therefore we Reject H0 and accepts H3. To determine where the difference is, Mann-Whitney tests were and from the tests statistics, User Interface Design of UM website is significantly different from those of and websites, whereas User Interface Design of website has no significant difference from that of website. Hypothesis 4 H0: There is no significant difference in Performance and Effectiveness across categories of H4: There is significant difference in Performance and Effectiveness across categories of Universities The summary of the ranked data for each university websites in terms of Performance and Effectiveness, with its associated degree of freedom of 2 has an asymptotic significance of which is less than α, therefore we Reject H0 and accepts H4. To determine where the difference lies, MannWhitney tests were conducted, and from the tests statistics, Performance and Effectiveness of UM website is significantly different from those of and websites, whereas Performance and Effectiveness of website has no significant difference from that of website. 14

6 Hypothesis 5 H0: There is no significant difference in Usability across categories of H5: There is significant difference in Usability across categories of The summary of the ranked data for each university websites in terms of Usability, with its associated degree of freedom of 2 has an asymptotic significance of which is less than α, therefore we Reject H0 and accepts H5. To determine where the difference lies, Mann-Whitney tests were conducted and from the tests statistics, Usability of UM website is significantly different from those of and websites, whereas Usability of website has no significant difference from that of website Discussions Usability of Universities websites This subsection presents the results obtained from the analysis of the quantitative and qualitative data obtained from the respondents regarding the usability of the three Malaysian university No Table 4. Usability categories and criteria for the University websites Usability Category 1 Display space Content, organisation, & Readability 2 Scroll left and right Accessibility Distracting or irritating elements Orphan page Placement and content of site map or menu Information search 8 Link colours Up-to-date information Download time Back button Open new browser windows Respond according to users expectations Web advertising Follow real world conventions Hyperlink description Consistent design Use of colour Organisation of information Navigational aids Usability Criteria Navigation & Links User Interface Design UM,, Performance & Effectiveness UM,,, UM,, UM,, UM,, UM, The usability level in terms of the four usability categories and criteria in relation to the three university websites from Table 16 are summarised below: Content, Organization, Readability: There are eight usability criteria in this category, out of which websites satisfies all the eight criteria, website satisfies seven criteria and UM website only satisfies one criterion. Therefore website had the highest ratings compared to the 15

7 other websites in terms of its content, organization & readability; website had the second highest ratings. However, UM website had the lowest ratings. Navigation and Links: There are nine usability criteria in this category, out of which websites satisfies all the nine criteria, website satisfies seven criteria and UM website only satisfies one criterion. Therefore website had the highest ratings compared to the other websites in terms of its navigation & links; website had the second highest ratings. However, UM website had the lowest ratings. User Interface design: There are seven usability criteria in this category, out of which websites satisfies all the seven criteria, website satisfies five criteria and UM website only satisfies one criterion. Therefore website had the highest ratings compared to the other websites in terms of its user interface design; website had the second highest ratings. However, UM website had the lowest ratings. Performance and Effectiveness: There are fifteen usability criteria in this category, out of which websites satisfies all the fifteen criteria, website satisfies twelve criteria and UM website satisfies three criteria. Therefore website had the highest ratings compared to the other websites in terms of its user interface design; website had the second highest ratings. However, UM website had the lowest ratings. The strengths and weaknesses related to issues of usability in the universities websites which were obtained from the qualitative data (open-ended questions). The strengths or strong points of the universities websites were in its support of English language, social network connectivity to Facebook, Twitter and YouTube. While the common weaknesses found were in the font size and the overcrowded of advertisements in websites and unappealing aesthetic features Conclusion The results has identified the strengths and weaknesses associated with each Therefore this model can serve as guideline for evaluating website usability in order to know if a particular website has meet the needs of its intended users or not, assist the web designers to know the usability aspects that need to be improve and in building more usable website. Educational websites designers are advised to focus on issues such as font size, colours, language support and social network connection. It is also recommended that students should be allow to participated in designing educational website since they are the primary users of these websites, their suggestions and criticisms regarding the website should be looked into which can serve as guide to the designers. 5. References [1] Mentes, A. P. D. S. A. & Turan, A. P. D. A. H. (2012). Assessing the Usability Of University Websites: An Empirical Study On Namik Kemal University. Tojet, 11(3). [2] Nielsen, J., Usability 101: Introduction to usability. < [accessed ]. [3] Yusof, U. K., Khaw, L. K., Ch'ng, H. Y., & Neow, B. J. (2010, June). Balancing between usability and aesthetics of web design. In Information Technology (ITSim), 2010 International Symposium in (Vol. 1, pp. 1-6). IEEE. [4] ISO (1998) ISO : Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs). Part 11: Guidance on Usability. Geneva: International Standards Organisation. Also available from the British Standards Institute, London. [5] Kim, Y. M. (2011). Users' perceptions of university library websites: A unifying view. Library & Information Science Research, 33(1), [6] Chiew, T. K., & Salim, S. S. (2003). Webuse: Website usability evaluation tool. Malaysian Journal of Computer Science, 16(1), [7] Abdullah, R., & Wei, K. T. (2008). Usability Measurement of Malaysia Online News Websites. International Journal of Computer Science and Network Security, 8(5), [8] Asiimwe, E. N., & Lim, N. (2010). Usability of government websites in Uganda. Electronic Journal of e-government, 8(1), [9] Du Toit, M., & Bothma, C. (2009). Evaluating the usability of an academic marketing department's website from a marketing student's perspective. 16

8 [10] Mustafa, S. H., & Al-Zoua bi, L. (2008, December). Usability of the Academic Websites of Jordan's Universities An Evaluation Study. In Proceedings of the 9th International Arab Conference for Information Technology (pp ). [11] Abdul Aziz, M., Wan Mohd Isa, W. A. R., & Nordin, N. (2010, December). Assessing the accessibility and usability of Malaysia Higher Education Website. In User Science and Engineering (i-user), 2010 International Conference on (pp ). IEEE. [12] Bairamzadeh, S., & Bolhari, A. (2010, July).Investigating factors affecting students' satisfaction of university In Computer Science and Information Technology (ICCSIT), rd IEEE International Conference on (Vol. 1, pp ).IEEE. 17