ICT Innovations 2011 Web Proceedings ISSN 1857-7288 67 Exploring Students' use of e-learning systems in Higher education Maja Ristova 1, Biljana Citkuseva 1, Vladimir Trajkovik 2, 1 Faculty of Electrical Engineering University Goce Delcev, Stip 2 Faculty of Electrical Engineering and Information Technology Ss Cyril and Methodius University, Skopje Abstract. In this paper we will present results of our evaluation of e-learning system usage in different education institutions in R. Macedonia. We will present the results we obtained using evaluation approach based on students quality of experience. This method of evaluation emphasizes on E-learning systems acceptability and adaptability to students needs. Keywords: e-learning, higher education, Moodle, FEITle, LMS, Quality of Experience 1. Introduction In the last years, e-learning has been gained a huge acceptance among educators, especially in Higher Education Institutes (HEI). This happened due to the exponential growth of the use of Internet and educational technologies that allow deep transformations in the teaching-learning process paradigm. Internet has been reinforcing its role as a mediator-facilitator platform in education, allowing the creation of the new context where learning opportunities can be expanded and potentially collaborative and where the resources are widely available [1]. In the Republic of Macedonia, this process is especially emphasized in the last 10 years, through the implementation of several projects of national interest. These projects are typically funded by big funding institutions such are: USAID, European Council and Ministry of Education and Science of R. Macedonia. Most of the projects accepted approach defined by adoption of the open source platform Moodle (Modular Object- Oriented Dynamic Learning Environment) and the education of specialized staff to assist teachers and students on a daily basic. At the same time, some higher education institutions are developing their own specialized LMS, providing different ways of interaction and possibility to integrate to more than one LMS using open standards such are: SCROM and learning objects. This paper aims to compare Quality of Experience provided to students when using two approaches for establishing e-learning systems described in previous paragraphs.
68 ICT Innovations 2011 Web Proceedings ISSN 1857-7288 2. LMS compared 2.1. Moodle Moodle has interesting characteristics as it is a LMS that offers a general set of activities adaptable to the content of the most diversified subject. It has several activities that can be used as educational instruments. It is possible to add different documentation to the subject, to suggest assignments, promote discussion forums and to create term glossaries in a cooperative form. The assessments and the self-evaluation of knowledge are done through the pages consisting of lessons and questionnaires. In order to create a dynamic use of Moodle, training sessions on continuous base are organized. At the same time different approaches are used to motivate teachers in the usage of such instruments, generation more creative educative offers for the students. 2.2. FEIT learning environment FEIT Learning Environment (FEITle) is online system, environment for communication, exchanging materials and ideas within the courses in the Faculty of engineering and information technology, Skopje. Students can access to all materials added into the courses (books, presentations, test results, exams, materials for laboratory exercises and etc), access to announcements for the activities of the courses, access to Wiki and Forums, where all the members of one course can communicate and exchange ideas between each others, review of their results, and access to calendar where the individuals can see the certain dates of activities through the year. The FEITle system is released in use from 2006, where all the teachers and students use it daily. 3. Quality of experience Our growing experience with a variety of technology, in particular social software, has led to a digital literacy through which end users are quick to access whether the software will meet their needs. We have all experienced the difference between a website that is inviting, as well as those that are unorganized or messy. The quality of experience is center of our exploration and our learning. Quality of experience is the sum of content and learning situation and social interaction for students. In an online learning environment, it is the sum of interaction, interactivity design, social connection, and pedagogy, [2]. Gaggioli et al.(2003) presented a model in which they argue the importance of understanding quality of experience as both a psychological and cultural phenomenon. To understand an experience, one has to understand the context, as well as personal
ICT Innovations 2011 Web Proceedings ISSN 1857-7288 69 challenges and skills processed by user. In an online learning environment, learning includes variables other than pedagogy and content and didactics. It includes the culture, the communicative, and the psychological. In the following figure is presented a model where we attempt to interconnect the difference dialogues in quality both from education, technology and business sectors to suggest the need to further explore quality of experience as an essential factor in measuring learning in technology supported learning environments, [3, 4]. Fig. 1. Quality of experience Therefore, factors that impact quality of experience are presented in a circular form like: access to technology, user friendless, audio quality, visual quality, reaction time, authenticity, human and technology presence, as well as learning style, communication style, learner expectation, experience with digital media, and interactivity design. 4. The study The main goal of our study is to provide feedback from the students and teachers, to summarize their opinion about learning systems. So we create a survey and delivered it to the learners. The implementation of the survey is based on e-learning system implemented at educational institutions where the first implementation is massive and institutional (UGD), and second is not massive and incident (FEEIT). At UGD is
70 ICT Innovations 2011 Web Proceedings ISSN 1857-7288 implemented Moodle open source e-learning system, they are 13 th faculties that are using this system or that is almost 10 000 users students and teachers, and FEETLE is system which is implemented only in FEEIT, Skopje, where the numbers of users reach 4 000. The functionality of the systems in both cases is in high-level. 4.1. Participant The participants are full time registered students. We collected 190 answers, 40 from University Goce Delchev, Shtip - (UGD), and 150 answers from students at Faculty of Electrical Engineering and Information Technology - (FEEIT) from University Cyril and Methodius, Skopje. 4.2. Method The research presented in this paper is based on the implementation of the e-learning system at UGD and UKIM, and achieved result of his usage. The main aim of the survey is to provide feedback from learners, which is one of the requirements of the learner-centered learning paradigm. We wanted to check, whether the system satisfies the needs of the students, and whether it facilitates and enhance the way of learning. From the result of the survey we have solid results; about how much students are familiar with the system, how often they used it and for what, and do they like the purposes of such system. 4.3. Research questions Evaluation of the survey is based on two parts: knowledge and usability. On the knowledge part were created questions in order to provide information about student s familiarity of e-learning system implemented on their faculties, and on the part of usability was present evaluation sought to ensure that the system not only functions as expected by the users but is also usable. Results show that Moodle system (e-learning system installed at UGD) and FEITle system (implemented at FEEIT), are suitable e- Learning systems which contribute interests and feedbacks from students and teachers side. All of the questions were in the e-learning area and not all questions were opened for the students. The availability of the questions is based and depends on previous answer(s). 4.4. Results and discussion First part of the survey is based on nine questions; in both cases, 89% of students were male between 20 and 30 years. The third question show, that mostly of the people who made the survey are full time students and only 12% are part time.
ICT Innovations 2011 Web Proceedings ISSN 1857-7288 71 With the next question we wanted to check the degree of their education, 85% were undergraduate students, 15% were in master studies, and none of the people that participated in the survey were in doctorial studies. Then follow a question only for undergraduate students, to check the year of studies, and 37% answered in second year, 43% in third year, and 1% first and the rest forth year. The major part people that participate in the survey are experience student and they have to be familiar with their studying process in their faculties. 80 60 40 20 I II III IV 0 Fig. 2. The year of studies of undergraduete students The next question is about if whether they have basic knowledge about the implemented e-learning system in their educational institution, 100% answered positive at UGD and 79% at FEEIT, 16% negative, and 5% didn t know. More than 20% didn t know about the situation in their faculties.
72 ICT Innovations 2011 Web Proceedings ISSN 1857-7288 119 UGD FEEIT 40 Yes 0 No 24 0 7 Don t know Fig. 3. Basic knowledge of implemented e-learning system into faculties This is a one of the things we want to eliminate with implementation of this system, we want to reach better informing of our students, every one of them to have 24/7 daily access to information about curriculum for all subject, news of every staff related of the studding process. According to the results obtained in the research leads to the question, why students of FEEIT where the number of students that participated in the survey is significantly greater than the students at UGD, but more of 20% of them didn t have basic knowledge of the integrated e-learning system in their faculties? They are many factors way our students don t know some basic things about their faculties, where I will put aside the following: 1. The first important thing is, as much as the number of students participating in the survey larger, the results will be more and more varied. We don t believe that in UGD every student know about implemented e-learning system, these results are accidentally obtained, however we are trying to reach that level. 2. Not enough interest from students or instructors side to use the e-learning system into teaching and learning purposes, 3. Not enough flow of information to students: as much as the group of students who are attending the lecture larger, the more difficult is to keep the attention of students, that brings to insufficient information and communication between the instructor and students. 4. Lack of present on lecture or exercises lead to minimal opportunity that the students will get all the missed information s.
ICT Innovations 2011 Web Proceedings ISSN 1857-7288 73 The next question was enabled only for students who answered positive in the previous one. In this question we wanted to check whether our students know the name of the e- Learning system, 72% answered correct, the others didn t know or their answered was incorrect, anyway this is good result. The next question was open for all students, and it was about if the students have heard or used other e-learning systems and only 32% declared with positive answer. The last question of the first part was about favorite way of learning, the question was open for all students. In this question 61% are for seeing and experimenting, 19% experimenting and felling, 17% listening and watching and only 1% for watching. Most of students want active learning process, where they will have some part of exercises after or in duration of the lecture. Students receive more knowledge in carrying out practical work than having classical lecture and being passive listeners. Second part of the survey is based on 13 questions and most of them were visible only for students who are familiar with implemented e-learning system in their faculties. The first question is: if using this kind of software will improve the quality of learning and teaching, where 37% totally agree, 30% partly agree, 23% with no opinion, 3% were partly disagree, and 7% doesn t agree. This results show us the situation we were expected, not every student want to use this kind of software every day. Then we ask them, if the learning materials are easy to reach, 49% totally agree and 45% partly agree. Students have positive opinion about system interface; they easily found materials, announcement and other activities for their subjects. UGD FEIT 54 59 24 13 0 1 2 3 1 2 Totally agree Partly agree No opinion Partly disagree Disagree Fig. 4. Availability of materials The next question was open for all students and it was about usefulness of e-learning, where 82% answered positive, and 15% with maybe, they doubt of the usefulness of the system, and the rest of them were negative or with no opinion. The majorities were positive about the question and we believe that the number will rise through the years, and
74 ICT Innovations 2011 Web Proceedings ISSN 1857-7288 every student will be comfortable and satisfied with significance of e-learning system. The benefits of using these systems are: time and location flexibility, money and time saving, self-studying, collaborative learning environment, better and more frequent access to instructors and unlimited use of learning materials. For students who answered that they are already using e-learning system, we ask them which activities are inadequate and which are useful. We listed following possible answers: assignments, quizzes, calendar, forum, communicator and other. For inadequate 67% checked other, 10% calendar and 9% quizzes the rest was for assignments, forum and communicator. The 74% of students were for assignment and 14% that forums are more useful activities in e-learning system. At the following picture it s shown the number of used activities into the courses through one academic year in UGD. 8000 7000 6000 5000 4000 3000 2000 1000 0 496 50 53 24 1024 99 1 2 13 46 0 53 279 8 6904 1247 0 Assignment Chat Choice Database Forum Glossary Hot Potatoes Journal Lesson Quiz SCORM Survey Wiki Workshop File Text LAMS Fig. 5. The use of Moodle Activities and Resources till September 2010 The fifteen question was about, if the students have knowledge that the e-learning system is adaptable for working in groups. At UGD 63% of students answered positive and the rest were not sure and little of them answered negative, at FEEIT 55% were not sure, 25% negative and only 15% positive. The answered of the students depend of the conditions set up by the subject instructor. If the instructor have created workshops, or give assignments to the students to work in groups, then the students will knew the answer of the question. The next question is: if the students need manual for using e-learning system, and the answer in both cases were negative, 63% of students think that using this kind of system is
ICT Innovations 2011 Web Proceedings ISSN 1857-7288 75 easy and simple, some of them want to have addition manual (14%) and the rest have no opinion. In following question we ask them if they have problem with finding the materials for learning, 67% answered with no problem, 15% have problem only the first time and the rest needs professional help. The eighteen question was about the interface organization, 61% said that the interface organization is very precise and clear, 35% answered with not quite good organization, and little of them have no opinion, and don t like the interface. Then the following question was about, is it important to knew the content of the course before beginning, and 83% answered positively. Students want to know the content of the subject and duration of the lectures, the materials they have to obtain and conditions for passing the exam. The next question was about the goals for course creation, 64% declared that they want course creation for delivering materials, forums and assignments, 15% for delivering materials and assignments, 11% only for delivering materials. FEIT UGD A B C D E F No opinion A + B + C A + C For Assignments Forums and Disscusions Delivering materials 3 1 25 24 5 12 1 1 2 14 6 96 Fig. 6. Purposes for course creation The next is about things who limited the following and doing the task of the course: at UGD the major number of students answered with other things of the offered answers, and some of them with not having enough time, at FEEIT the most students answered with not having enough time, then not enough clearly instructions and some answered for poor and inadequate materials. The last question of this survey was about purposes of not coming at the lectures: at UGD most of them answered because health problems or other things, at FEEIT most of them answered for not having enough time for learning or working on the material or other thing not listed in the offered answers, or not having enough interest for the subject. Advantage of using this e-learning is for those students who are disabled to present at the lecture, to be able to get the information and be in time with ongoing events from different place and time.
76 ICT Innovations 2011 Web Proceedings ISSN 1857-7288 5. Conclusion FEITle is a system created from the students and teaching staff. And Moodle in other hand is open source system, upgraded several times. So Moodle system is more complex then FEITle, have more activities and resources which can be added into the courses. But both of them accomplish the main things, like putting online materials in the courses, creation of wikis and forums where all the members of the course can be active and exchange ideas and problems between each other. With this research we don t want to replace traditional learning with e-learning. But we are trying to reach balance between them, to implement and use the benefits of e-learning and to better the quality of traditional instructor based learning, where the main goal is to reach better results and making interesting and dynamic environment for learning. Naturally, not every student finds e-learning to his or her liking, or not all instructors want to use e-learning. The use of a technological LMS constitutes an innovation opportunity in the teaching/learning process, and can be seen as an interesting leverage for the creation of the new educational context. As shown by our experience there is a lot of opportunists in terms of integrating and anchoring technologies to support developing HEI their educative environment. References 1. Manuel Moreira da Silva, Paula Peres, Celia Tavares, Luciana Oliveria: B-Learning An Institutional and Operational Development Tool, Proceeding if ICERI Conference, 16 th 18 th November, Spain, 2009, 2. Aad van Moorsel: Metrics for the Internet Age: Quality of Experience and Quality of Business, Fifth Performability Workshop, Germany, September 2001, 3. Kristen Snyder, Luisa Panichi and Ola Lindberg: On the issue of quality of experience in technology supported learning, Designs for learning, Stockholm, December 2010, 4. Gaggioli, A., Bassi, M., & Delle Fave, A.. (2003). Quality of experience in virtual environments. In G.Riva, F. Davide, & W.A Ijsselsteijn (Eds.). Being here: Concepts, effects and measurements of user presence in synthetic environments, Amsterdam: IOS Press, 2003, 5. M.Ristova, V.Trajkovik: E-Learning system into higher education, 8 th International Conference for Informatics Technology, Bitola, 2011.