E-learning effectiveness

Transcription

1 E-learning effectiveness Halachev, Peter Mihailov Bulgarien Academy of Sciences, Bulgaria Key words: e-learning, higher education, effectiveness evaluation Abstract: Nowadays, higher education is related to the progress of information and communication technologies. Bulgarian universities increasingly use e-learning and need large investments for it. In order to rationally manage the invested funds, it is necessary to evaluate the effectiveness of investments in education. This report includes a bibliographic study of the essence of e-learning effectiveness, the factors influencing it and the methods for its calculation. On the basis of surveys at the University of Chemical Technology and Metallurgy, of studies and statistical dispersion analysis, proofs are found for the existence of a link between the way of training - traditional, blended and electronic and students success. This link is evaluated using a correlation quotient. A survey was conducted concerning students assessment of the advantages of different ways of training. Finally, conclusions and recommendations are offered with reference to extending the application of modern forms of education in high schools in Bulgaria. I. Introduction The social and economic development, as well as the prosperity of any nation, is connected to the development of its human capital. Training highly qualified people requires major investments. One of the priorities of the Universities in Bulgaria is to provide highly efficient, high quality and affordable education. Along with traditional training in universities, other forms are being applied - blended, computer-assisted, computer-based and e-learning. E-learning is learning supported or enhanced through the application of Information and Communications Technology [1]. E-learning is increasingly used because of the economies of time. The speed of training increases - according to studies by 50 percent compared to traditional forms of learning [2]. Also, elearning provides a way to learn quicker at a reduced cost, increased access to training and clear accountability for all participants in the learning process. Organizations implementing e-learning offer their staff a tool to turn the change into an advantage. [3]. Successful development and implementation of e-learning programs depend on the return on investments. Therefore, evaluating the effectiveness of e-learning is of great importance for any university. This article explores some aspects of e-learning effectiveness based on the bibliography, interviews and surveys at UCTM. Conclusions and suggestions are offered, as a result. II.Bibliography The increased use of e-learning in recent years imposes the question of measuring its effectiveness. Many scientists and researchers use Effectiveness and Efficiency in 1(7) ICL 2009 Proceedings - Page 551

2 parallel terms. According to Rumble: Efficiency is the ratio of output to input. [4] and Effectiveness is concerned with outputs. [5]. The effectiveness of e-learning is influenced by many different factors. The works of many authors define these factors in one way or another. A research by UNESCO in 2002 [6], studies the economic factors affecting open and distance learning as: trainees number, educational process scale, traditional form training program duration, costs allocation, students support degree and others. Most studies examine only one or several factors that influence the effectiveness of e- learning. Additional factors - outside of the influence of the instructor and technologies - should be studied the course content and the characteristics of students. Leidner and Jarvenpaa s study [7] examines factors that influence e-learning effectiveness. They are defined as a list of factors related to the e-learning environment: technology, student, teacher, content, pedagogy... etc., having impact on the e-learning effectiveness. Piccoli, Ahmad and Ives [8] conducted a study to develop an efficient e-learning system framework. The key factors, according to them, are students, instructors, technology and content. According to Osika [9] critical factors for the e-learning effectiveness are students motivation, instructors objectives, learner-centered education, etc.. Levy [10] focused on the students' perception of the e-learning system effectiveness. He measured students' satisfaction with an e-learning system and offered some factors affecting such measurement. On the basis of these studies, table 1 summarizes the factors offered by the [8], [9] and [10]. Author Learners Course content Teachers Technologies motivation conceptual teaching style quality Piccoli, maturity knowledge accessibility reliability Ahmad, Ives experience procedural technology of affordability [8] knowledge computer anxiety control factual knowledge motivation the usage of other assessment learner-centered technical online resources clear course education competence of objectives Osika, E., students Camin, D [9] interaction with other students and with teachers teacher-student teaching attitude interaction interaction with Levy, Y. [10] classmates price of the course family support the availability of alternative content simplicity of the learning tasks Table 1: Factors, influencing the learning effectiveness. technical support system errors internet speed 24/7 availability The term effectiveness indicates the extent of implementation of the objectives, as well as the results achieved in e-learning. There are pedagogical, economical and social aspects to it. The pedagogical and social aspects are related to higher accessibility of higher education and to the improvement of the forms and methods of training. Economic efficiency is expressed as: 2(7) ICL 2009 Proceedings - Page 552

3 internal efficiency - rational use of educational institutions resources and meeting the needs of all participants in the learning process; external - growth of material wealth created as a result of increased educational, professional and qualification levels of workers in all sectors, relative and absolute reduction in production and other costs. Evaluating e-learning effectiveness is a comprehensive process, as it concerns the interests of training organizations, teachers and trainees. Such evaluation requires the use of accurate qualitative and quantitative criteria. In economic theory and practice, performance indicators of the ratio type are used for this purpose. The ratio contains in its numerator the criteria that we want to increase - income or quality, and in the denominator - those that want to reduce - the price of training. A widely used indicator of business success in the economy and assessing the effectiveness of training is return on investment (ROI). According to J. and P. Phillips the return on investment is based on the net benefits divided by project costs [11] Net project benefits ROI(%) = 100 Project costs Well developed training creates a very high ROI-ten to one hundred times the investment. Training is one of the most far-reaching and highly leveraged expeditors. [12] William Horton [13] suggests the following basic formula for calculating ROI in e- learning: Benefits - Costs ROI = 100 Project costs E-learning effectiveness could be studied in absolute terms - the difference between the number of accepted and graduated students. In this report, the relationship between these variables is studied in relative terms, i.e. using a performance coefficient /Kel/ representing the ratio between the number of graduate and enrolled students. Number of graduated students Kel = Number of enrolled students III. Research on training effectiveness A study on the number of accepted students, the number of those enrolled in the last year and the number of graduated students was conducted at UCTM. The purpose of the study is to determine the impact of e-learning on successful course completion. Three courses were tested. We used the success rate as an indicator. The maximum value of this coefficient is 1. The survey results and the coefficient values are presented in Table 1. Subject 2006/ / /2009 Industrial management 0,96 1,00 1,00 Manufacturing automation 0,65 0,60 0,69 Ecology 0,88 0,93 1,00 Table 1: Values of the success rate quotient. 3(7) ICL 2009 Proceedings - Page 553

4 For , there is a trend, valid for all three courses: increase in the number of graduated students and decrease in the number of students dropped out of training. This is illustrated in Chart Industrial management Manufacturing automation Ecology 0 06\07 07\08 08\09 Chart 1: students graduation trend. There are different reasons for students dropping out of the course of study. Some students stop their studies for economic reasons associated with the need to work while studying, i.e. the inability to attend lectures. There are also psychological problems associated with the adaptation of the first grade students to the new environment - more than 50 percent of students come from other cities and towns. In 2008, an Electronic Materials Unit was created at the University in order to achieve extended use of e-learning. Also, an Electronic academic literature fund that can be used upon registration was established. Links were created to materials in the pages of the respective departments, as well to materials on other sites authorized for general access. The number of electronic learning materials available on the web-site of the University has increased. The pages of the individual departments and in the virtual library contain 25 training units, and 17 subject profiles are actively used in training activities. The number of registered users is 3500, the home page visits are more than 10,000 on some subjects. In addition to the development of electronic courses a study on the extent of their usage was held at UCTM. The study explores the influence of the use of electronic forms on the learning process, i.e, students score after training. 15 students were divided into three groups according to the degree of attendance at traditional F2F /Face to Face/ lectures. Group A includes students with 85 percent attendance of the traditional lectures, Group B includes students 50 percent attendance /Blended Learning/, and Group C - with less than 15%. Table 2 represents the Informatics test results of different student groups. The maximum score was 80. Student group Test scoring Number of students Average group in the group score X Y N Y 1 Group A 65,60,65,80, Group B 80,75,80,70, Group C 63,60,70,47, Table 2: Survey results 4(7) ICL 2009 Proceedings - Page 554

5 Many different factors affect the success of students. A lot of them are subjective: educational level in secondary education, personal characteristics and motivation of students, professionalism and experience of the teacher. There are important non-subjective factors of successful training. They are: complexity of the studied subjects, the used learning method, the used technology. The purpose of this study is to define the existence of a link between the way of training /traditional, blended, electronic/ and the test results of students. For the purposes of the study, the method of training /x/ is assumed to be a factorindependent variable that cannot be measured quantitatively a "weak" scale. The test results of the students /y/ can be quantity measured, which is numerically presented as a "strong scale. The appropriate method for determining the statistical relationships between variables of this type is the dispersion analysis. It is used to analyze and test the hypotheses concerning the relationship between the training method and students success. Our zero hypothesis /H0/ and our alternative hypothesis /H1/ are: H0: there is no significant relationship between the training method and students success at the test. H1: the training method affects students success. The level of significance is assumed to be α = 0,05. After the calculations for the F-criterion, we receive 5.98for empirical feature of the hypothesis /Fem/. The determine theoretical characteristics of the hypothesis /Ft/ using the F-distribution table, with importance level α = 0,05 and freedom degrees 2 and 12. The theoretical feature of the hypothesis is The comparison between empirical and theoretical characteristics of the hypothesis shows that the empirical characteristic is more significant than the theoretical one. For this reason, we can accept that the alternative hypothesis is true and we can state with significance level of 0.05 that the factor training method has a significant statistical impact on the quality of training. The correlation coefficient is 50%, which indicates the strength of the studied relationship. In other words, 50% of the differences in students success are driven by differences in the way of training. A survey was conducted concerning the views of students on the advantages of e- learning compared to the blended and traditional learning. The indicators according to which the students had to evaluate the three types of training are presented in Table 3. Indicator Indicator value (in %) Group A Group B Group C 1 Knowledge acquisition Training time savings Financial costs savings Learning material interactivity Accessibility of the teacher Communication between students Convenience of training time for the student Table 3: Comparison between traditional, blended and e-learning. 5(7) ICL 2009 Proceedings - Page 555

6 The data from the survey indicates that students prefer traditional training because of its higher quality, but they appreciate the benefits of e-learning because of the time and financial costs savings. The data from the survey is represented in Chart Group A Group B Group C Chart 2: Comparison between traditional, blended and e-learning. Electronically trained students appreciate the interactivity, economy of time and funds for this type of training, but they feel isolated from their colleagues and tutor. In the process of improving higher education effectiveness, universities could include in their training the positive characteristics of traditional education that are missing in e- learning. Different strategies are offered in the existing literature. J.A. Gasker and T. Cascio [14] examine the role of in strengthening the relations teacher-student and student-student. Hodgkinson [15] proposes electronic learning should include initial F2F introductory classes in order to familiarize the students with the study topics, and to get to know the teachers and each other. G. Abrams and J. Hefner [16] reported an experiment conducted in Colorado University in USA consisting in training through broadcasting a traditional lecture that can be attended physically or through the Internet. IV. Conclusion On the basis of the surveys and studies, the following conclusions can be made: The application of electronic and blended learning improves the quality of higher education and, as a result, reduces the number of student dropouts. The training method - traditional, blended or e-learning - is a factor that influences the success of trainees and the strength of this relationship is significant. On the basis of the specific study, it can be said with 5% error risk that 50% of the differences in students' success in training are determined by the training method. Students preference is determined by factors as high quality, low cost and time for learning, interactivity, and opportunities for interaction with the teacher and colleges. Electronic and blended learning are still at an early stage of development in UCTM, but even this early stage shows that the expansion of contemporary learning forms contributes to the improvement of the quality of higher education. 6(7) ICL 2009 Proceedings - Page 556

7 Further studies are needed to compare the effectiveness of traditional, blended and e- learning. High quality, low financial cost, and training flexibility have been the key factors in the choice of higher education programs. In the development and implementation of e- learning at UCTM, it is necessary to find effective strategies that combine the positive characteristics and minimize the negative aspects of traditional and electronic learning. V. Literature 1.Learning and Skills Development Agency, Keegan, D. (1995). Distance education technology for the new millennium: compressed video teaching. ZIFF Papiere. Hagen, Germany: Institute for Research into Distance Education. (Eric Document Reproduction Service No. ED ). 3.Imel, S. (1998). Myths and realities of distance learning. Columbus, Ohio: ERIC Clearinghouse on Adult, Career, and Vocational Education. Ohio State University. (Eric Document Reproduction Service No. ED ) 4.The Costs and Economics of Open & Distance Learning, Greville Rumble, 1997, p The Costs and Economics of Open & Distance Learning, Greville Rumble, 1997, p Open and distance learning. Trends, policy and strategy considerations. Division of Higher Education, UNESCO 2002, 95 p., pp Leidner and Jarvenpaa (1995) 8.Piccoli G., Ahmad R. and Ives B.: Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25(4) Osika, E., and Camin, D., (2005). Concentric Model for Evaluating Internet-Based Distance Learning Programs. 18th Annual Conference on Distance Teaching and Learning 10.Levy, Y. (2006). Assessing the Value of E-Learning Systems. Information Science Publishing. 11.How to Determine ROI in People, Projects, and Programs, J. and P. Phillips, 2007, p The Training Measurement Book, Josh Bersin, p Evaluating e-learning, William Horton, 2001, p Empowering Women through Computer-mediated, Class Participation // Affilia, 16 (3), 2001, pp , 15.M. "Student Perceptions of Face-to-Face Induction for On-line Programmes" / / Quality Assurance in Education, 10 (4), 2002, pp (September / October 2002). Blending Online and Traditional Instruction in the Mathematics Classroom ". The Technology Source. Retrieved on 20 March 2007, from thematics_classroom/. 7(7) ICL 2009 Proceedings - Page 557