Using Moodle Academy To Prepare Technical College Faculty for Online Teaching: An Evaluation Model Ronald McBride, Ph.D. Northwestern State University College of Education Natchitoches, LA 71497 mcbride@nsula.edu Amber Thompson, Ed. S. Isothermal Community College (USA) Spindale, North Carolina athompson@isothermal.edu Introduction Abstract: As institutions continue to pursue and value online learning as a legitimate process to deliver courses and degrees to diverse student populations it becomes increasing important to consider professional development for faculty on using course management systems such as Moodle using training programs such as Moodle Academy. This study is an evaluation of Moodle Academy as applied to a community/technical college in North Carolina. Some fifteen years ago interest in distance learning was growing; online learning was all but nonexistent. Things have dramatically changed. This is especially true for community colleges and two year technical institutes. According to the US Department of Education, National Center for Education Statistics (NCES, 2008) community colleges and two-year technical institutions were the place for students to find distance learning opportunities. In the school year 2006-07, two-thirds of the degree offering institutions in the US offered some type of distance learning courses. The number of students enrolled in online classes increased from 1.54 million to 2.14 million in 2090 (TCOHE, 2010). Furthermore, an overwhelming 97% of private and public funded associate degree institutions reported offering distance education programs. This percentage far outweighed the four-year colleges and universities (NCES, 2008). Community and technical colleges are an important link between main street and higher education (Thompson, 2009). Fisher (2009) reported that community college clientele were very diverse in age, education level, skill, and background. Instructors at the community college faced time constraints, overloads in classes, and most importantly technological deficits in the requirements of distance education (Fisher, 2009). Distance instruction required technological knowledge and skill sets that may have been new to the average faculty member (Moore, 2005). New technologies could have induced negative attitudes and frustration if not properly introduced and encouraged (Lewis & Abdul-Hamid, 2006). Distance instruction required technological knowledge and skill sets that may have been new or missing to the average faculty member (Moore, 2005). New technologies and teaching systems could have induced negative attitudes and frustration if not properly introduced and encouraged (Lewis & Abdul-Hamid, 2006). Even with a history of preparing faculty to teach online classes, many community and technical colleges struggle with the dilemma of faculty development for teaching online. Because faculty are typically at different stages with the use of technology in the classroom, using appropriate software for particular teaching applications, and have issues with online management and operating systems it is important to evaluate professional development systems to prepare faculty to teach online in the community and technical college. Moodle Academy is such a professional development tool.
The Study It is the purpose of this study to examine and evaluate the impact of Moodle Academy as tool for professional development at a post-secondary technical college, Isothermal Community College in North Carolina, to prepare faculty to teach online, raise their skills using technology, and provide a common thread for teaching classes online regardless of the subject area. Moodle Academy was designed to train and prepare faculty to use Moodle, an open source course management system. Participants in the study from Isothermal were guided through introductory topics such as connecting through email, uploading and downloading files, and accessing and updating Moodle course documents and settings. In an attempt to determine if faculty at a community college could benefit from providing professional development to raise the level of technological knowledge of faculty and staff, the Isothermal distance learning steering committee found that not all employees were at the same level of technological competence. Initial findings from the steering committee showed that technological needs had to be met first in order to go campus-wide with distance learning. Another reason for conducting this study was to make faculty and related staff aware of the possibilities and positive advantages of using a course management system such as Moodle for online instruction. Without proper knowledge of the learning management system an instructor may feel uncomfortable developing an online presence. A Moodle Academy professional development course was created for all Isothermal Community College faculty and related staff to help guide them through the process of designing a Web-based course. The professional development activity consisted of three tiers: a) online presence in a traditional course, b) hybrid course delivery, and c) online course delivery. The Need for Online Instruction at the Community and Technical College The literature is flush with studies that suggests online programs address issues of great importance to community and technical colleges, that is time and schedule management, enhancing the skill level of faculty poised to teach online, and identifying course management systems that meet the needs of an institution. In addition, the literature suggests that institutions that ignore current trends in online delivery will be left at a distinct disadvantage. Isothermal Community College should be a competitor in the college system instead of losing potential students to institutions that do offer more suitable opportunities. The distance learning percentage of Full-Time Enrollment (FTE) compared to the total college FTE was found to be relatively low in comparison to colleges of similar size in the North Carolina Community College System (NCCCS, 2008). That enrollment could be obtained through more online course offerings without taxing campus resources (Artino, 2008). The NCCCS study showed that in comparison to adjacent community colleges, the average FTE generated per faculty member is lower. Isothermal was found to have a lower FTE count in distance learning than its fellow institutions ranking nearly at the bottom with a mere 6.1%, with 5.2% being the least (NCCCS). The need was evident. Technological Barriers Facing Online Instructors Distance learning instructors should be prepared to handle the technologies that accompany the instructional tasks of teaching online. Failures that happen during course set up and delivery are due to naïveté and a deficient background in regular use of technology (Trentin, 2006). Experienced face-to-face teachers that only dabbled in online instruction perceived the phenomenon to be inadequate in comparison to a fully on-campus seated course (Moore, 2005). However, McLinden, McCall, Hinton, Weston, and Douglas (2006) observed that with proper training to prepare faculty and provisions of necessary resources and support a virtual learning environment could be possible. Moreover, Conceicao (2006) recognized long term experience in online learning environments unfolded new common practices more so than just focusing on the initial design of online courses. The greatest barrier in becoming a distance learning instructor was the communication gap between the knowledgeable but technically illiterate instructor and the student (Hylton, 2006). Bridging the technical divide is one challenge that educators are still dealing with today. Other challenges faced by online educators are the academic integrity of materials students submit and the multigenerational classroom. The Internet can serve as a hindrance in the learning process (Moore, 2005). The cut and paste tool keeps students from synthesizing the knowledge because the concepts are not actually absorbed into long-term memory. Younger students are usually well adapted to technological tools, while adult learners may need more practice (Bishop, Giles, & Bryant 2005). Test security is a major issue because tests can no longer be factual questions based on rote memory (Bangert, 2006). The instructor has to raise the cognitive level of questions to
resemble open ended queries that cannot be Googled. Distance learning required the instructor to develop active learning strategies, cooperation among students and varied learning configurations (Bangert, 2006). For many Isothermal instructors, the fear of practicing new pedagogy and lack of technical knowledge were the prohibiting factors in their willingness to teach online. Effective Online Teaching Practices Many studies had focused on effective teaching practices in online courses. There were many approaches, but the findings seem to report the same conclusion. Effective practices can be classified in four general categories: a) knowledge of subject area, b) course design, c) interactivity among all class participants, and d) class management (Christianson et al., 2002; Hylton, 2006; Lewis & Abdul-Hamid, 2006; Lin et al., 2005; Williams, 2006, Young, 2006). Course management includes every aspect of the course as it is being delivered. Course management software creates a perpetual classroom where class is always in session (Hylton, 2006). Communities of learners can participate both synchronously and asynchronously (Hylton; Trentin 2006). Instructors with experience in online teaching reported increases in planning time and course administration activities (Lin et al., 2005; Conceicao, 2006). Taking a different approach, Young (2006) measured faculty effectiveness through student perception. When properly designed and implemented, online courses and their assessment can foster success for students and teachers. Online instructors believed Web-based teaching and learning was rewarding in different but positive new ways (Conceicao, 2006). Relating to the importance of online teaching proficiency Thompson noted, Isothermal Community College must grasp the distance learning concept (2009, p. 17)). She continued, Recommendations made by the NCCCS report included an increase in support to promote faculty readiness and confidence (p. 17). Isothermal acted; the activity promoted knowledge and awareness among distance learning instructors, thereby, promoting confidence in incorporating online instructional practices. A Model For Online Faculty Professional Development Using Moodle Academy There exists a number of models that outline faculty preparation for teaching online even in the community and technical college settings. But the issue is can an institution devote resources to employ extensive training systems when some are already available such as Moodle Academy? Isothermal Community College had 130 faculty and instructional staff members who were required to teach as part of their daily workload. Only one-fifth of the entire faculty taught courses entirely online, and only one-fourth of all faculty members had some kind of online presence in their courses. This means the majority of these individuals had a reluctance to participate in distance learning and online instruction. The target population was approximately 15 people because of the limited seating capacity of the Moodle Academy workshop. Technical support and instructional professional development need to be provided for faculty to revamp a traditional course into an online setting (Lewis, 2007). Concepts that must be taught to emerging online instructors are cooperation techniques in a virtual classroom, electronic feedback, instructor presence and organization, and sound methodological techniques (Lewis & Abdul-Hamid, 2006). By conducting this study at Isothermal Community College, the faculty and related staff had the opportunity to complete a specially designed Moodle Academy professional development activity that promoted traits of an effective online instructor The Design For Evaluating the Model This study was an evaluation research study. The study attempted to identify the instructors response of professional development and what knowledge they gained about online instruction practices. Kirkpatrick s four level evaluation model was used in this study to determine if the Moodle Academy workshop was successful. Isothermal Community College had 130 faculty and instructional staff with one-fifth of the faculty teaching courses entirely online, and one-fourth having some kind of online presence in their courses. The target population was approximately 15 people because of limited seating capacity of the Moodle Academy workshop. The Moodle Academy workshop is a canned program that directs participant specifically to using management systems like Moodle. The testing instruments for this study were a survey that reflected perceived importance in the feelings of online instructors and a test that covered the technological skills needed to design an online course for Isothermal Community College. The survey was used as an evaluation of reactions and follows Kirkpatrick s first level of
evaluation. The survey was divided into two parts. The first part asked about faculty demographics such as online background and technological skills. The second part of the survey was a faculty self-assessment instrument where faculty members were asked questions related to motivation and effective teaching practices. Part two of the survey had two subsets. The subsets were labeled clearly to show the difference to the participant. The subsets were Distance Learning Motivators and Distance Learning Inhibitors. Each question in the subsets was answered according to a Likert Scale of one to five of the extent to which they agree with factors related to online instruction. One meant they strongly disagreed, and five meant they strongly agreed. The second instrument pertained to the second level of Kirkpatrick s evaluation model. A pretest and posttest was given to the participants before and after the workshop. The pretest and posttest was the same. The test was made of 10 multiple choice questions that represented the technological skills required to set up and conduct a Moodle online course for Isothermal Community College. Both the survey and the test took approximately 15 minutes to complete each time and were presented together for convenience to the participant. A t-test was used to analyze the faculty and staff responses as a whole to determine if the pre-survey and pretest and post-survey and posttest means were significantly different. The analysis of variance method was used to determine if there was a significant difference in the means of responses across surveys and tests. The Results of the Evaluation of Moodle Academy An examination of the distance learning motivators revealed a positive or direct correlation between the attitudes of instructors that were teaching online or could have been teaching online. An analysis using Pearson's correlation coefficient supported this observation with r=.884, p<.005. The participants reported more agreement with distance learning motivators after the workshop than they did before the workshop. On the one to five scale, the overall mean score for all distance learning motivator questions before the workshop was 3.71 and the mean after the workshop was 4.27. Attitude and motivation for distance learning increased by 11%. The difference of means before and after the workshop was tested using a paired two sample t-test. The results were shown to be significant with t(7) = 1.89 and p=.002. Table 1 indicates the question averages for each motivator. Distance learning inhibitors were examined using the paired two sample t-test as well. The mean of inhibiting factor scores varied only slightly. The before the workshop mean was 3.81, and the after the workshop mean was 3.53. Some of the individual inhibitors were scored higher after the Moodle Academy workshop than before. Table 2 indicates these values. As anticipated, the difference was not significant, t(4) = 2.13 and p=.13 Test results indicated the pretest and posttest for this study was the same instrument designed to evaluate the knowledge gained by participants because of their training received from the Moodle Academy workshop. The difference in means from the pretest to the posttest were found to be significant, t(14) = 1.76 and p=.003. The mean score for the pretest was 60%, and the posttest mean was 78%. The variance of the pretest was 371, and the variance of the posttest was 131. The standard deviation of scores was closer together in the posttest than the pretest with values of 19.3 for the pretest and 11.5 for the posttest. These findings indicated more consistency of participant results in the posttest. Implications of the Study Evidence was found that the Moodle Academy workshop increased the participant s familiarity with the Moodle course management system and how courses are to be displayed for use at Isothermal. The results were shown to be significant, and the findings confirmed an 18% increase in knowledge of skills relevant to online teaching practices required for online courses at Isothermal. Furthermore, attitude of participants increased dramatically toward using Moodle. They saw the advantages of the Moodle course management system during the training event. After this brief introduction, participants understood how Moodle could diversify traditional teaching approaches and influence the learning environment with this new type of social interaction. Concerns about the Distance Learning Inhibitors actually increased after the workshop was completed. The area with the most negative impact was apprehension over the quality of work in an online environment. After the workshop, participants had more anxiety about a lack of technical knowledge and skill sets required to use Moodle effectively. Other concerns were apparent with the workload conditions in preparing and teaching online courses. Conclusion Institutions that continue to value reaching diverse student populations, wherever they are, will need to develop a strategy to prepare faculty to teach online regardless of the academic environment and content area. It is
human nature for faculty to be reluctant to embrace online teaching. But facts are the facts, online learning is growing in popularity and is not a passing fancy. Administrators and institutions like community and technical colleges continue to struggle with ways to prepare faculty for teaching online while maintaining a positive attitude about the process. Moodle Academy provides a solution for faculty professional development. The model established at Isothermal and its subsequent evaluation provides impetus to the notion that professional development for faculty through Moodle Academy fosters positive results in faculty attitude about online teaching. The findings also reinforce the original objectives from Isothermal Community College, and those of many community and technical colleges, stating that professional development is not only an important part of preparing faculty to teach online but helps with the dispositions that ultimately affect the quality and substance of online delivery. References Artino, A. R. (2008). Motivational beliefs and perceptions of instructional quality: Predicting satisfaction with online training. Journal of Computer Assisted Learning, 24(3), 260-270. Bangert, A. W. (2006). The development of an instrument for assessing online teaching effectiveness. Journal of Educational Computing Research, 35(3), 227-244. Bishop, D. C., Giles, S. M., & Bryant, K. S. (2005). Teacher receptiveness toward web-based training and support. Teaching and Teacher Education: An International Journal of Research and Studies, 21(1), 3-14. Christianson, L., Tiene, D., & Luft, P. (2002). Examining online instruction in undergraduate nursing education. Distance Education, 23(2), 213-229. Conceicao, S. C. O. (2006). Faculty lived experiences in the online environment. Adult Education Quarterly: A Journal of Research and Theory, 57(1), 26-45. Chronicle of Higher Education (2010). Online Learning: by the numbers. Retrieved December 28, 2010 from http://chronicle.com/article/online-learning-enrollment/125202 Fischer, K. (2009). As the auto industry shrinks, a community college retools. Chronicle of Higher Education, 55(35), A1. Hylton, M. E. (2006). Online versus classroom-based instruction: A comparative study of learning outcomes in a diversity course. The Journal of Baccalaureate Social Work, 11(2), 102-114. Lewis, C. C., & Abdul-Hamid, H. (2006). Implementing effective online teachsupporting practical teaching models: A case study of "school for all". Computers and Education, 44(2), 155-172. McLinden, M., McCall, S., Hinton, D., Weston, A., & Douglas, G. (2006). Developing online problem-based resources for the professional development of teachers of children with visual impairment. Open Learning, 21(3), 237-251.
Moore, B. (2005). Faculty perceptions of the effectiveness of Web-based instruction in social work education: a national study. Journal of Technology in Human Services, 23(1), 53-66. National Center for Education Statistics. (2008). Distance education at degree-granting postsecondary institutions: 2006 07 (NCES 2009-044). Retrieved October 10, 2009, from U.S. Department of Education at http://nces.ed.gov/fastfacts/display.asp?id=80 North Carolina Community College System Office of State Budget and Management. (2008). Isothermal Thompson, Amber (2009). Evaluation of the Moodle Academy based upon Kirkpatrick s Four Level Evaluation Model. Unpublished manuscript, College of Education and Human Development, Northwestern State University, Natchitoches, Louisiana. Trentin, G. (2006). The xanadu project: Training faculty in the use of information and communication technology for university teaching. Journal of Computer Assisted Learning, 22(3), 182-196. Young, S. (2006). Student views of effective online teaching in higher education. American Journal of Distance Education, 20(2), 65-77.