An Evaluation of Student Satisfaction With Distance Learning Courses Richard Wagner Professor of Management Jon Werner Professor of Management Robert Schramm Director, On-Line MBA Program Background Information Since their introduction in the 1990s, distance-learning courses have grown dramatically in the number and variety of courses being offered. Today, both graduate and undergraduate courses are taught via distance learning (James, 1997). Many of these courses use the Internet as the primary means of communication between faculty and students, as well as among students, while others rely on a blended mixture of traditional classroom learning and distance learning methodologies. Phillips (1998) stated that on-line universities provide students with the flexibility to take classes at their own pace. A unique feature of much on-line learning is asynchronous collaboration. This allows students and faculty to interact with each other and the course material at a time and a place, which is convenient to them. This is in strong contrast to the synchronous environment that is prevalent in traditional classroom settings. Despite the many potential advantages of Internet-based courses, there has been a lack of systematic research on the relative effectiveness of such courses (Moesel, Dass, Werner, & Fouts, 1998). Kroder, Suess, & Sachs (1998) described issues they faced in launching a web-based graduate course in a collaborative effort between the University of Texas- Irving and Pace University in New York. For example, both students and faculty felt the web-based course required substantially more time than a traditional course in a classroom. Nevertheless, 80 percent of the students said they would take another on-line course in the future. James (1997) described a survey of MBA graduates of the University of Wisconsin-Whitewater that analyzed their reactions to a planned emphasis using on-line course offerings. In the search of the literature to date on distance learning, only limited research was found (University of Idaho, 1995). For example, Sherry, Fulford, and Zhang (1998) devised a measure of satisfaction with instruction for distance learners. Arbaugh (2000) found that perceived usefulness of the learning software as well as the perceived flexibility of taking courses on-line were both related to student satisfaction with their on-line courses. A review of the American Society for Training and Development s (ASTD) Web site (www.astd.org) finds a great deal of information on e-learning, including a Daily News section, which features articles on e-learning appearing in each days press. A sampling of two days (June 4 & 5) yielded an average of 17 articles per day on the subject. Many of these are related to vendor issues, several offer a very observational approach to how to and how not to do e-learning, but none of them offer any hard data on e-learning programs. ASTD also offers an electronic webzine, an Internet magazine, called Learning Circuits which deals exclusively with the topic of electronic learning (www.learningcircuits.org). A search of this web site offers a lot of hands-on tips for e-learning, but very little actual research on how to make it more effective. The On-line MBA at Whitewater 1
The began planning an On-Line MBA program in the fall of 1996. After researching several long-distance delivery systems, LearningSpace from the Lotus Corporation was selected as the primary instructional tool. In LearningSpace, students can access courses using an Internet browser such as Netscape or Windows Explorer or they can access the course using a Lotus client. The Lotus client is loaded directly on the computer hard drive for faster access to the program. In addition, students purchase CD-Roms, which contain instructor developed support material and course information. Research Into Distance Learning at UW-Whitewater The purpose our project is to explore On-Line learning in more depth than that of previous work. Due to the evolving nature of the On-line program itself, our research has also changed over time. Our primary research tool is a survey administered Administering surveys to students who take classes in the On-Line MBA program does the research. In the most recent phase we also give a survey to students in a traditional MBA program. This session traces the development and evaluation of our research. Study 1 The sample population for this study consisted of students enrolled in courses for the On-Line MBA program at the during the fall 1998, spring 1999, and fall 1999 semesters. The survey was given to 206 students in 10 different MBA classes, all of who met electronically. As independent variables, six issues were tested to determine their impact on student perceptions of the on-line course experience. These are: 1. Perceptions that training was sufficient 2. Perceived problems accessing the course 3. Accessing the course via the notes client server vs. an Internet browser 4. Accessing the course from home, school, or work 5. Part-time vs. full time employment status 6. Attending school part-time vs. full-time For the dependent variables, a factor analysis of the 31 remaining questions on the survey revealed five major issues. The results of this factor analysis identified the following five factors: Level of communication with the instructor Level of technical expertise and assistance needed Interactions with other students Effectiveness of delivery method Likelihood you would recommend this course to others 1. Does the amount of technical training received impact student perceptions? Students who felt they received adequate technological training had more positive feelings concerning the five issues studied. Statistically significant differences were found concerning the level of communication with the instructor and the perceived effectiveness of the delivery method, such that those who felt that they had been adequately trained were more positive about both issues. Interestingly, the relationship between training and interaction with other students approached statistical significance, with students who felt more positively about the training were also more likely to be positive about interaction with other students. It should be noted that students in general had overall positive perceptions concerning each category, i.e., the first four dependent variables were measured using a seven-point scale, and all means were above 4.0. The last variable, recommend course to others, was measured using a yes/no item, thus ranging from 1 to 2. This may explain why there was little difference between the groups concerning whether or not the students would recommend the course to others. 2. Do access problems impact students views of on-line courses? Experience has proven that ease of student access is a critical factor for on-line courses. Students (and faculty) quickly get frustrated with new technology that does not work properly or is not user-friendly. Moesel et al. (1998) suggested students who had greater problems accessing the course were less positive about on-line courses. The student s ability to access the course was significantly related to the perceived level of technical expertise needed for the course. Additionally, the relationship Duplication between or access redistribution problems prohibited and without two other written variables permission approaches of the author(s) statistical significance, 2
namely, level of communication with the instructor and recommending the course to others were both more positive when access problems were less of a problem. 3. Does the use of the Notes client versus the Internet to access their course impact student perceptions? Students were able to access courses in two ways, via Lotus Notes off their hard drive or via the Internet. Does the method of access make a difference? None of the dependent variables were significantly related to whether students used the Notes client versus the Internet to access their course. This is surprising, given the delays that often occur when students must use the Internet to access their courses. However, for this sample at least, means of access was not strongly related to any of the outcome measures. 4. Does accessing a course at home, work, or school impact student perceptions? How students access the course was significantly related to their willingness to recommend the course to others, with students who could access the course from work more likely to recommend the course to others. Additionally, students who could access their courses from work were more positive about the level of technical expertise needed, although this relationship only approached statistical significance. One of the many benefits of on-line courses is the time flexibility. A student s ability to access the course prior to work, during a mid-day break, or after work appear to find this flexibility to be a great benefit. 5. Are student perceptions impacted by employment status? In general, student s employed full-time had more positive attitudes towards on-line courses. Full-time employment status significantly impacted whether or not students would recommend the course to others. Further, the relationship between full-time work status and the perceived effectiveness of the delivery method approached significance. This could also relate to the issue of flexibility. Full-time workers taking a graduate program at night value the ability to fit the course of study into their schedule, and thus develop more positive perceptions of a program that allows for this flexibility. Full-time workers also don t have the time to learn how to do it, so sufficient training and assistance is more critical to them. 6. Are student perceptions impacted by full- versus part-time student status? As expected, part-time students generally perceived on-line courses more positively than full-time students do. Part-time students were significantly more positive about the on-line approach as an effective delivery method, and were more likely to recommend the course to others. Although not statistically significant, part-time students held more positive feelings concerning level of communication with the instructor and level of technical expertise required. These findings support current literature that suggests that one of the major benefits of on-line courses is that active people can benefit from the time flexibility of on-line courses. On-line courses allow students to do course assignments when work, family, and other responsibilities are not as pressing (Phillips, 1998). On the other hand, full-time students did hold significantly more positive perceptions concerning interactions with other students. This is likely because full-time students were on campus for other classes and therefore were able to meet in person with each other for group projects and other assignments. Study 2 A second study (still in progress) was begun in fall, 2001. The survey was given to 320 students enrolled in 14 On-Line MBA courses and to 246 students enrolled in 16 traditional (classroom) MBA courses. This study used the following factors: Instructor interaction Course structure/effectiveness Course content Interaction with other students Overall satisfaction with the course While this study is classified as a work in progress it is interesting to note that on all five factors noted above, the online courses were rated higher than the traditional courses. While some of these differences were not great, for three of the five factors the differences were statistically significant (course structure/effectiveness, course content, interaction with other students). 3
Conclusions While these research projects are on going they do offer some tentative conclusions about on-line learning. Our most recent findings that on-line learners have as high or higher satisfaction with the factors of the course are, of course, based on a relatively small sample, but do suggest that on line learning can be a very effective teaching tool. Our findings also support the idea that flexibility is a key issue in student preference for on-line learning. We are also finding that (like most areas) there is a strong learning curve, both for students and for faculty. Initially, concerns with technology dominated a lot of thinking, but through effective training, support and interaction these issues appear to diminish significantly. References Arbaugh, J.B. (2000). Virtual classroom characteristics and student satisfaction with internet-based MBA courses. Journal of Management Education, 24, 32-54. James, M. L. (1997). Delivering the MBA via the Internet: Where do we begin? Academy of Educational Leadership Journal, 1, 41-46. Kroder, S. L., Suess, J., & Sachs, D. (1998). Lessons in launching web-based graduate courses. T.H.E. Journal, May, 66-69. Moesel, D. D., Dass, P., Werner, J. M., & Fouts, P. A. (1998). Team process and effectiveness in interuniversity virtual teams. Paper presented at the Academy of Management meetings, San Diego, August. Phillips, V. (1998). Online universities teach knowledge beyond the books. HR Magazine, 43(8), July, 121-128. 4
Sherry, A. C., Fulford, C. P., & Zhang, S. (1998). Assessing distance learners satisfaction with instruction: A quantitative and a qualitative measure. The American Journal of Distance Education, 12(3), 4-28. Biographical Sketches Dick Wagner is a Professor of Management at the. He has a Ph.D. from Indiana University and has developed and taught MBA on-line classes at Whitewater. In addition he makes extensive use of online modules in his classroom courses. Dr. Wagner has presented papers on distance learning at the Midwest Computer Conference in 2000 and at the Midwest Academy of Management in 2001. He has presented over 50 papers on other topics at a variety of Conferences. Dr. Wagner has also published articles on distance learning in the Distance Education Report (2000), and the NABTE Review (2001). Address: Department of Management E-mail: Wagnerr@uww.edu Phone: 262.472.5478 Jon Werner is a Professor of Management at the. He has a Ph.D. from Michigan State University and has developed and taught MBA on-line classes at Whitewater. Dr. Werner has presented papers on distance learning at the Midwest Computer Conference in 2000 and at the Midwest Academy of Management in 2001. He has presented over 50 papers on other topics at a variety of Conferences. Dr. Werner has also published articles on distance learning in the Distance Education Report (2000), and the NABTE Review (2001). Address: Department of Management E-mail: Wernerj@uww.edu Phone: 262.472.2007 Bob Schramm is the Director of the On-Line MBA Program at the. He has an Ed.D. from Northern Illinois University and has developed and taught MBA on-line classes at Whitewater. Dr. Schramm has presented papers on distance learning at the Midwest Computer Conference in 2000 and at the Midwest Academy of Management in 2001. He has presented over 50 papers on other topics at a variety of Conferences. Dr. Schramm has also published articles on distance learning in the Distance Education Report (2000), and the NABTE Review (2001). Address: Department of Business Education/Computer & Network Administration E-mail: Schrammr@uww.edu Phone: 262.472.4980 5