AC 2012-3818: FACULTY PERCEPTIONS AND USE OF A LEARNING MANAGEMENT SYSTEM AT AN URBAN, RESEARCH INSTITUTION

AC 2012-3818: FACULTY PERCEPTIONS AND USE OF A LEARNING MANAGEMENT SYSTEM AT AN URBAN, RESEARCH INSTITUTION Julie M. Little-Wiles M.S.M., Ph.D. (A.B.D.), Purdue University, West Lafayette Julie M. Little-Wiles is a Ph.D. candidate at Purdue University s College of Technology in the Department of Technology Leadership and Innovation. Dr. Stephen Hundley, Indiana University-Purdue University, Indianapolis Stephen Hundley is Associate Dean for Academic Affairs and Undergraduate programs and Associate Professor of organizational leadership and supervision in the Purdue School of Engineering and Technology, IUPUI. Dr. Wanda L. Worley, Indiana University-Purdue University, Indianapolis Mr. Erich J. Bauer, Indiana University-Purdue University, Indianapolis c American Society for Engineering Education, 2012

Faculty Perceptions and Use of a Learning Management System at an Urban, Research Institution Abstract When universities develop and implement learning management systems for their institutions, how do the faculty feel about those systems? Do they believe they are engaging their students by using the learning management system? What elements do they consider crucial in a learning management system to benefit students, themselves, and their departments? This study examined an online learning management system in relationship to faculty in the School of Engineering and Technology at Indiana University-Purdue University Indianapolis (IUPUI). Survey questions focused on two main areas of inquiry, faculty usage and faculty perceptions of the learning management system. Researchers also gave faculty the opportunity to respond to four open-ended questions including what they liked most and least in the system, suggestions for improving their use of the learning management system, and finally, how they could use the learning management system more effectively in their courses. The last section of inquiry concentrated on faculty s perceptions of online courses to help researchers gauge interest, experience, and opinions on the subject as this particular institution does frequently offer such courses in their various programs. This research centered on what faculty use in a learning management system to aid their students in not only gaining knowledge, but also in engaging them in the course and area of focus. Specific elements were identified that (1) faculty commonly use within the learning management system, (2) faculty perceive students respond best to, and (3) faculty perceive students care most about. The results provide both administrators and faculty with general guidelines in developing and maintaining successful online learning environments. Standard tools available in the majority of learning management systems were evaluated. Determination of faculty s training experience and requirements are also discussed. Finally, an understanding of faculty perceptions of the learning management system will be summarized. Introduction Learning technologies continue to grow in popularity within higher education, and this includes the use of learning management systems (LMS). 1, 2 Learning management systems are designed to aid both the faculty and their institutions with the organization and the administration of both information and courses, and can even be used for registration and payment purposes as well. 3 Having one centralized location for all course administration is a convenience for both instructors and students. 4, 5 But the majority of time, neither faculty nor students are consulted when a university chooses a particular LMS to launch at their institution. Since the faculty are then expected to simply embrace the chosen system, how do they really feel about the system and its usefulness in their course instruction? Do they actually utilize it? And do they feel it aids their students in learning?

The purpose of this study was to better understand how the faculty at one urban institution viewed and made use of their mandated learning management system. The results can be beneficial in improving training, support, and usage of LMSs at multiple institutions. Researchers have already conducted a similar study with commuter students at this same urban institution to determine their perception and usage of the system. Those results revealed that although students may find issues with the LMS, such as how it is unpredictable or slow running at times, they would, in fact, embrace it when it is populated with the right tools and resources for their courses. 6 Since researchers now understand how students use the LMS at this specific university, they felt it was then critical to discover how the faculty at this same institution actually used the system and if they accepted the system as well as the students. Method Researchers sought to address three main questions in this study: 1. How do university faculty feel about the learning management systems in use at their institution? 2. Do they believe they are engaging students with their use of the learning management system? 3. What elements do they consider to be crucial or important in a learning management system? Participants in the study were full-time faculty representing nineteen different programs and every level of teaching experience from the School of Engineering and Technology at Indiana University-Purdue University Indianapolis. Following IRB approval, a survey was developed using Zoomerang survey software. Faculty responded to the online survey which contained 39 questions designed to garner information on faculty usage, attitudes and perceptions of Oncourse, the learning management system in use at this institution. Participation was voluntary and no individual identifying information was collected. Limited demographic information was gathered from each participant so that the degree of diversity of the respondents was known. Four of the questions were open-ended, urging respondents to share opinions about what they liked most and least about the learning management system, how their experience using the LMS in instruction could be improved, and finally, in what ways they would like to use the LMS more effectively. The survey was available for faculty participation for approximately one month. Close to a 39% participation rate was recorded from all 131 potential participants (full-time faculty population of the School of Engineering and Technology at IUPUI for the fall 2011 semester.) The survey took approximately five to ten minutes to complete and participants were able to skip over some of the questions if they desired, passing on to the next question to complete the survey. Several of the questions were intended to understand how faculty apply the various features of the LMS to their courses. Fundamentally, we were curious to know if faculty even used the LMS in course instruction or not. Do they post their syllabus? Do they use the gradebook feature to post assignment, midterm, and final grades? Do the instructors use any communication tools within the LMS for regular course communication? Are they uploading resources for their students such as handouts, reading assignments, video clips, and other course materials?

To discern faculty perceptions of the LMS, other questions were targeted toward their understanding of what tools students valued most within the LMS and which tools were most important in the standard LMS template. Faculty were asked to share their opinion of their experience level with the LMS, and if they believed additional training would be of assistance. The final group of questions investigated the level of interest in online course and program development within the learning management system. Results Results are categorized by the two primary areas of interest: faculty use of the LMS and faculty perceptions of the LMS. Concluding remarks focus on the subjects of LMS training and online courses/programs. Faculty Use Questions within this area were designed to understand the extent faculty used the LMS. Appropriately then, the first question asked on the survey was if the faculty were using the LMS (Oncourse) in any capacity in their courses. An overwhelming majority of faculty answered yes. See Table 1. Table 1. Faculty Use of the LMS Yes, always 94% Yes, some 4% No, none 2% Researchers then wanted to know if faculty posted their syllabus in the LMS. All respondents answered yes, all courses for a 100% response. The next question asked faculty if they used the message tool for the primary course communication method. The majority of the faculty answered yes to this question. See Table 2. Table 2. Faculty Use of the Message Tool Yes, always 78% Yes, some of the time 18% No, not at all 4% A follow-up question asked faculty what specific communication tools within the LMS they used for course communication. Faculty were allowed to choose more than one response. Faculty overwhelmingly answered messages and announcements. See Table 3.

Table 3. Communication Tools Used by Faculty Tool Messages 92% Email Archive 34% Announcements 90% Chat 24% Forums 38% Calendar 16% None 0% Other, please specify 14% s given under the Other category included Resources, Gradebook, Web Content, Drop Box, the course homepage within the LMS, and Assignments 2 where assignments are downloaded by students. It was apparent that the faculty used at least one communication tool within the LMS as the category of None did not collect any responses. The distribution of materials for the benefit of the students was the next inquiry. Did faculty upload and distribute course materials (e.g., PowerPoints, lecture notes, videos, readings, etc.) in the LMS? Again, the majority (92%) of the respondents answered Yes, always. See Table 4. Table 4. Faculty Use of the LMS for Resource Distribution Yes, always 92% Yes, some of the time 8% No, not at all 0% The majority of the faculty gave a positive response to the distribution of course materials through the LMS, but what specific tools did they use? The tool reported used most often was resources followed by assignments. See Table 5. Table 5. Tools Used for Course Material Distribution Tool Resources 94% Messages 52% Announcements 36% Calendar 6% Assignments 64% Web Links 38% None 0% Other, please specify 6% Faculty were again given the option to select as many tools that applied. Responses in the Other column included forums, the syllabus and links to Adobe Connect.

The Gradebook tool was next on the list of examination. Researchers first wanted to know if the faculty were using the gradebook to post assignment or project grades. Ninety percent (90%) of the respondents answered Yes, always. See Table 6. Table 6. Gradebook Tool Usage with Assignment/Project Grades Yes, always 90% Yes, some of the time 6% No, not at all 4% Finally, faculty were asked if they used the same Gradebook tool to deliver test grades to students. Seventy-six percent (76%) of the respondents answered Yes, always. See Table 7. Table 7. Gradebook Tool Usage with Test Grades Yes, always 76% Yes, some of the time 12% No, not at all 12% As the results demonstrate, fewer faculty posted test scores in the LMS than they did with assignments or projects students completed. Faculty Perceptions Researchers were then interested in understanding faculty attitudes and perceptions of the LMS as this will often impact the use of the system. The first question posed was how important it was for students to have access to their grades online. Faculty responses were mixed; however, the majority of the respondents answered Important, Very Important, and Extremely Important. See Table 8. Table 8. Perceived Importance of Grades Online Extremely Important 54% Very Important 30% Important 12% Somewhat Important 4% Next, faculty were asked how important they thought it was to populate the LMS with the right materials (assignments, syllabus, PowerPoints, lecture notes, discussions, etc.) in their courses for students. This time, faculty responses were more consistent with 65% answering Extremely Important and another 33% answering Very Important. See Table 9.

Table 9. Perceived Importance of Materials Online Extremely Important 65% Very Important 33% Important 2% Somewhat Important 0% Faculty were then asked a series of questions about their use of various LMS tools and their perceived usefulness of those tools for students. Researchers advised the faculty to define the term usefulness in terms of delivering course content and student benefit. The first tool discussed was the Syllabus with 82% of the faculty responding yes they use the Syllabus tool and 18% no to its use. Responses to its usefulness were more varied, with only 12% answering Extremely Important. See Table 10. Table 10. Usefulness of the Syllabus Extremely Important 12% Very Important 34% Important 37% Somewhat Important 17% Then faculty were asked if they uploaded materials with the Resources tool. The majority (94%) answered yes and only 6% answered no. The usefulness of the Resources tool ranked more uniform. See Table 11. Table 11. Usefulness of the Resource Tool Extremely Important 40% Very Important 47% Important 11% Somewhat Important 2% The third tool in question was the Messages tool. Ninety percent (90%) of the faculty said that yes they used this tool, while only 10% responded with a no. They ranked the usefulness as follows:

Table 12. Usefulness of the Message Tool Extremely Important 42% Very Important 40% Important 9% Somewhat Important 9% The most popular tool proved to be the Gradebook feature with 96% of the faculty replying yes to its usage and a mere 4% no. Surprisingly though, the results for its usefulness were again somewhat diverse with only 40% answering Extremely Important. See Table 13. Table 13. Usefulness of the Gradebook Extremely Important 40% Very Important 46% Important 12% Somewhat Important 2% The final tool faculty were asked about was Assignments. Eighty-six percent (86%) of the faculty reported using the Assignments tool, while 14% did not. However, only 47% of the respondents found the Assignments tool Extremely Important. See Table 14. Table 14. Usefulness of the Assignments Tool Extremely Important 47% Very Important 37% Important 12% Somewhat Important 5% In summary, according to the survey results, all of the tools investigated were used by the majority of the faculty, and even though each tool s usefulness often varied, all ranked higher than Not Important by the faculty. Faculty were then asked a series of four short answer questions to better gauge their embracement and relationship with the system. The first was how they could use the LMS more effectively. Instructor responses included such items as grading assignments and tests more efficiently, more chats, the use of forums for online discussions, online collaborations, real-time communication, use more online quiz tools, post video tutorials, and create more online courses. The major theme that became obvious was that the faculty desired to create more interactive course work for their students within the LMS environment.

The next question was what the faculty liked most about the LMS in use. Overall, the faculty agreed that a major benefit was the convenience of having all classes and material in one centralized location that can be accessed from anywhere at any time. Other remarks involved student benefit by allowing students who missed class to have access to materials and allowing students to see where their grades stand. Remarks involving instructor benefits included the Forum tool for reading responses, having the Messages tool for sending and receiving messages, and not having to print handouts and distribute them in class. Researchers then asked the opposite question, what the faculty liked least about the LMS. Major themes recorded included slow performance and timing out or crashing. Instructor complaints related to these issues included having to save their work often so not to waste time re-doing work. Other issues included the Chat feature which lags too much and the Testing tool which is difficult to use. One final remark requested that the instructor be able to view exactly what the students view while in the system. (Note: the LMS does allow faculty to switch to student view.) The last question inquired how the faculty s experience using the LMS could be improved. Responses conveyed the need for additional training, especially specialized training over certain features. Other requests included the ability to bulk upload individual files to the resources section, better interface of the message system with other email accounts, and the creation of video tutorials online that address how to use the LMS. (Note: A wide array of video tutorials are available to faculty on how to use every part of the LMS.) Training Interestingly, when faculty were asked if they felt they were using the LMS effectively, they were divided evenly in their response. Fifty percent (50%) answered with Yes, absolutely and the other 50% with Yes, but perhaps there is room for improvement. None of them answered No, not at all. Along this line of questioning, researchers then asked if the faculty felt they needed more training on the LMS in use. Responses were diverse with at least a third desiring no additional training. See Table 15. Table 15. Faculty Training Needs Yes, definitely 4% Yes, but only in certain areas 64% No, not at all 32% What specific tools/areas do the faculty want training in? Faculty were directed to choose as many tools or areas that applied. Forty-one percent (41%) answered the Tests & Surveys tool. See Table 16.

Table 16. Faculty Training Needs Gradebook 11% Messages 2% Announcements 2% Resources 7% Forums 18% Tests & Surveys 41% Calendar 18% Assignments 14% Web Links 16% None 34% Faculty were then asked if they felt they were given the right resources and supported to use the LMS by their school. Fifty-four percent (54%) responded with Yes, definitely, 38% with Yes, but only in certain areas, and a small percentage (8%) with No, not at all. Researchers then asked when faculty requested help with the LMS (with such tasks like setting up classes, assignments, assessments, etc.) if they felt they received adequate help. The majority (72%) answered Yes, definitely, 26% answered Yes, but only in certain areas, and a mere 2% with No, not at all. Based on this faculty group s response, general training seems less of an issue, but more specialized and specific training is desired. Online Courses The last group of questions dealt with the faculty s perceptions, use, and interest in online courses. First, the faculty was asked if they had taught an exclusively online course and the results were divided pretty evenly with 46% at Yes and 54% at No. Faculty were then questioned if they would be willing to teach an exclusively online course with a larger portion of them responding Yes at 73% and only 27% of them at No. There was an even larger percentage of the faculty interested in teaching a hybrid course (partially face-to-face and online) with 88% of them willing to consider it and only 12% not willing. But when asked if they were already developing an online course, 78% answered No and only 22% answered Yes. There was another split answer when asked if the faculty would like training in online course development and strategies with 51% at Yes and 49% at No. Researchers then wanted to know exactly how the faculty felt about their school offering online courses and degrees. For online course offerings, there was a large gap with 78% reacting with a Yes and 22% with a No. Online degrees, though, left the faculty divided once again with 47% agreeing and 53% not agreeing. Conclusions and Recommendations The results of this study suggest that the faculty at an urban institution will embrace a learning management system. Several issues must first be addressed, however, in order to provide the

faculty an improved environment for quicker adaption and use. Administration should be concerned with the number of answers indicating how unreliable the LMS is in general, and how the slow response time of the LMS frustrates and hinders the productivity of the faculty. Another issue that should be addressed further is that of training. The majority of faculty indicated that they would like some sort of training on the LMS, and often specifically with various tools or advanced development. The majority does feel supported by the school with resources and additional help, but also feel that they do not have any input into the design or features of the system. Most of the faculty are using the LMS to communicate to students, distribute materials including the course syllabus, and post grades. Researchers were more surprised that the faculty were split in response to the questions of whether or not they were using the LMS effectively in their courses and if they would like more training on course development and strategies, since they had indicated in an earlier question that 68% of them (combined) would like more training on the LMS as previously discussed. The faculty were also almost equally divided in their experience with teaching an online course, and a large percentage was willing to teach both online and hybrid courses. Unfortunately, only a small number of them are currently developing online courses for their school and cited the reason as time. As one instructor remarked, they need more time to adequately develop on-line courses. We are not given extra time to set these up. This research brings to light the issues and concerns faculty have toward the use of their LMS. To build upon this preliminary research, it might be worthwhile to conduct focus groups or even interviews with the faculty to reveal additional understanding into the factors that both hinder and enhance the use of the LMS in course instruction as summarized. Finally, further research may also need to center on administrative and policy issues, including technical support, costs, upgradability, permissions and authorizations, and the maintenance of the technology that could influence the value and use of the LMS in instruction. Due to the interest of the development of online courses, benchmarking with other urban institutions could also prove advantageous to this institution. Bibliography 1. Petherbridge, D., & Chapman, D. (2007). Upgrading or Replacing Your Learning Management System: Implications for Student Support. Online Journal of Distance Learning Administration, X(I). 2. Watson, W.R., & Watson, S.L. (2007). An Argument for clarity: What are Learning Management Systems, what are they not, and what should they become. TechTrends, 51(2). 3. Lonn, S., & Teasley, S. (2009). Saving time or innovating practice: Investigating perceptions and uses of Learning Management Systems. Computers and Education 53(3), 686 694. 4. Little-Wiles, J., & Naimi, L. (2011). A Study of Traditional Undergraduate Student Engagement in Blackboard Learning Management System. Proceedings from 118 th ASEE 2011 Conference. Vancouver, Canada. 5. Little-Wiles, J., & Naimi, L. (2011). An Examination of Faculty Perceptions and Use of Blackboard Learning Management System. Proceedings from 118 th ASEE 2011 Conference. Vancouver, Canada.

6. Little-Wiles, J., Hundley, S., & Bauer, E. (2010). Designing an Online Learning Management System for a Growing Student Population: the Urban, Commuter Student. Proceedings from 117 th ASEE 2011 Conference. Louisville, KY.