ADVANCING PEDAGOGY: EVIDENCE FOR THE ROLE OF ONLINE INSTRUCTOR TRAINING IN IMPROVED PEDAGOGICAL PRACTICES

Transcription

1 ADVANCING PEDAGOGY: EVIDENCE FOR THE ROLE OF ONLINE INSTRUCTOR TRAINING IN IMPROVED PEDAGOGICAL PRACTICES Kristin Koepke University of Wisconsin-La Crosse Alexander O Brien University of Wisconsin-La Crosse ABSTRACT An Online Instructor Training (OIT) program at the University of Wisconsin-La Crosse provides a threeweek asynchronous training, focused on design, delivery, and facilitation of online courses. OIT trainees from 2010 and 2011 participated in a three-phase evaluation of the OIT. In Phase I, common conceptions (myths) about online education were assessed both pre and post OIT. In Phase II, trainees provided data about how OIT influenced the implementation of new pedagogical strategies in their online courses, as well as in their face-to-face courses. In Phase III, trainees provided additional in-depth qualitative data about the pedagogical themes that surfaced in their courses as a result of the OIT. Results indicate that OIT significantly changed trainees beliefs about some common conceptions (myths) of online education, and that OIT led to a significant increase in the implementation of 21 of 21 pedagogical strategies in online courses, and 5 of 21 pedagogical strategies in face-to-face courses. KEYWORDS faculty development, pedagogy, online instructor training, evidence-based research, best practices in online teaching I. INTRODUCTION The University of Wisconsin-La Crosse (UW-L) is a primarily brick-and-mortar institution where online education has grown in demand due to faculty and student interests. Faculty interested in teaching online were originally self-taught until in January 2010 when an asynchronous Online Instructor Training (OIT) was offered in a fully online format within the UW-L learning management system, Desire2Learn (D2L). The decision to offer the OIT fully online was based on previous research suggesting that such training should be conducted online, so as to provide instructors with a learning environment similar to the environment experienced by their students [1]. Further, the importance of faculty training programs for online and distance faculty has been indicated by many researchers [1, 2, 3, 4]. After teaching an online course, faculty often assess the appropriateness and success of various teaching strategies both in the online and face-to-face environments, challenging their own beliefs about established methods of teaching [2, 3, 5]. Instructor training and support is critical to online program success and further implies that training should be focused on instructional design principles and application to the new teaching environment [3]. Sun and colleagues found that student satisfaction is most highly associated with the quality of the online course, so such training opportunities should not only help faculty refine online pedagogical strategies, but should also ultimately benefit students [6]. According to Clay, there are four critical stages for distance faculty development: awareness, consideration, implementation, and innovation [7]. The UW-L OIT program helps faculty development Journal of Asynchronous Learning Networks, Volume 16: Issue 2 73

2 Pedagogical Practices throughout the awareness, consideration, and implementation stages. II. BACKGROUND The OIT program at the UW-L was started in January 2010, and provides an opportunity for faculty and staff members interested in online learning to complete a three-week asynchronous training about how to design, deliver, and facilitate online courses. The OIT is designed to familiarize instructors with best practices in online teaching, and employs an active learning approach in which participants engage in an online learning environment [2], while applying principles of course design and delivery to a course they are preparing to teach online [3]. In addition, the OIT allows for interdisciplinary discussions about effective course design. The training places the instructor in the role of an online student, an approach found to be effective in helping online instructors consider the student s perspective while developing online courses [8, 9, 10, 11]. Curriculum is focused on defining and creating a quality online learning experience, effective design of content and activities, implementation of successful assessment and feedback techniques, and defining and understanding the instructor s role as facilitator. The OIT is focused on three course competencies. These competencies are accomplished through a series of learning objectives that actively work each instructor through parallel activities as both a student in the training and an instructor in their own course development process (Fig. 1). The first competency focuses on helping faculty in the training understand the items that can be included in the design, delivery, and review phases of teaching an online course. The main document used to communicate this competency is the UW-L Online Course Evaluation Guidelines [13]. These guidelines were developed internally to serve as a standard set of criteria [14, 15, 16, 17, 18]. The UW-L Online Course Evaluation Guidelines [13] are intended to be used in four primary ways: as a tool for selfassessment by instructors during the design and delivery of their online courses; to provide constructive feedback to instructors on the design and delivery of their online courses; in the UW-L Online Education Grant review process to determine whether grant recipients have fully completed the design of their online courses; and in the OIT to help participants identify the features of effective online course design. Each guideline is supported by research on best practices for online learning. For example, one guideline in the design section states, discussion topics that stimulate thought and reflection related to course content. This guideline and the suggestions for its implementation are supported by the Crest+ model of writing effective online discussion questions [19]. The second competency focuses on exposing instructors in OIT to various methods for creating quality learning units in D2L. This includes proper organization and user interface for easy navigation during online learning. Such organization has been show n to facilitate student learning outcomes [20]. While developing these D2L units in OIT, instructors are encouraged to include measurable learning objectives, written narratives, quality discussion activities, and appropriate assessments. The third competency helps faculty in OIT understand various considerations for online learning. This includes understanding online learning from the perspective of the student, which has been shown to enhance student learning outcomes [1], and is accomplished through participation in discussions and activities, which results in a greater appreciation and understanding for instructor presence and interaction in the online environment. This competency also highlights the support available at UW-L for faculty teaching online including the Center for Advancing Teaching and Learning (CATL), the OIT sponsoring department. 74 Journal of Asynchronous Learning Networks, Volume 16: Issue 2

3 Course Competency 1: Apply principles from the UW-L Online Course Evaluation Guidelines to the design, delivery, and review of an online course Course Competency 2: Create quality learning units in D2L for use in an online course Course Competency 3: Understand various considerations for the online learning environment Learning Objective: Define facilitation as it applies to the instructor in the online learning environment Learning Objective: List and describe at least five effective practices for designing and delivering online courses that will be employed in participants online course Learning Objective: Write at least three instructional objectives for three different levels of Bloom's taxonomy Learning Objective: Apply principles of the UW-L Online Course Evaluation Guidelines to the design of the one constructed module or unit of an online course Learning Objective: Explain the resources and support available to UW-L Online instructors Learning Objective: Demonstrate active participation in an online course by being an active participant in this training, posting at least four substantial posts on four separate days of the module week Learning Objective: Use D2L to construct one module or unit of an online Learning Objective: Use D2L to construct one module or unit of an online course Figure 1. OIT competencies and learning objectives III. METHODOLOGY Improvements in pedagogical practices resulting from OIT were assessed using a three-phase study. A. Sample UW-L employs 531 full- and part-time instructors, and approximately 20% of those instructors have taught an online course at UW-L. Between 2010 and 2011, 67 instructors from myriad disciplines completed the OIT 84% of whom had not previously taught an online course. Of those 67 instructors, 20 completed Phase I, 20 completed Phase II, and eight completed Phase III. B. Phase I: Conceptions of Online Education - Survey To assess how conceptions of online education were influenced by the OIT, participants were asked to rate their agreement, on a five-point scale (1 = strongly disagree, 5 = strongly agree), with 10 common conceptions (myths) about online education derived from the literature (Table 1). Twenty participants completed the survey both prior to and after completing the OIT. Both administrations of the survey were conducted via the university s online survey system, Qualtrics. The pre-test was administered one week before the start of the OIT to all participants. The post-test was administered one week after the completion of the OIT to all participants who successfully completed all assigned tasks for the training. The ten statements included on the survey were based on common conceptions of online education, often referred to as myths in the literature. However, myths was left out of the survey name, questions, and description so as to avoid any inherent biases associated with that word (instead, conceptions was used). The ten conceptions selected for the survey were conceptions commonly, but informally, heard from faculty at UW-L. Data from only those participants that successfully completed the training, the pretest survey, and the posttest survey are included in the results. The survey was intended to measure possible attitudinal change toward online education as a result of completing the OIT. C. Phase II: Pedagogical Implications of Online Instructor Training Survey The Phase II survey was administered to OIT participants who had successfully completed all assigned tasks for the training, and had taught at least one online course after completing the OIT. Twenty participants completed a survey administered via the university s online survey system, Qualtrics. To Journal of Asynchronous Learning Networks, Volume 16: Issue 2 75

4 Pedagogical Practices assess the pedagogical implications of OIT, participants were asked to rate the extent to which they agreed, on a four-point scale (1 = strongly disagree, 4 = strongly agree), with 21 statements about how they had changed their pedagogical techniques, both in online as well as in face-to-face formats, as a result of participating in OIT (Table 2). The 21 statements regarding pedagogical changes were broken into three broad categories: Technology; Course design; and Course delivery. These three areas are three keys curriculum areas in the OIT. Open-ended questions were also included to allow participants to provide examples of technology, course design, and course delivery changes as a result of OIT. The survey was intended to show indicators of instructional design principles learned in the OIT that had been applied to both online and face-to-face teaching. D. Phase III: Pedagogical Implications of Online Instructor Training Interviews The final question of the Pedagogical Implications of Online Instructor Training Survey (Phase II) asked if the participant would be willing to complete a face-to-face interview with the study s principal investigator. Of the 20 participants who completed Phase II, eight agreed to complete a thirty-minute oneon-one interview. During the interview, participants were asked to provide specific examples of the statements with which they agreed during the Phase II survey (Table 2). Each participant was asked the same questions and was allowed to answer freely while their answers were digitally recorded. Example questions include: How have you changed the way you use and think about D2L as a result of completing OIT?, Explain and define any technological improvements you made to your class as a result of completing OIT, Share any ideas you are still looking to implement regarding course delivery improvements, and What was the greatest lesson you got from completing OIT? After all interviews were completed, responses were grouped according to theme. The Phase III interviews were intended to delve deeper into the ways the OIT may or may not have impacted thinking and acting differently with technology integration, course design, and course delivery in both online and face-to-face courses, as a result of the OIT. IV. RESULTS A. Phase I: Conceptions of Online Education Survey Mean responses to the statements on the pretest and posttest of the Phase I survey were compared using paired-samples t-tests (α =.05) to determine whether common conceptions of online education were influenced by OIT. Results are presented in Table 1. Table 1. Results of paired-samples t-tests comparing levels of agreement with common conceptions of online education prior to OIT (pretest means), and after OIT (posttest means). Level of agreement measured on a five-point scale (1 = strongly disagree, 5 = strongly agree). Statement Pretest mean Posttest mean Significance Students get less attention from their instructor in an online course. * Online courses lack academic rigor Journal of Asynchronous Learning Networks, Volume 16: Issue 2

5 Statement Pretest mean Posttest mean Significance Online courses promote one-way learning (teacher --> student), and lack community. It is easier to cheat in an online course than a face-to-face course. * Face-to-face course structure and approach can translate directly to the online environment Online courses lack resources for students. * Students in online courses alter participation due to potential anonymity of the computer. All students will want to take online courses versus faceto-face courses. ** Instructors that teach online need to be extremely technically savvy in order to offer a quality course Online courses save the instructor time ** significant at α =.05 * significant at α =.10 Results indicate marginally significant decreases in levels of agreement with concepts 1, 4, and 6 (Table 1), and a significant increase in levels of agreement with concept 8 (Table 1) as a result of OIT. This suggests that after completing OIT, instructors were less likely to believe that: students get less attention from their instructors in online courses than in face-to-face courses; it is easier to cheat in an online course than in a face-to-face course; and online courses lack resources for students. In addition, instructors were more likely to believe that students will want to take online courses versus face-to-face courses. B. Phase II: Pedagogical Implications of Online Instructor Training Survey Mean responses to the statements in the Phase II survey were analyzed using a single sample t-test (α =.05), which compared the response means for each statement to an expected value of 1 (strongly disagree). These analyses assume that if participants had not implemented a pedagogical change as a direct result of OIT, that they would respond strongly disagree to any statement regarding that particular change. Results are presented in Table 2. Table 2. Level of agreement with statements about pedagogical changes made to online courses and to face-to-face courses, as a result of OIT. Responses measured on a four-point scale (1 = strongly disagree, 4 = strongly agree). Journal of Asynchronous Learning Networks, Volume 16: Issue 2 77

6 Pedagogical Practices Statement Mean responses regarding online courses Sig. Mean responses regarding face-to-face courses Sig. As a result of OIT, I have made the following TECHNOLOGY changes in my course: Tried features in D2L not previously used. 3.33* *.016 Tried additional technologies (non-d2l tools) in my course (for example, Jing, Voice Thread, SoftChalk, Merlot) not previously used. Added audio files to my course (either pre-existing or self-created). Added video files to my course (either pre-existing or self-created) 2.78* * * * *.005 As a result of OIT, I have made the following COURSE DESIGN changes in my course: Written measurable course objectives. 3.33* Mapped out my course to ensure all assessment activities align with objectives. Used feedback from students during the class to revise course content (other than end-of-semester student evaluation feedback). Added more ways for students to access and learn the content. 3.00* * * * Created a more detailed course syllabus. 3.33* *.015 Created D2L discussion boards so that students could ask general course questions of the instructor, socialize with peers, introduce themselves, etc. 2.89* Structured discussions to be more productive. 3.00* Used active learning techniques. 2.89* As a result of OIT, I have made the following COURSE DELIVERY changes in my course: 78 Journal of Asynchronous Learning Networks, Volume 16: Issue 2

7 Increased my participation in class discussion. 2.89* Facilitated discussions more effectively. 2.67* Changed the quality of my interaction with students in discussions. Responded more quickly to student learning needs. 2.67* * Gave multiple types of individual feedback to students (narrative, quantitative, audio, rubric-based, etc.). 2.78* Gave more frequent feedback. 3.00* Gave feedback more promptly. 2.78* Gave all-class feedback. 3.11* I am more available to students. 2.89* t could not be computed because the standard deviation was 0. * significant at α =.05. Of the 21 questions assessing whether participants agreed with statements about having implemented changes as a result of OIT, results indicate that participants agreement ratings were significantly greater than 1 (strongly disagree) for all statements, as they pertained to the participants online courses. These results suggest that when it comes to the participants online courses, OIT resulted in increased comfort with the tools in D2L (technology), a greater awareness of how objectives can help map a course to the corresponding activities to meet the course objectives (course design), and a stronger awareness of the importance of student-instructor interaction and instructor accessibility (course delivery). When indicating their agreement with the same statements applied to the participants face-to-face courses, responses to five of the 21 statements (1, 3, 4, 6, and 9) (Table 2) were significantly greater than 1 (strongly disagree). This suggests that in addition to pedagogical changes in online courses resulting from OIT, participation in OIT can and does lead to pedagogical changes to face-to-face courses, in terms of technology use and course design. C. Phase III: Pedagogical Implications of Online Instructor Training Interviews All interviewed participants commented on three common themes. First, participants indicated some level of change in pedagogical approaches regarding the creation of course objectives, and mapping of those objectives to the various activities and assessments incorporated into their classrooms. For example, four of the eight interviewees indicated that they had always used both learning objectives as well as applied assignments, but that prior to OIT, they had never clearly highlighted the connection for their students. The following quote provides an accurate illustration of this theme: Journal of Asynchronous Learning Networks, Volume 16: Issue 2 79

8 Pedagogical Practices When I prepare for my next class, how my thinking has changed with course design is start with the learning objectives or these are the things I want students to learn. Next, think about what kind of content I would give the students in regards to those objectives. And, then, think about how to appropriately assess or evaluate those. In the past I haven t really thought like that. It has always been here s a class and here are the important things that I think should be covered and I didn t start with the learning objectives first. That is probably the most significant way my thinking has changed; start with what the students should learn and then build everything around that and the alignment will come no matter what the mode of delivery for the class. This change applied to both online and face-to-face courses, and was consistent with evidence from Phase II. Second, participants indicated that if time were not a factor, they would create and integrate more media into their online and face-to-face courses, a sentiment also supported by evidence from Phase II. The following quotes highlight the interviewees desire to incorporate instructional media as they continue to develop both their online and face-to-face courses: Since taking the training [OIT] I now do all of my grading in D2L for all of my classes. I have been starting to integrate Adobe Acrobat Pro and Jing to give audio feedback to students in both my face-to-face and online courses. I have been doing more course-wide feedback in my face-to-face class and that was influenced by the online training. It just happened to be very effective in my online course so I translated that into the online course. In D2L grading I can do a general comment and individual comments and I have learned how useful it is. I now give general responses to the class and if a student didn t get full credit I give individual comments on why points were deducted and I use D2L to communicate all that feedback. Third, participants indicated that they think differently about their teaching approaches and practices as a result of participating in the OIT program, consistent once again with data from Phase II. The following quotes illustrate participants conceptual changes about both online as well as face-to-face courses: I think differently about how to get students actively engaged with content and am continually looking for innovative and fun ways to get the content. Overall, the training [OIT] supported the idea of thinking about teaching in ways you can improve. Some people loose site of that after they have taught for a while. So, going through training like this where you do some reflection, evaluate your teaching, and think about innovative ways to improve it was one of the most beneficial things from this training [OIT]. V. DISCUSSION The results of the three-phase analysis of the OIT at UW-L suggest that the training is effective at dispelling misconceptions about online education, helping trainees to implement new pedagogical strategies into their online and face-to-face courses, and challenging beliefs about established methods of teaching. Consistent with previous research, OIT programs such as UW-L s should result in increased student satisfaction with online courses, and sustained successful online programs [4, 6]. Future 80 Journal of Asynchronous Learning Networks, Volume 16: Issue 2

9 directions include continued assessment of OIT trainees, as well an assessment of student perceptions of online courses offered by instructors who have, and who have not, participated in OIT. A. General Discussion As indicated in the Results section, the Phase I survey illustrated a change in level of agreement with four of the 10 conceptions of online learning, and the Phase II survey found that participants had changed their approaches to online, as well as face-to-face courses. Based on feedback from the participants in OIT, these reported changes came mostly from intentional design of the OIT curriculum. A Phase I finding states that after completing OIT, instructors were less likely to believe that students get less attention from their instructors in online courses than in face-to-face courses. It is likely this is impacted by participants receiving a great deal of attention from the OIT facilitator, much of which is in the form of very personalized feedback. In addition, the curriculum for OIT discusses ways to help facilitate learning through continual feedback and provides suggestions for how to do so. An additional Phase I finding indicates that after completing OIT, instructors were less likely to believe that it is easier to cheat in an online course than in a face-to-face course. We believe that this change is due to three factors: (1) OIT provides various strategies to deter cheating when designing courses and assignments; (2) participants complete a variety of assessments in our learning management system in the role of student; and (3) participants are challenged to think strategically (through course mapping exercises) about varied assessments that could impact academic integrity. The concept of course mapping exercises and aligning course activities with objectives was reiterated as a main theme in Phase III, and supported by results from Phase II, both in participants online courses, as well as their face-to-face courses. Overall, these results support the importance of writing quality course objectives and systematically thinking about how course activities are mapped back to those objectives. In Phase III, many participants reported a realization that the time invested in creating and finding appropriate media (including audio and video files) provided benefits for all students, regardless of the mode of delivery for the course. These perceived benefits, combined with the lessons learned by using D2L in the role of both student and teacher, prompted many participants to begin implementing technology and tools once believed to be solely the domain of online courses into their face-to-face courses as well. Additionally, results from Phase I indicate that after completing OIT, instructors were less likely to believe that online courses lack resources for students. We believe that this change in level of agreement is because OIT demonstrates various ways to present content and exposes even technically savvy instructors to new technologies or rarely used tools within a learning management system. Many of the participants in OIT had used D2L prior to training, but OIT exposed these instructors to different ways to use tools within D2L, and helped them to understand various external technologies that have educational value for online courses. Again, participants in Phase III indicated that learning how to effectively use various technologies in OIT, and through their online course delivery, they were able to more effectively and efficiently use the same or similar technologies for the same benefits in their face-toface courses. Finally, after completing OIT, instructors were more likely to believe that students will want to take online courses versus face-to-face courses. We believe that this change in level of agreement is due to seeing and experiencing effective pedagogical delivery in the online mode, as well as experiencing the convenience of the online delivery mode for both students and instructors. In addition, one of the main benefits of OIT is that it allows each participant to experience the online environment as both a student in OIT, as well as an instructor in a sample D2L course, where they actively build sections of their online course throughout OIT. Due to this structure, assignments in OIT are subtly asking participants to think about their audience in every component of their course design. The curriculum of OIT is focused on three categories (technology, course design, and course delivery) and deliberately shares commentary, resources, discussions, activities, and assessments about each of these categories. Therefore, it was not surprising that instructors indicated that the OIT impacted their practices in online courses, but it was also Journal of Asynchronous Learning Networks, Volume 16: Issue 2 81

10 Pedagogical Practices encouraging to see many of the practices being considered, and occasionally implemented in face-to-face courses as well. B. Future Directions and Conclusion The purpose of the current research was to identify areas in which the OIT at the UW-L can be improved during future offerings, but the results also highlighted improvements that can be made to the evaluation of the OIT. Specifically, the results indicate that the wording of the survey questions in Phase II led to some confusion for participants. Some participants had not taught the same course in online and face-toface formats, and as such did not respond to the questions regarding pedagogical changes to their face-toface courses. Other participants were not sure how to answer questions about pedagogical strategies that they had been using prior to OIT. During future evaluations of OIT participants, the wording of these questions will be modified to provide greater clarity to the respondents. At present, nearly 80% of the instructors who offer online courses at the UW-L have participated in the OIT. Of those instructors, approximately three quarters participated in the OIT prior to teaching online for the first time. Plans for future research include a) comparing the student evaluations of online courses taught by instructors who have completed the OIT to those taught by instructors who have not completed the OIT, and b) comparing student evaluations of online courses taught by the same instructor both before and after that instructor completed the OIT. Since the star t of OIT in January 2010, faculty have taken the training to learn to be an effective online instructor. The professional development opportunity is proving to be a worthwhile venture for faculty that complete the training because lessons learned in OIT can be transferred to other modes of course delivery such as face-to-face. The findings in this study assure the OIT designers that the experience is educational and beneficial to those participants and aids faculty in providing a more quality learning experience for their students in all classes. VI. ABOUT THE AUTHORS Kristin Koepke is an Instructional Designer at the University of Wisconsin-La Crosse in the Center for Advancing Teaching and Learning. In this position, she collaborates with faculty to apply effective online pedagogy and offers development workshops and training seminars for faculty and staff interested in teaching online, integrating technology into the classroom, and using instructional design principles. Koepke has served in various elearning Instructional Design positions in the past including management and consultant roles at various types of organizations including non-profit and for-profit companies. Alexander O Brien, Ph.D. is an assistant professor of psychology at the University of Wisconsin-La Crosse. He teaches Introduction to Psychology, Experimental Psychology/Research Methods, Learning and Memory, and Sensation and Perception. As an instructor of both face-to-face, as well as online courses, he is interested in empirically identifying best practices for delivering content, achieving student learning outcomes, and maximizing student satisfaction with the classroom or online experience. VII. REFERENCES 1. Yang, Y., & Cornelious, L. F. Preparing Instructors for Quality Online Instruction. Online Journal of Distance Learning Administration 8(1) (2005) Gold, S. A Constructivist Approach to Online Training for Online Teachers. Journal of Asynchronous Learning Networks (JALN) 5(1): (2001) Journal of Asynchronous Learning Networks, Volume 16: Issue 2

11 3. Levy, S. Six Factors to Consider when Planning Online Distance Learning Programs in Higher Education. Online Journal of Distance Learning Administration 6(1) (2003) McKenzie, B. K., Mims, N., Bennett, E., & Waugh, M. Needs, concerns and practices of online instructors. Online Journal of Distance Learning Administration 3(3) (2000) Bates, A. W. Managing technological change: Strategies for college and university leaders. Jossey- Bass: San Francisco, CA, Sun, P., Tsai, R. J., Finger, G., Chen, Y., & Yeh, D. What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education 50(4): (2008). 7. Clay, M. Development of Training and Support Programs for Distance Education Instructors. Online Journal of Distance Learning Administration 2(3) (1999) Ladhani, Z., Chhatwal, J., Vyas, R., Iqbal, M., Tan, C., & Diserens, D. Online Role-Playing for Faculty Development. Clinical Teacher 8(1): (2011). 9. Cox, M. Introduction to Faculty Learning Communities. New Directions for Teaching and Learning 97(1): 5-23 (2004). 10. Cowham, T., & Duggleby, J. Pedagogy and Quality Assurance in the development of Online Learning for Online Instructors. Journal of Asynchronous Learning Networks (JALN) 9(4): (2005). 11. Babinski, L. M., Jones, B. D., & DeWert, M. The Roles of Facilitators and Peers in an Online Support Community for Beginning Teachers. Journal of Educational and Psychological Consultation 12(1): (2001). 12. Duncan-Howell, J. Teachers Making Connections: Online Communities as a Source for Professional Learning. British Journal of Educational Technologies 41(2): (2010). 13. Koepke, K. & Udermann, B. UW-L online course evaluation guidelines. Informally published guidelines, Center for Advancing Teaching & Learning, University of Wisconsin-La Crosse University of Illinois: Illinois Online Network. Quality Online Course Initiative January 10, Humbolt State University. Assessing Online Facilitation Instrument: A 2007 TIGERS Project Sponsored by CSU Center for Distributed Learning January 10, University of Wisconsin-Milwaukee. Handbook for Peer Evaluation of Online and Blended Courses. 17. Pennsylvania State University. Quality Assurance elearning Design Standards January 10, California State University, Chico. Rubric for Online Instruction December 15, Akin, L., & Neal, D. CREST+ Model: Writing Effective Online Discussion Questions. MERLOT Journal of Online Learning and Teaching (JOLT) 3(2) (2007) Yu-Chang, H., Huifen, L., Yu-Hui, C., & Dwyer, F. M. The Effects of Web-Based Instruction Navigation Modes on Undergraduates Learning Outcomes. Journal of Educational Technology & Society 12(1): (2009). Journal of Asynchronous Learning Networks, Volume 16: Issue 2 83