E-coaching and Feedback Practices to Promote Higher Order Thinking Online

E-coaching and Feedback Practices to Promote Higher Order Thinking Online David S. Stein, Ph.D. Associate Professor, Workforce Development and Education Constance E. Wanstreet, Ph.D. Adjunct Assistant Professor, Workforce Development and Education Paula Slagle Graduate Student, Workforce Development and Education Lynn A. Trinko Graduate Student, Workforce Development and Education Introduction This study examined the effect of coaching in teaching presence and social presence on higher order thinking in online communities of inquiry. Coaching occurred before each chat, and feedback was provided immediately afterwards. The findings suggest that if a group is coached continuously in teaching presence and social presence over time, it can increase the frequency of higher order thinking in chats compared to an un-coached group. Coaching and Feedback Discussion that leads to shared meaning is an expectation in many online courses (Wanstreet & Stein, 2011). Stein et al. (2007) suggested that individual meaning can be transformed to shared understanding during chats through questioning and collective exploration as a group. However, Wanstreet and Stein (in press) cautioned instructors not to assume that learners have the necessary skills to integrate information and resolve issues under discussion. They suggested that learners may need coaching and feedback to move beyond merely stating their opinions toward synthesizing ideas and developing a response that improves upon what is known about a subject. Coaching is a tool that many universities use to help students more efficiently handle course content or set goals for their education (Robinson & Gahagan, 2010). However, coaching students to improve their higher order thinking skills online is less prevalent. Higher order thinking involves synthesizing and integrating information to move the group to resolution of the issues under discussion (Garrison, Anderson, & Archer, 2000). Because the course under study involved inquiry-based discussion, the Community of Inquiry model was chosen to provide the conceptual framework (Garrison et al., 2000). The model assumes that learning through discussion involves the interaction of three overlapping elements: teaching presence, social presence, and cognitive presence (Garrison et al., 2000). Teaching presence involves course design and administration, discourse facilitation, and direct instruction. Social presence is the ability of learners to project their personal characteristics to others, and cognitive presence involves meaning-making through sustained communication. Copyright 2011 The Board of Regents of the University of Wisconsin System 1

Of particular interest to this study is the effectiveness of a teaching presence and social presence coaching and feedback intervention in increasing cognitive presence. The nature of coaching in teaching presence was to encourage groups to name a moderator and summarizer for the following week so that undue time was not spent organizing themselves each week and to summarize their answer and gain agreement from the other members that the response reflects their perspective before moving on to the next part of the discussion question. Summarizing, in effect, synthesizes the perspectives, which indicates integration, evidence of higher order thinking. The social presence coaching involved suggestions for dialogue that helped establish a cohesive group. Fundamentally, coaching is a process that enables cognitive, emotional, and behavioral changes to occur (Grant, 2001) by unlocking a person s potential to perform at a maximal level (Whitmore, 1995). In this study, coaching is defined as a task-based focus that has deliberative and motivational support to enhance learning and performance (Averweg, 2010; Bluckert, 2005; Longnecker, 2010). In the course under study, coaching was conducted electronically. E-coaching has been characterized as a developmental partnership that is enabled through computer-mediated communications, such as email, online chat, or threaded discussion (Averweg, 2010, p. 48). According to Averweg, e-coaching can be more time efficient than coaching conducted face-to-face, achieving goals more quickly and in fewer sessions. Regarding feedback from instructors to online learners, the conventional wisdom is the more feedback the better. Immediate feedback is necessary to keep learners engaged, correct errors, and meet learner expectations that their work is noticed (Tallent-Runnels, Cooper, Lan, Thomas, & Busby, 2005). Feedback is also useful to keep learners on task and to provide guidance as to navigating through an academic chat room (Stein et al., 2007). Stein and Wanstreet (2008) have suggested that in the absence of feedback, learners in the chat room will allocate their time in social, teaching, and cognitive presence in a similar way from chat to chat. Over time, learners do not seem to change their strategy for achieving resolution, nor do learners change the pattern of how they allocate their chat time. Loewen and Erlam (2006) varied the type of feedback in an online class on language acquisition. Feedback was either implicit (response is correct or not) or explicit (response is correct or not and the reasoning behind the correct response). The researchers found no significant difference in the performance of the groups on either oral or written examinations because of the type of feedback received. The researchers noted that feedback was not immediate due to the ways in which chat messages are received. Also noted was the idea that when feedback was provided, learners were not asked to make an immediate correction. Thus a delay in receiving and acting upon feedback might hamper performance. Feedback has been studied in the group development literature as it relates to time and group efficacy. Pescosolido (2003) found that fostering the impression of group efficacy early on led to improved shortterm performance and long-term effectiveness. Baker (2001) studied how group efficacy changed over time as groups received feedback on a meaningful task. As team members worked on problem-solving tasks over time, their assessment of group efficacy increased if they had received regular feedback. The project was designed to answer the following research question: What effect did coaching and feedback in teaching and social presence have on cognitive presence in synchronous discussions? Method and Procedure This study assessed how teaching presence and social presence help groups move to higher levels of learning. The groups were part of a graduate/undergraduate-level course at a large Midwestern university in the history and philosophy of adult education in America. Online learner-led discussions are a feature of the course. Copyright 2011 The Board of Regents of the University of Wisconsin System 2

Learners were randomly assigned to groups. One group was randomly selected for continuous coaching and feedback interventions by the co-investigator. A second randomly selected group served as the control. All groups received feedback on the content of their chats from the instructor of record. The coaching intervention occurred shortly before each chat, and feedback occurred within one hour after each chat. Teaching presence coaching and feedback focused on naming a moderator and summarizer for the following week, summarizing their answer, and gaining agreement that the response reflects the input of all group members. Social presence coaching focused on promoting the use of cohesive language, such as we, our, and us. Feedback assessed how well the group achieved the goals of the coaching. A quantitative content analysis of transcripts from group chats was used to determine frequencies of cognitive presence indicators. Four transcripts from each group were analyzed to track changes over time. Three coders working independently determined the units of meaning (in this study statements and paralanguage) that represented cognitive presence according to the template developed by Anderson, Rourke, Archer, & Garrison, (2001). Reliability testing for cognitive presence was conducted on the transcripts using Krippendorff s (2004) alpha (α =.96..97, and.99) and surpassed the theoretical minimum of 80% (Riffe, Lacy, & Fico, 2005). Summary of Findings A mixed MANOVA was conducted to assess differences between learners who were coached in teaching and social presence and those in the control group in the frequency of cognitive presence at the beginning and end of the term. Results indicated no significant multivariate main effects of group (coached or uncoached; F (3,8) = 1.6, p =.26) or time (beginning or end of term; F(3,8) =.79, p =.54) or group by time interaction (F (3,8) = 3.4, p =.07). However, the follow-up repeated measures ANOVAs for each dependent variable show that the interaction between group and time is significant for cognitive presence (F (1,10) = 6.2, p =.03). There was a change over time for cognitive presence between the coached and control group. The coached group produced a statistically significant higher frequency of cognitive presence than the control group and, with more integrative statements than the control group, had more evidence of higher order thinking at the end of the term. Conclusion These results suggest that time by itself is not going to bring about a change in the frequency of cognitive presence; but if a group is coached in teaching presence and social presence over time, it can increase the frequency of higher-order cognitive presence compared to an un-coached group. The idea of coaching in teaching and social presence reflects the assertion by Garrison et al. (2000) that those presences support cognitive presence. The results also suggest that coaching in discussion processes needs to occur continuously throughout the course. In addition, coaching that occurs shortly before a chat and feedback that is provided immediately afterwards can increase the level of cognitive presence. References Anderson, T., Rourke, L., Archer, W., & Garrison, R. (2001). Assessing teaching presence in computer conferencing transcripts. The Journal of Asynchronous Learning Networks, 5(2). Retrieved from http://www.aln.org/publications,jaln/v5n2/v5n2_anderson Averweg, U. R. (2010). Enabling role of an intranet to augment e-coaching. Industrial and Commercial Training, 42(1), 47-52. Copyright 2011 The Board of Regents of the University of Wisconsin System 3

Baker, D. F. (2001). The development of collective efficacy in small task groups. Small Group Research, 32(4), 451-475. Bluckert, P. (2005). The foundations of a psychological approach to executive coaching. Industrial and Commercial Training, 37(4), 171-178. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. Grant, A. M. (2001). Towards a psychology of coaching. Sydney: University of Sydney Coaching Psychology Unit. Krippendorff, K. (2004). Content analysis: An introduction to its methodology 2 nd ed.). Beverly Hills, CA: Sage. Loewen, S., Erlam, R. (2006). Corrective feedback in the chatroom: An experimental study. Computer Assisted Language Learning, 19(1), 1-14. Longnecker, C. O. (2010). Coaching for better results: Key practices of high performance leaders. Industrial and Commercial Training, 42(1), 32-40. Pescosolido, A. T. (2003). Group efficacy and group effectiveness: the effects of group efficacy over time on group performance and development. Small Group Research, 34(1), 20-42. Riffe, D., Lacy, S., & Fico, F. G. (2005). Analyzing media messages: Using quantitative content analysis in research, 2nd ed. Mahwah, NJ: Lawrence Erlbaum. Robinson, C.. & Gahagan, J. (2010, September-October). Coaching students to academic success and engagement on campus. About Campus, 15(4), 26-29. Stein, D. S., & Wanstreet, C. E. (2008). Chats and shared understanding: How instructors can help learners use academic chat rooms. Distance Learning, 5(1), 49-54. Stein, D. S., Wanstreet, C. E., Glazer, H. R., Engle, C. L., Harris, R. A., Johnston, S. M., Simons, M. R., & Trinko, L. A. (2007). Creating shared understanding through chats in a community of inquiry. The Internet and Higher Education, 10(2), 103-115. Tallent-Runnels, M., Cooper, S., Lan, W., Thomas, J., & Busby, C. (2005). How to teach online: What the research says. Distance Learning, 2(1), 21-27. Wanstreet, C. E., & Stein, D. S. (2011). Gender and collaborative knowledge building in an online community of inquiry. In V. C. X. Wang (Ed.), Encyclopedia of information communication technologies and adult education integration (pp. 707-722). Hershey, PA: IGI Global. Wanstreet, C. E., & Stein, D. S. (In press). Presence over time in synchronous communities of inquiry. The American Journal of Distance Education. Whitmore, J. (1995). Coaching for performance: A practical guide to growing your own skills. London: Nicholas Brealey. About the Presenters David Stein currently serves as an associate professor in the College of Education and Human Ecology at. Dr. Stein specializes in adult teaching and learning. He has conducted workshops on principles of adult teaching and has served as a consultant to professional associations and other universities on adult education. Presently, Dr. Stein is researching online learning and its influence on adult learning. He has presented at national and regional conferences and has written extensively on how adults learn. Address: College of Education and Human Ecology 305 W. 17th Ave. Columbus, OH 43210 Email: stein.1@osu.edu Phone: 614-292-0988 Copyright 2011 The Board of Regents of the University of Wisconsin System 4

Constance E. Wanstreet is an adjunct assistant professor in the College of Education and Human Ecology. Dr. Wanstreet has developed and implemented training programs for adult learners in workplace settings and has served as a consultant to the Ohio Board of Regents. She has presented at national and regional conferences, primarily on how adults learn in online environments. Address: 2200 Olentangy River Road Columbus, OH 43210-1035 Email: wanstreet.2@osu.edu Phone: 614-688-8242 Paula Slagle is a Ph.D. student in Workforce Development and Education at and an adjunct faculty member at Franklin University and The University of Maryland where she teaches human resources and business administration. In her position as vice president of operations learning and development for JPMorgan Chase, she develops and evaluates the effectiveness of training programs within the organization. Address: 5895 Winebrook Dr. Westerville, OH 43081 Email: pylesp@franklin.edu Phone: 614-204-5632 Lynn A. Trinko is a doctoral candidate in Workforce Development and Education at Ohio State. She currently serves as the director of educational and media technology at Lima campus. Lynn specializes in educational technology in distance education and pedagogy. Currently, Lynn's research interests are online interactive learning, videoconferencing, and coaching. She has presented at national and regional conferences. Address: 335 Galvin Hall Lima, OH 45804 Email: trinko.1@osu.edu Phone: 419-995-8412 Copyright 2011 The Board of Regents of the University of Wisconsin System 5