Team-Based Learning (TBL) Assessed for Business Courses at SUNY-Delhi

Transcription

1 1 Team-Based Learning (TBL) Assessed for Business Courses at SUNY-Delhi Akira Odani, Ph.D. (Business Department), Jack Tessier, Ph.D. (Liberal Arts and Sciences Division), State University of New York, College of Technology, Delhi, New York SLIDE #1: COVER SLIDE #2: OUTLINE 15 Minutes Presentation, and 7 min questions OUTLINE: 1. SUNY-Delhi: Profile 2. Business Department and Classes: Traditional 3. Business Classes taught using TBL since Fall 2011: Description 4. Characteristics of TBL taught by the author: RATs, Grade Weight Distribution, Team Members Peer Evaluation 5. Results: Better Attendance and Better Overall Course Grade Scores 6. Ideas for Improvement SLIDE #3: HOMEPAGE 1. SUNY-Delhi Profile SUNY-Delhi is one of 64 campuses across the state. Statewide SUNY system accommodates close to 468,000 students and includes large research-oriented institutions such as SUNY-Albany, Buffalo, Binghamton, SUNY at Stony Brook in Long Island, and the Fashion Institute of Technology in New York City. SUNY-Delhi

2 2 is one of the Colleges of Technology that were originally conceived as vocational schools. SUNY-Delhi was founded in 1913, exactly 100 years ago, as an agricultural college; it is now a thriving school of 3,100 students offering 14 Bachelors and some 50 Associate Degree programs. It prides itself in achieving national recognition in the fields of Veterinary Science and Building Construction. SUNY-Delhi s Culinary Arts and Restaurant & Food Service Management programs have won national championships. SUNY-Delhi is a decidedly education-oriented college with small classes and with a student/faculty ratio of 19.5:1. 2. Business Department Classes I began teaching at SUNY-Delhi in 2006 in the Business Department after over 30 years of managing a Marketing research Company of my own. The Business Department offers accounting, business administration, and marketing classes. I taught Introduction to Business, Small Business Management, Entrepreneurship, Principles of Marketing, and Principles of Advertising. The department has a faculty of 12 full-time and 10 adjunct professors. For the first five years of my career there, I taught in the traditional lecture format. I soon learned that the most demanding challenge of all was to draw and keep students attention in the classes, have them do the reading, and get them engaged in the classroom discussions. The attention span of my students was much shorter than that of my graduate school days, some 40 years ago. I learned that this was not limited to my own classes, but common across most of the business faculty. It has long been a challenge for our instructors to overcome students inertia and passivity. SLIDE # 4: HITTING THE WALL Instead of deepening my academic knowledge on various

3 3 subjects that I taught, I resorted to sharpening my hitherto non-existent skill of telling jokes (a definite deficit for a male born in Japan) and introducing videos, movie clips, and colorful power-point slides into my presentations. Even those props designed for bite-sized attention of the youth have limits in keeping them alert and engaged. Not only was I not successful in making my presentations attractive, I was becoming increasingly cynical, disappointed in my skills, and I experienced declining self-esteem. Only after five years in my new career as a college professor, I was hitting a wall. I was exhausted in finding solutions. SLIDE #5: LIGHT BULB 3. Business Classes taught using TBL since Fall 2011 Then I was lucky enough to run into an on-campus professional development workshop offered by a colleague in the Liberal Arts Division on the subject of team-based learning. All through the summer months of 2011, I devoured the content of a TBL classic, Team-Based Learning: A Transformative Use Of Small Groups In College Teaching, by Michaelsen, Knight and Fink. I was overjoyed that I found a tool to overcome my despair. I was to teach the mandatory four courses, five classes, i.e., 15 hours of teaching per week that Fall. I abandoned the lecture notes and power-point slides of the past, instead produced RATs, study-guides for each chapter, AFA questions, prepared for team formation guidelines, designed a grading system, grade weight setting exercise, written appeal forms, and peer evaluation forms for all four courses. I jumped into the world of TBL with both feet practically overnight, burning the bridge of traditional teaching behind me, convinced that I found a key to becoming a successful educator.

4 4 4. Characteristics of TBL taught by the author RATs: TBL textbook by Michaelsen and others advocate that RATs should be given at the beginning of each major unit of instruction, i.e. possibly several times per semester. I was skeptical about the fewer frequency of RATs applied to the existing population at my college. Bundling two or more chapters-worth of basic concepts into one RAT seemed to be too much to ask considering the reading and retention capability of my students. At the last minute of preparing for TBL in the fall of 2011, I switched my strategy and decided to offer RATs for every chapter of the textbook. That meant as many as 15 to 22 RATs per semester depending on the textbook used for a course. As of today, in the middle of my second year of using TBL, I am still convinced that this was the right decision for the student population I have. Grade Weight Distribution: I began at the beginning to let the students choose the grade weight distribution without any restraints and one class chose 30%/70% ratio for individual/team distribution. The result at the end of the semester was that more than one student was demoted from an individual performance grade of B to F when team scores were adjusted based on the outcome of peer evaluation. These were clearly extreme results for which the instructor had to intervene subjectively. Currently, I am offering 40/60% range as the limits to grade weight distribution. All the classes I am teaching this semester chose 40% for individual and 60% for team distribution, reflecting students thinking that they would be better protected by the expected superior team performances. I am not satisfied with this mechanism as there is a speculation that this ratio results in inflating the final grades.

5 5 Written Appeals: In the first semester of TBL, I distributed the written appeal forms and yet students ignored this procedure and blurted out their questions or protests immediately after they saw the results of the RATs. Asking them to submit written appeals, though quite useful and productive for sharpening their analytical skills, seemed to me to be too cumbersome and required too much precious time. Currently, I am not offering the appeal forms, and I simply ask the class if they have any questions or protests on the results of the RATs. These immediate oral Q&A sessions seem to work well providing students additional opportunities to reflect on key concepts. SLIDE #6 AND #7: PEER EVALUATION FORMS Peer Evaluation: I have experimented with two or three examples of peer evaluation formats suggested in Michaelsen s textbook and others offered in a workshop at last year s training session at the TBLC conference in Atlanta, GA. I now use the 12-criteria format shown on the slide. Before the mid-term exam I have students assess their team members, a trial peer assessment, and then an anonymous summary of comments is given later. I am satisfied with the results as it gives students the opportunity to modify their in-team behavior mid-semester. At the end of the semester, I use the aggregate scores of the result as a multiplier to the total team performance score for the individual members of a team. SLIDE # 8: STATISTICAL ANALYSIS 5. Results I analyzed the students performance data across all the courses taught in two separate academic years. Academic year was taught in the traditional method and the academic year was taught using the TBL method. Courses taught included Introduction

6 6 to Business, Marketing, Advertising, and Small Business Management. The total number of students in pre-tbl classes was 259, while the TBL year had 251. Students learning outcomes were assessed by students course grade scores, the final exam scores, and the missed hours per class. The statistical analyses were conducted using Minitab Version 16 (Minitab, Inc., State College, PA USA) at a= Overall for all classes, in the Pre-TBL classes, students achieved an average course grade score of 1.93 (0.08), while TBL classes earned an average course grade score of 2.40 (0.08) with P < The improvement in GPA is a significant 24%. In terms of number of missed classes per student, the Pre-TBL classes (n = 243) had 6.12 (0.34) absences, while the TBL classes (n = 197) had 3.04 (0.32) absences per student, with P < Students missed classes much less frequently, by about 50%, in TBL classes. Not good with the final exam results: In comparison to the traditional lecture format, students academic performance in terms of overall grades and their attendance in TBL classes were significantly superior using the TBL format. It is encouraging to note that the average course grade score of all TBL courses are better than the Pre-TBL classes. Once we pay attention to the results of the final exam scores, however, the results were mostly reversed. In other words, the TBL courses produced lower final exam scores compared to all the Pre-TBL courses. It is also significant to remember that the final exams were taken only by individuals, not being helped by team efforts. The obvious question becomes: Is TBL ineffective in enabling students to achieve the desired learning outcomes?

7 7 Is the 60% or higher allocation to team performance favorably affecting the final course grade score, covering up the poorer learning outcomes of individuals? Some of my colleagues question the effectiveness of TBL based on this data and suggest that I compare the individual results excluding the impact of the team performance. I would like to collect more data as to why final exam scores were brought down in TBL classes. In terms of direct observation of classes, there is no doubt to me that the students were much more engaged and actively involved in their learning process. There is active questioning and reflecting on more complex questions posed by AFAs among the team members, between teams, and with the instructor. I have a greater amount of time to be engaged in discussion with students on pertinent questions. SLIDE #9: QUESTIONS 6. Ideas for improvement It has been only a year and half since I began adopting TBL in my business classes. The results are decidedly encouraging and I have no desire whatsoever to go back to the traditional lecture format. That does not mean, however, that I am completely satisfied as you can see from the statistical analysis. I see much room for improvement in grade weight distribution mechanism and better learning outcomes. I grapple with the idea of prescribing the individual/team weight distribution so that team scores may become less critical for the final grades. I would like to continue learning from you all, the more experienced colleagues of the TBL community. I welcome any questions.