Consider any discipline- or department-specific issues that affect CIS scores

Transcription

1 Reviewing teaching Faculty Course Instructor Surveys Course Instructor Surveys (CIS) are a standardized mechanism for feedback to faculty from students and can assist in the continuous improvement of a faculty member s teaching. Research indicates that care must be taken in order to glean accurate interpretations of CIS scores. In an attempt to get more meaningful information, academic units might consider implementing one or more of the following when using CIS scores for evaluation of teaching and for promotion purposes. Aggregate similar scores for comparison Take into account the acceptable temporary dip in scores that sometimes occurs as a result of innovative teaching techniques, or when teaching a course for the first time Consider any discipline- or department-specific issues that affect CIS scores Make a meaningful attempt to understand and take into consideration the well-documented biases inherent in student ratings Some examples of these strategies can be found in Appendix I. CIS, at the end of this section. Faculty Peer Observation Peer observation is a mechanism for constructive feedback and continuous improvement. Peer review and discussion can be used for increasing departmental communication about teaching, for evaluation purposes (review and promotion) and for improving teaching. The observer must be provided with the instructor s syllabus, exam samples, and other significant teaching materials used (for example, the course web site). Prior to the peer observation process, the departmental leadership should discuss examples of and criteria for excellent teaching as well as the warning signs of teaching that may need improvement. Because teaching styles vary, observers should be open to consideration that an instructor s style, however different, may be effective. The observer should give constructive comments and feedback to the lecturer and may provide evaluative comments to the department chair as requested. Tools for Effective Observation Evaluations should include the use of short forms that merit careful attention by the reviewer. Questions on the forms should call for a narrative response or a choice among three or four responses. See APPENDIX II.

2 Each peer evaluation/observation report should include: Number and title of course observed Date of report Name and signature of observer Date of pre-observation meeting between observer and instructor, at which the syllabus and assignments are reviewed, special instructor concerns are addressed, and a mutually agreed class and date are specified Date of classroom observation(s) An instrument that reflects methods by which instructor engages students in active learning Date of post-observation meeting of observer with instructor, at which the observation was discussed; and Instructor s signature affirming that the discussions took place. Preparation and Training for Effective Peer Observation Before peer evaluations are conducted, peer evaluators should be given detailed guidance and an opportunity for training. As a minimum, the departmental criteria for effective teaching should be discussed. Observers should be requested to recognize instructors have different teaching methods and to consider the effectiveness of teaching styles that might differ from their own. Evaluation templates should be provided to guide the evaluator s observations of teaching.

3 Reviewing teaching APPENDIX I. CIS Examples of methods to get more accurate information from Course Instructor Surveys (CIS). An example of how departments might aggregate similar Course Instructor Survey (CIS) scores for comparison to a faculty member s CIS scores. All of this can easily be done on a spreadsheet containing an academic unit s CIS results for all instructors. o Group courses that have historically similar CIS scores as compared to other classes. These collections might be created by using some combination of class size, course level, major versus non-major course, labs versus field experiences versus classroom courses, or by other factors as determined by the department or academic unit. The question(s) on the CIS that will be used to create these groups would also need to be determined by the department or academic unit. o For each group, compute the mean and standard deviation of the appropriate score on the CIS. o For each instructor, look at this CIS score and determine the instructor s z-score, which is the ratio of (the instructor score minus the average score) divided by the standard deviation. This gives the number of standard deviations this instructor s score is from the mean within that group of classes. o These groups should be reevaluated each year for validity. This is an imperfect process. Dips in CIS scores may occur as a result of innovative instruction. The department should make it clear that reasonable innovation is encouraged, and that resulting CIS dips will not harm the innovator s evaluation or chances for promotion. CIS scores may be lower than desired during the initial one or two semesters of teaching but may improve significantly in subsequent semesters. Trends of this type should be noted, and initial scores should not be weighted as heavily when assessing the faculty member for promotion. To avoid biases in student ratings, departments may choose additional data to be used when evaluating faculty teaching, and faculty members could be encouraged to gather such data o Peer evaluations o Departmentally generated or faculty member generated evaluation forms, perhaps used mid-semester, with documentation on any adjustments that were made in response to the results o Solicitation and collection of informal feedback

4 APPENDIX II. Examples of Peer Observation Procedures 1. Chemistry Teaching Evaluation Form 2. Biology Instructional Office Teaching Observation Form 3. Center for Teaching & Learning CTL Peer Teaching Evaluation Proposal

5 Reviewing teaching 1. Chemistry Teaching Evaluation Form Faculty member observed: Class observed: Faculty member conducting evaluation: Peer Observation of Teaching Context or Background Information: Describe the setting in which the lesson took place, relevant information about the makeup of the class, and any other descriptive characteristics that would provide appropriate context to the observation. Observation Area 1: Instructor Goals/Intentions for Class Session Focus your comments on whether the goals were: 1) clearly stated or portrayed in an obvious fashion, 2) appropriate to the focus of the course, 3) explicitly connected to the flow of previous or future classes. Observation Area 2: Student engagement with the subject matter Examine the degree to which student engagement occurred 1) over a substantial portion of the class meeting time, 2) by a broad segment of students attending the class, 3) in appropriate forms such as discussion, listening/processing, performing, reading, reflecting, speaking, or writing. Observation Area 3: Examination of student achievement of goals Focus your comments on how the instructor developed an understanding of student achievement of goals by methods such as 1) questioning students on course material, 2) observing student performance(s), 3) studentstudent discussion, 4) informal assessment techniques, 5) quizzes, or 6) other methods. Observation Area 4: Was there anything else worth comment, not noted above? Areas to be considered for improvement? Something outstanding or of special interest that you observed during the class?

6 BIO Observation Form Instructor: Observer: Date: Class: Classroom Observation Report Needs to Improve Does Well Outstanding Content organization 1. Presents a brief overview of the lecture o o o NA 2. Relates content to previous knowledge o o o NA 3. Summarizes major points at the end of class o o o NA 4. Sequences topics logically o o o NA 5. Uses visuals during class o o o NA 6. Appropriate amount of material used in lecture o o o NA Presentation 1. Explains important ideas clearly and simply o o o NA 2. Periodically checks student understanding o o o NA 3. Receptive to student questions and comments o o o NA 4. Promotes student participation o o o NA 5. Promotes critical thinking o o o NA 6. Demonstrates command of the material o o o NA 7. Appropriate pacing of the class o o o NA Use of Media 1. Uses chalkboard/overheads/powerpoint effectively o o o NA 2. Audio/visual materials add to students comprehension o o o NA Individual Style 1. Projects voice so easily heard o o o NA 2. Varies voice to maintain interest o o o NA 3. Articulates clearly o o o NA 4. Effective body gestures o o o NA 5. Eye contact with students o o o NA 6. Enthusiastic and confident o o o NA Instructional materials used in the course: Learning objectives Lecture slides / notes Illustrations Videos/Animations/internet Readings Other (classroom response system Comments The following page is the most important part of the evaluation. In your evaluation, please note that different Instructors use different teaching methods (lecture only, flipped classrooms, a mixture of the two). Please feel free to comment on the teaching method and to evaluate the Instructor s effectiveness with their method of choice.

7 Reviewing teaching Bio observation form cont. Comments Strengths in Classroom: Weaknesses in Classroom: Comment on Syllabus: Is the course organization clear? Is the schedule of events on each class day clear? Are the exam dates and formats clear? Is the grading scheme clear and fair? Will students know their grades as the course progresses? Overall, are the expectations of the students clear? Suggestions for Improvement of Syllabus: Comment on Exam(s): Does the exam test understanding or memorization? Are the questions unambiguous? Suggestions for improvement: OVERALL EFFECTIVNESS RATING: Low High DATE: SIGNATURE OF REVIEWER:

8 Peer Teaching Evaluation Proposal Center for Teaching and Learning Karron Lewis, Ph.D. Background and Rationale

9 Reviewing teaching The Center for Teaching and Learning (CTL) has engaged in a nation-wide scan of practices at peer institutions related to the most common elements of faculty peer-evaluation systems. This proposal is informed by the results of this benchmarking activity. Faculty Peer-Observation of Teaching We want to design an observation process that engages both the observing teacher and the teacher being observed in a collegial conversation about effective teaching practices. To accomplish this, we designed an approach that emphasizes constructive dialogue over a reductionist, checklist approach. Observations should include two of the most fundamental aspects of effective teaching: level student engagement in class and active assessment by the instructor of students understanding. Faculty peer observations should include the following elements: Be a critically reflective and collaborative process in which the instructor under review works closely with a colleague or group of colleagues to discuss his or her teaching. Provide training to those who will be a peer reviewer. Include a process for mentoring / coaching. Include self-assessment. Provide a review of educational materials, e.g. syllabi and other materials in various media (including course web sites) prepared for instructional use. Provide a review of instructional delivery that primarily includes open-ended feedback. Describe the elements to be included in the peer review each year (syllabus, student work, classroom teaching) and the frequency with which these elements will be reviewed. Pre-Feedback Resources from CTL To support a system of evaluating teaching effectiveness the Center for Teaching and learning will: 1. Offer consulting for peers on how to give peer feedback 2. Work with departments to foster conversations about teaching and peer evaluations Elements of the Peer Evaluation of Teaching Evaluation System Proposed Process for Peer Teaching Evaluation System 1. Department selects peer to work with an instructor and sets the frequency for peer reviews 2. The instructor and peer have a pre-observation conversation a. Should occur in the first 5 weeks of the semester b. Syllabus, sample activities, and sample assessments are given to the peer 3. Peer conducts a review of course materials (see Peer Review of Syllabus form and Peer Review of Class Assignments and Assessments form) a. Should occur between the 5 10th week of the semester 4. Peer conducts classroom observation (see Peer Observation of Teaching form) a. Should occur between the 5 10th week of the semester 5. The instructor and peer have a post-observation conversation (see Peer Observation of Teaching Conversation Synopsis form) a. Should occur 1-2 weeks after the classroom observation b. Peer shares results of course material review and classroom observation

10 c. Peer summarizes results of conversation and both peer and instructor sign the form

11 Reviewing teaching Preparation for Observation In order for the peer reviewer to situate a classroom observation within the context of the total course and the instructor s development, a conference should be scheduled. Sometimes, this may be an extended discussion, while at other times, a note or telephone conversation may have to suffice. The following form provides examples of the kinds of information that might be sought from the instructor before a classroom observation takes place. Pre-Observation Conference Form Prior to the scheduled observation, the peer reviewer might use the following form (or an adaptation of the form) to structure the discussion of the teaching context with the instructor to be reviewed. Information can focus on class goals, students, learning activities, and particular teaching style. The peer reviewer should request that the instructor bring a copy of the syllabus, text, and any pertinent material to help the reviewer understand the content and cognitive level of the course. Instructor: Date: Time: Course Number: Course Title: Course Meeting Time: Level of Students: 1. What are the goals for the class that I will observe? 2. What are your plans for achieving these goals? 3. What teaching/learning activities will take place? 4. What have students been asked to do in preparation for this class? 5. Will this class be typical of your teaching style? If not, why? 6. (For formative review) What would you like me to focus on during the observation? 7. Are there other things that I should be aware of prior to the observation?

12 Peer Evaluation Form Peer Review of the Syllabus 4=Strongly Agree 3=Agree 2=Disagree 1=Strongly Disagree NA=Not Applicable Syllabus Areas Course Information Best Practices NA The objectives are appropriate to the course. Class materials are appropriate to the course. The syllabus provides clear roadmaps for the course. Course Policies are clearly stated (e.g., criteria for grading, makeup exams). Required university statements are present (e.g., academic misconduct). Criteria for grading are clearly delineated. Instructor Information The percentage of the grade for course assignments and exams is clearly stated. Comments:

13 Reviewing teaching Peer Review of Class Assignments and Assessments 4=Strongly Agree 3=Agree 2=Disagree 1=Strongly Disagree NA=Not Applicable Areas Best Practices NA Class Assignments Assessments The assignments probe different student skill sets. Assignments are clearly aligned with course objectives. Assignments are spread appropriately across the semester. The difficulty of assignments across the semester is appropriate for the course level. The workload required by the assignments is appropriate to the credit load for the course. The instructions for these assignments are clear. The assessments probe different student skill sets. Assessments are clearly aligned with course objectives. The difficulty of assessments is appropriate for the course. The instructions for these assessments are clear. Criteria for each assessment are clearly delineated. Comments:

14 Peer Observation of Teaching Context or Background Information: Describe the setting in which the lesson took place, relevant information about the makeup of the class, and any other descriptive characteristics that would provide appropriate context to the observation. Description: Observation Area 1: Instructor Goals/Intentions for Class Session Focus your comments on whether the goals were: 1) clearly stated or portrayed in an obvious fashion, 2) appropriate to the focus of the course, 3) explicitly connected to the flow of previous or future classes. Comments: Observation Area 2: Student engagement with the subject matter Examine the degree to which student engagement occurred 1) over a substantial portion of the class meeting time, 2) by a broad segment of students attending the class, 3) in appropriate forms such as discussion, listening/processing, performing, reading, reflecting, speaking, or writing. Comments: Observation Area 3: Examination of student achievement of goals Focus your comments on how the instructor developed an understanding of student achievement of goals by methods such as 1) questioning students on course material, 2) observing student performance(s), 3) student-student discussion, 4) informal assessment techniques, 5) quizzes, or 6) other methods. Comments:

15 Reviewing teaching Post-Observation Questions This conversation with the peer faculty member would occur after the class and prior to providing the written feedback. The following are examples of reflective questions that are often used in post-classroom observation discussions: 1. Tell me how you think today's class went. 2. Did the lesson proceed in the way you had planned? Why? 3. Did the students react to the lesson in the way you thought they would? 4. During the lesson, did you feel confident and enthusiastic? Why? 5. Do you think the students learned all that you wanted them to learn in this session? What brings you to that conclusion? 6. What did you do to encourage the students to actively participate in the lesson? 7. What did you learn about teaching from this class? 8. What did you learn about student learning from this class? 9. What targets for improvement have you set yourself for this class, and are they realistic? 10. If you taught this class again tomorrow, what would you do differently? and why? NOTE: Choose two or three of these questions that will help you get feedback from the faculty member. Don't use all of them at one time. Adapted from: Killen, R. (1995) "Improving Teaching Through Reflective Partnerships." To Improve the Academy,

16 Peer Observation of Teaching Conversation Synopsis Summary of conversation: What are the peer observer s specific recommendations and instructor s planned future actions related to setting goals, actively engaging students, and assessing student achievement of goals? Instructor Name (Printed) Date Instructor Signature Observer Name (Printed) Date Observer Signature