Course Evaluations at the Kelley School of Business Administration of course evaluations According to official Kelley School of Business policy, every class taught at Kelley, including summer session classes, must be evaluated by students. In the MBA Program, the MBA Association (a student association) administers a parallel course evaluation system, the results of which are shared on-line with students. Indiana University (and therefore the Kelley School of Business) uses Explorance Blue technology to administer course evaluations online. About two weeks before your course ends, but before final examinations, your students will automatically receive email invitations to their official university email accounts to complete an online course evaluation for your course. Embedded in the email invitation is a link to the online system. Students click and complete the evaluation; it s just that simple. Additionally, both students and instructors may access a list of their current course evaluations at http://ocqbest.indiana.edu. Students will have approximately two weeks to evaluate the course; reminders are automatically sent to those who do not complete the evaluations. The online system will alleviate the need to schedule valuable class time to administer evaluations. It will also simplify the faculty s responsibilities during a very busy time during the semester, as faculty will have no role in the evaluation process at all. Literally everything is taken care of for the instructor! The Office of the Vice Provost for Undergraduate Education (OVPUE) answers Frequently Asked Questions (FAQ) about the all-campus online course questionnaire at the following website: http://ovpue.indiana.edu/assessment/surveys/ocqfaq.html. One particular question that often arises with respect to online course evaluation systems is the rate of response (RoR). Typically, students will respond at a rate of 50% or lower with online systems. Although studies have shown that the lower RoR yields reliable data, many instructors nevertheless worry about the lower rate. To improve your ROR, remind your students every time you see them during the evaluation period. Explain that course evaluations are important to you and to the school. Good evaluation data will help you continue to improve the course and the data will help the school ensure that Kelley maintains its top-notch reputation for teaching. It is NOT appropriate to award students points, a grade increase, or extra credit for completing the evaluations. Students must have the opportunity to opt out without sacrificing or forfeiting course credit of any kind, in any way. Some instructors may wish to schedule class time to allow students to complete course evaluations on their devices in class. Should you decide to do so, you MUST NOT be present while students are completing their evaluations. Further, it is NOT appropriate to offer students pizza, doughnuts, cookies, brownies or any other kind of treats while they complete course evaluations. A few weeks after the semester concludes the data will be processed, with results available generally speaking about 3 4 weeks after finals week. Data summaries will then be available to instructors on-line at http://kelley.iu.edu/set/.
Description of Course Evaluation Items The first nine items are common for all courses taught on the Indiana University Bloomington campus. The resulting data for the first four are used for promotion and tenure purposes and are not available to students. 1. How clearly were course learning goals and objectives communicated to you? 2. How effectively was class time used to help you learn? 3. How effectively did out-of-class work (assignments, readings, practice, etc.) help you learn? 4. How available was the instructor to provide help when needed (in person, by email, office hours, etc.)? The next five items will not be used for tenure and promotion purposes. The resulting data will be available to students with access through IU-authenticated login. Student-access results will appear on a dashboard with visual displays of descriptive statistics. 5. How likely would you be to recommend this course with the instructor? 6. How much did the instructor motivate you to do your best work? 7. How much did the instructor emphasize student learning and development? 8. Compared to other courses you ve taken how much time did this course require? 9. In a typical week, about how much time did you devote to this course? (Do not count scheduled class time, labs, etc.) Items 10 through 20 are unique to the Kelley School of Business. Of those, items 11 through 18 carry special significance, as they comprise the Dean s Eight, an index of highly correlated items used alongside the data from item 18 by itself to make decisions about promotion, tenure, teaching awards, retention and renewal of contract for non-tenured faculty, and for a number of other personnel-related issues. Items 19 and 20 are open-ended questions designed to afford students the opportunity to articulate their individual thoughts about the course. 10. The subject matter of this course has been mentally challenging. 11. The instructor made the subject matter more meaningful to me through the use of examples and applications. 12. I feel I learned a lot in this course. 13. The instructor was well prepared for class meetings. 14. The instructor displayed an enthusiastic interest in the subject matter of this course. 15. The instructor created an environment in which students felt comfortable asking questions and expressing their views. 16. Explanations of the material in this course were clear and to the point. 17. All things considered, I have benefited from having taken this course. 18. Overall, I would rate this instructor as outstanding. 19. What did you like most about this course or instructor? 20. What did you like least about this course or instructor?
Analyzing Your Course Evaluations The data returned from your student evaluations will be available to you in two formats, one generated from the IU Campus and one internally by the Kelley School of Business. 1. IU Bloomington Data Summaries When the data have been processed, you will receive a notice with the following link, allowing you to access your data in the IU Campus format: http://ocqbest.indiana.edu. Below is an example of the IU Campus format for item 18, Overall, I would rate this instructor as outstanding. OCQ (Online Course Questionnaire) Report Individual Report for Smith, Johanna (Q350) [course number] 12345 [section number] IUB Spring 2015 Online Course Questionnaire Project Audience 31 [enrollment for the course] Responses Received 13 Response Ratio 45.2% Overall, I would rate this instructor as outstanding. Question Strongly Slightly Undecided Slightly Strongly Overall, I would rate this instructor as outstanding. 0.0% 0.0% 0.0% 0.0% 7.7% 38.5% 53.8% Question Course Department* Mean Response Count Standard Deviation Mean Response Count Standard Deviation Overall, I would rate this instructor as outstanding. 6.5 13 0.7 5.8 120 1.3 *NB: the descriptive statistics for Department in the IU Campus format refers to the entire business school, NOT to departments within the business school.
2. Kelley School of Business Data Summaries and Charts You will also receive an email with a link, http://kelley.iu.edu/set, to your data in the Kelley format. After you authenticate, you will have access to reports that provide descriptive statistics as well as visual charts to help you interpret your data. Indiana University Kelley School of Business Spring 2015 Instructor Teaching Evaluation for Course: Q350 Section: 12345 Instructor: Johanna Smith Campus: Bloomington Number Enrolled: 31 Strongly Slightly Slightly Strongly Course Mean Undecided Total St. Dev. 1 2 3 4 5 6 7 Item 18. Overall, I would rate this instructor as outstanding. Frequency 0 0 0 0 1 5 7 13 6.46 Percent 0.0 0.0 0.0 0.0 7.7 38.5 53.8 100.0% 0.66 The table above represents an excerpt from a larger document with data from all 20 course evaluation items. To read your data, note the following: Frequency = the number of students who selected that response for the given item. Here, for example, one student selected slightly agree ; five students selected agree ; seven students selected strongly agree. The class mean appears to the far right in the frequency line, though it is unrelated to frequencies. It is simply the average of all the students selections. Here, the mean is 6.46. Immediately below the mean is the standard deviation. The greater the standard deviation, the greater the spread among student responses. In other words, high standard deviations means your students responded all over the place. A low standard deviation means your students responded relatively uniformly, as in this example. The course total to the left of the mean (here 13) is the total number of responses. To figure the rate of response, simply divide the course total by the number enrolled in the header. Here, the rate of return would be 42%. In addition to the tabular reports, you will also have access to data charts to help you interpret the results of your student evaluations. CRED Charts (acronym explained below), are provided for the Dean s Eight Index and for Item 18. They are designed to give you a longitudinal look at the data from your student evaluations as well as a comparative perspective. The chart below is an example for Item 18 Overall I would rate this instructor as outstanding. The X axis provides the semester and year in which the course was taught. S = Spring; M = Summer; F = Fall. The Y axis provides the response categories. Note that the Y axis sets the midpoint of possible selections (4 = undecided) at 0. This is to make the chart more readable by lengthening the distance
between 4.0 and 7.0. At the Kelley School of Business, means less than 4.0 are rare. The Black dots and line represent the mean score for the instructor for the course in which the evaluations were completed. The red X s and connecting line represent the mean score for the program in which the course was taught (e.g., the Undergraduate Program). The black circles and dotted lines represent the department in which the course was taught (e.g. finance). The red asterisks and vertical lines show the standard deviation for the program s mean, indicating the range within which the vast majority of responses fell. In this example, Johanna Smith, the instructor, should interpret the increase in her mean score as significant improvement in her teaching, especially since the mean was below the program and department means in spring 2013, near parity with the program and department means in spring 2014 and above the other means in spring 2015. CRED (Campus Required Evaluation Dossier) Chart