A. Student Learning Outcomes (SLO) for the M.S. Degree: Students will

Transcription

1 1 A. Student Learning Outcomes (SLO) for the M.S. Degree: Students will 1. Demonstrate general content proficiency in three subject areas of the chemical sciences, and specific content proficiency within the designated primary research area. a. 90% of M.S. students will score at least 50% on ACS standardized exams in three subject areas selected from Analytical, Physical, Inorganic, and Organic Chemistry, and Biochemistry. One of these areas must be in the primary research area. b. 90% of M.S. students will meet or exceed expectations on a written thesis, portfolio, or published paper rubric. c. 90% of M.S. students will meet or exceed expectations on an oral comprehensive exam given during the final defense. d. 90% of M.S. students will meet or exceed expectations in external employer evaluations. e. 100% of M.S. students will receive their degree within a five semester consecutive term. 2. Effectively engage and instruct undergraduate students in the introductory chemistry laboratory. a. 100% of M.S. students will comply with all teaching laboratory safety policies and procedures as defined by a safety rubric. b. 90% of M.S. students will be judged as effective laboratory instructors by 70% of undergraduate students in their assigned teaching labs. 3. Deliver and defend an oral presentation of research. a. 90% of M.S. students will meet or exceed expectations in two seminar presentations of faculty-selected scientific journal articles in the candidate s primary research area. b. 90% of M.S. students will meet or exceed expectations on a final oral defense of research rubric.

2 2 B. Explanation of Methods The following table summarizes the direct methods that will be used in the M.S. Assessment Plan to gather quantitative data on each student learning outcome (SLO) listed in the previous section. Aggregated data will be used for regular programmatic review and revision. The timeline for the M.S. Assessment Plan begins with Semester 1 and denotes the semester that a new graduate student enters the program, as we admit students in both fall and spring semesters. The Director of Graduate Studies and departmental Assessment Committee will be responsible for aggregating and reporting the data to faculty for regular review and discussion. Summary of methods, timeline, persons responsible, and aligned learning outcomes Methods Description Timeline Person(s) Responsible Standardized Testing To meet target, students will score 50 percent (half of all questions answered correctly) on three content exams developed by the ACS Division of Chemical Education Examinations Institute. Semesters 1 and 2 Divisional faculty and Director of Graduate Studies SLO Addressed 1a Oral presentation rubric This rubric will assess both general education and technical standards for an effective seminar given to peers and faculty. Additional indicators in the rubric will be applied to the final oral defense. Seminar: Semesters 3 and 4 Defense: Semester 5 Seminar Coordinator and Research Advisor 3a, 3b Written presentation rubric This rubric will assess the quality of the written thesis or portfolio prior to faculty revision. To measure candidate growth, the tool will be re-applied as the candidate continues to revise and develop the final version that is submitted to the Graduate School. Semester 5 (Applies only to M.S. thesis and teacher certification options. A separate rubric will be applied to a published paper for the M.S. essay option). Thesis Defense Committee 1b Comprehensive Exam An oral exam given during the final defense will assess the candidate s GRE-level content proficiency in the primary Semester 5 Thesis Defense Committee 1c

3 3 research area. TA (Teaching Assistant) safety rubric This rubric will be applied to all teaching assistants to assess consistent and proper use of safety practices and procedures in the teaching laboratory. Attendance and participation in all TA-related safety activities in CHEM 690 (a professional development course) will also be measured. Safety: Every semester CHEM 690: Semesters 1 and 2 Lab Manager and Safety Committee 2a TA (Teaching Assistant) teaching rubric A rubric will be applied to all teaching assistants to measure their consistent use of academic integrity standards and teaching effectiveness in the chemistry instruction laboratories. Every semester Coordinator for Secondary Science Education Certification and Office Assistant Specialist 2b External evaluation Employers will be surveyed for satisfaction levels. Additionally, students entering and exiting the M.S. program will be surveyed for their views on quality of training. 2 years and 5 years after M.S. conferral Department chairperson and Office Assistant Specialist 1d, 1e Summary of how SLOs align with assessment methods SLO Standardized Testing Oral Presentation Rubric Written Presentation Rubric Comprehensive Exam TA Safety Rubric TA Teaching Rubric External Evaluation 1a 1b 1c 1d 1e 2a 2b 3a 3b

4 4 C. Available Data Data is limited since the Department of Chemistry and Biochemistry has not developed an Assessment Plan for its M.S program prior to this point. As a result, a formal list of learning outcomes and associated methods for gathering data has been lacking, until now. This section summarizes the data we do have and attempts to align it to the nine Student Learning Outcomes listed in section A. Standardized Testing (SLO 1a) During the week prior to the start of classes, all entering graduate students are required to take the ACS standardized exams in four chemistry subject areas (organic, inorganic, analytical, and physical). Aggregated results for a seven semester period from fall 2009 through fall 2012 (data were not available for spring 2012): Of students that took the exams in this period, only 11% scored at least 50% in three subject areas (n = 46). Ten of these students are current declared M.S. students. In this subgroup, 20% met the SLO 1a target by scoring at least 50% in three subject areas. None of these students were required to take the exams a second time. Oral Presentation Rubric (SLO 3a, 3b) There are two primary times in the M.S. program where students give oral presentations. These include CHEM 615, a seminar course where students give a presentation on a science journal article; and the oral thesis or portfolio defense, where students present their final research outcomes to the public and a thesis defense committee. Rubrics have not been applied to any of these. As a result, assessments have been informal and specific data related to actual learning outcomes have not been gathered or aggregated. The only data we do have are the letter grades given for CHEM 615, which are 96% A. No data is available on actual student learning outcomes. D. Use of Results Available data for the M.S. program have not been reviewed by faculty in the department, and much data is missing or absent since the SLOs in section A are mostly new. The new Director of Graduate Studies (DGS) for the department (appointment effective July 1, 2012) communicated the urgency of this problem in a meeting with the department s Graduate Program Committee on August 21, Evidence was shown to all members, which included the UAP evaluation rubric that was applied to our graduate programs and relevant copies of past letters and s. The current DGS has also had several meetings or exchanges with Sherrill Morris to learn about our status in the current eight-year re-accreditation cycle and to discuss actions that are now mandatory. In response, our Graduate Program Committee approved the SLOs in section A on September 25, The DGS and Graduate Program Committee (which also functions as the Assessment Committee) will develop the missing assessment methods listed in section C. A formal presentation will be made to the entire faculty later this semester to seek majority buyin. Implementation efforts have already begun and these are detailed in the next section. E. Gaps in Current Data A major concern expressed in past reviews by the Academic Planning Council was the lack of outcomes, assessment methods, and their implementation. We now have M.S. outcomes. Although most methods have not yet been developed, a goal set by our Assessment Committee is to have these in place by spring

5 and to hold students accountable for each method as support and training resources are put into place. As evidence of our commitment to this process, new pages on our departmental web site have been created to inform and provide resources to both faculty and students of our Assessment Plan. In particular, the page at now lists timeline documents (more are being developed) that are designed to integrate deadlines, forms (both graduate school and departmental) and assessment into a single package. More specifically, these documents will reference the Student Learning Outcomes articulated in this document. Timeline documents are available that show the first three semester timeline for the M.S. program and the final, defending semester. Interested evaluators can view the current timeline documents by clicking the above link. They will provide a good sense of both format and implementation process. In addition to being embedded in the timeline documents, the SLOs and method descriptions have been posted separately on the web page to share them at an early stage with students already in the program. An Assessment section now exists on the page that will link to relevant SLO-aligned methods, as we develop and implement them. An M.S. method (T.A. Teaching Rubrics) is already available for review and is being implemented this fall 2012 term. It shows the format that will be applied in all our rubrics. The following paragraphs address more specific gaps. SLO 1a: Standardized ACS exams have been administered to all incoming graduate students during orientation week. The results have been used as an advising tool to suggest courses that the student should audit or attend. In practice, there has been no follow-up on this advice. This new SLO will require that the student demonstrate ACS exam proficiency in three subject areas by the end of semester 2 (similar to requirements in graduate chemistry at University of Wisconsin at Milwaukee and the University of Illinois at Chicago). SLO 1c: During the oral defense of thesis or portfolio, the M.S. student is expected to present their research and respond to questions about the methodology, data analysis, and possible future work. As a final content proficiency measure, M.S. students will now be required to take a comprehensive oral exam concurrent with the oral defense that examines a broader GRE-level content proficiency in the primary research area. Separate rubrics will be developed to assess the oral research and written defense components. SLO 2a, 2b: In the past, safety violations by graduate teaching assistants were noted by the Laboratory and Safety Manager, but were never used to evaluate the degree progress of the candidate. In addition, teaching was assessed using a single question that undergraduates in the instructional laboratories would answer: Would you have this TA again? Safety has now been integrated with teaching into the M.S. Assessment Plan, and separate rubrics (included in the Appendix) are now being applied in the fall 2012 term to evaluate teaching effectiveness and compliance with all required safety practices. Quantitative data will be available at the end of this semester.

6 6 Appendix The following rubrics have been developed and are being applied in the fall 2012 term TA Teaching Rubric, p.7 Aligned with SLO 2b TA Safety Rubric, p. Aligned with SLO 2a

7 7 TA Teaching Rubric Applies only to Introductory Chemistry Teaching Labs (CHEM 111, 212, 213) Name of TA: Course: Section: Indicator Beginning (1) Progressing (2) Proficient (3) Advanced (4) Score The NIU Online Tutorial on Academic Integrity is mandated and policies are implemented. Instructions for completing the online tutorial were not clearly stated at the beginning of the semester. Instructions for completing the online tutorial were clearly stated at the beginning of the semester but the Certificate of Completion was not collected. Instructions for completing the online tutorial were clearly stated at the beginning of the semester and the Certificate of Completion was collected. Instructions for completing the online tutorial were clearly stated at the beginning of the semester; the Certificate of Completion was collected; and there was evidence of consistent enforcement. Objectives at the beginning of the lesson are clearly stated Lesson objectives are not communicated to students at the beginning of the lab. The TA communicates objectives to students The TA communicates objectives to students and students are able to seek clarification of the lesson objectives to reach understanding. Students are able to effectively communicate the lesson objectives to one another. TA is organized and prepared to begin class Unorganized and wastes time. Unprepared to teach. Usually organized and prepared. Lesson and lab is organized and class always begins on time. TA is prepared to teach the lesson. Lesson and lab is organized and begin on time with no delays throughout. TA is prepared and knows the content well. Monitors student understanding No attempts are made to determine if students understand the lesson. Occasionally checks for understanding. Consistently monitors student performance and understanding. Anticipates areas that may be difficult to understand continually and checks for understanding. Understands how to ask questions to stimulate thinking and discussion among lab partners Lower level or no questioning with little time for student response. Uses a variety of questioning techniques to stimulate thinking and discussion. Inconsistent in providing adequate response time. Uses a variety of questioning techniques to stimulate thinking and discussion. Students given adequate time to respond. Uses questioning techniques to stimulate higher level thinking skills that promote meaningful interactions.

8 8 Gives instructions clearly and completely Instructions are unclear and confusing. Provides basic instruction. Instructions are clear and complete. Instructions are clear, complete and all expectations are clearly stated. Shows energy and enthusiasm for teaching and subject matter Shows little or no energy or enthusiasm toward teaching and subject matter. Energy and enthusiasm for teaching and subject matter are inconsistent. Demonstrates sincere energy and enthusiasm for teaching and subject matter. Energy and enthusiasm for subject matter inspires students Writes and speaks clearly and correctly Frequent errors in written and oral communication. Occasional errors in written and oral communication. Written/oral communication is always informative; expressed in standard English. Written and oral communication is exemplary. Monitors and adjusts teaching strategies to meet the need of students No evidence of monitoring or adjusting teaching strategies. Occasionally monitors and adjusts teaching strategies. Consistently monitors student learning to adjust teaching strategies. Makes individual adjustments in teaching strategies based on individual student learning. Fairness of grading evaluation criteria made clear to students Evaluation criteria and standards have not been developed. Evaluation criteria and standards have been developed, but are not clear or have not been communicated to students. Evaluation criteria and standards are clear and have been communicated to students. Evaluation criteria and standards are clear and have been communicated. Instructor checks to make sure everyone understands the expectations verbally and in writing. Feedback to students Minimal feedback given to students. Feedback to students is general in nature and/or delayed. Feedback to students is individualized and completed in a timely manner. Feedback to students is individualized and completed in a timely manner and causes the student to reflect on their own learning. Available outside of class for additional help Never available. Occasionally available by appointment only, office hours limited. Available by appointment during office hours, or study groups meet at least once a week. Office hours are adequate. Study groups meet at minimum one time per week.

9 9 TA Safety Rubric Name of TA: Current Semester: Attended safety presentation during orientation week Yes No Attended the fire safety demo and applied proper technique during the fire extinguishing activity Yes No Participated in the IDOL/OSHA audit review session Yes No Participated in the Environmental Health and Safety Presentation Yes No Wore safety goggles while instructing and supervising lab Always Often 1 Sometimes 5 points Never 15 Points Undergraduate students wore safety goggles and displayed proper clothing while in the laboratory Always Often 1 Sometimes 5 points Never 15 Points Properly cleaned lab after each session and at the end of the semester Always Often 1 Sometimes 5 points Never 15 Points Lab waste was properly sealed and labeled Always 1 Sometimes 5 points Never 1 Must score at least 105 points to comply with departmental safety regulations