Most discussion centered around evaluation of assessment tools, e.g. need to alter certain tools to measure cumulative learning.

Transcription

1 Summary of Actions Proposed or Taken to Improve Student Learning Feedback from the Assessment Council on and Excerpts from Exemplar PLO Assessment Reports BA ENGLISH Action Plans Thoughtful descriptions of faculty discussions. No changes in program discussed (senior thesis discussion resulted in including a secondary source and annotated bibliography). Most discussion centered around evaluation of assessment tools, e.g. need to alter certain tools to measure cumulative learning. For Outcome #3: During discussion, faculty agreed that the three essays were not adequate tools to measure the stated learning outcomes because they explored the works of one or two specific writers; instead of using essays based on specific studies of individual writers, assessment should employ a tool (such as a midterm or final exam) that measures cumulative knowledge, as emphasized by the rubrics. Also, this learning outcome (#3) should measure breadth of knowledge instead of in-depth knowledge, as a survey course should cover an extensive time period and a broad range of readings. The first specific outcome should therefore read: Demonstrate breadth of knowledge of one period of English literature. Faculty agreed that the learning outcomes were sound, but the tool was inadequate. In order to assess these four outcomes, alternate tools should be developed (for example, using both exams and essays as the bases for assessment, or requiring students to submit portfolios). Faculty also agreed that when learning outcomes are formulated for specific courses, the English Department should also suggest specific tools (signature assignments) for instructors to consider in assessing student learning. Faculty spent some time discussing the nature of a thesis, and they agreed that there should be a fifth outcome assessing grammar, diction, and syntax. The first outcome ("presents an original thesis and clear supporting arguments") should be rephrased ("presents a compelling argument"), and the second ("incorporates well-chosen quotations and effectively relates them to the thesis") should be changed to "incorporates evidence and effectively relates it to the thesis." The fourth outcome should include the possibility of APA documentation ("showing an understanding of APA or MLA style"). Faculty also discussed the logic of the scoring scale (should 3 be "satisfactory" or "average"?). Finally, faculty agreed that the prompt for the assignment should have been submitted by the instructors of the courses. Faculty agreed that the tool for assessment should be a senior thesis, a lengthy essay that allows students to bring to bear what they have learned in the course of their study at Notre Dame. The instructor of the course, Vince Fitzgerald, distributed copies of the senior theses to faculty before our scheduled meeting on Thursday, May 14, 2009, to discuss them. At this meeting faculty discussed six student theses, applying the five criteria of the rubric. One paper did not include secondary sources (criterion 2) and another did not have a works cited page or bibliography. Even though these papers were quite good, they did not meet the expectations of the learning outcomes for this course (demonstrate proficiency in research techniques). They therefore 1

2 received low scores on the first two criteria. The other four papers received scores of 3 ( satisfactory ) and higher. In the ensuing discussion faculty agreed that only the first two criteria assess the stipulated learning outcome measuring proficiency in research techniques. Faculty decided that the last three criteria, which do not measure this outcome, should be eliminated. The second criterion (Research, evaluation, and incorporation of secondary sources into literary analysis) should be expanded to include assessment of use of primary sources (Research, evaluation, and incorporation of primary and secondary sources into literary analysis). There should also be a third criterion taking a broader view of research techniques (for example, Ability to conduct and integrate research into a sustained and consistent thesis). Faculty discussed the tool itself (the senior thesis). Students in the course also submitted an annotated bibliography, which was not used as an assessment tool. There was agreement that senior theses should integrate secondary sources. The next time this course is assessed (in spring 2010), perhaps there should be two assessment tools, the senior thesis and the annotated bibliography. Faculty also agreed that the criteria should be defined. (What, for instance, does satisfactory really mean?) Faculty concluded that this course serves its purpose as a capstone experience permitting students to bring to bear accumulated knowledge and skills, but the rubrics need to be changed so that they address only the sixth learning outcome of the program (proficiency in research techniques). Senior theses should also integrate secondary sources and include works cited pages or bibliographies. For next Assessment Cycle ( ) Report actual student data results, with benchmark for satisfactory specified. Collect data in senior courses (not intro) in order to assess cumulative learning over the course of the program, and in order to ensure adequate numbers of majors for assessment. BS BIOLOGY Action Plans The Assessment Reports were detailed and had data and Rubrics for all the PLOs. However, in some of the reports data disaggregation was required as some had both Biology and Biochemistry majors taking the same course. Revised curriculum map with required coursework built for: Career Development Course in the Sophomore /Junior year Writing for the Sciences course in the Sophomore/Junior Year Revisit the PLOs and see if we need additional or different outcomes for assessment, also improve the Rubrics to align them with specific projects/ types of assignments for assessment Closing the Loop: For next Assessment Cycle ( ) : 2

3 a. Include more overall assessment of outcomes relevant to the field and subject. b. Include criteria and assessment of what is good enough c. Disaggregation of data BS PSYCHOLOGY: Action Plans Analysis Valid, appropriate use of descriptive and inferential statistics, if rather too exhaustive in early assessment; interesting comparison of evening and day students in #2; in mixed courses, should compare for significant differences between psych to other majors as well (#3) Action Plan: Based on faculty discussion Make sure to have faculty discussion on all assessments; More direct teaching of critical thinking Use day and evening courses to increase Ns, allow for further comparison Separation day and evening research courses already in place Try out a greater focus on writing the proposal in research as well Closing the Loop: Use ETS Major Field Test for #1 instead of modified GRE; Direct quantitative measurement for #4 MBA: Action Plans Re Leadership PLO #1 1. We currently offer a Leadership Concepts course as part of our Masters in Public Administration (MPA) program. Our Graduate Curriculum and Admissions Task Force will look at the possibility of requiring this course in the MBA curriculum as well. The committee will also discuss how content from this course can be embedded in other MBA courses throughout the curriculum. 2. Faculty will be asked to brainstorm and implement ways to improve their teaching of Leadership skills for each of their classes. Best practices will be documented and shared amongst faculty. 3. Our ACBSP and WASC assessment processes for Program Learning Outcomes and Institutional Learning Outcomes assessment will serve to focus our efforts on continually assessing and improving students Leadership skills. 4. Our MBA of the Future will directly address the concern of embedding more leadership content into our MBA curriculum. Right now, we believe there might be an opportunity to 3

4 leverage students work in groups as a Leadership learning tool. We will explore this option further. Re Communication PLO #2 1. Our Graduate Curriculum and Admissions Task Force will look at the possibility of adding a Business Communication course into the MBA curriculum. The committee will also discuss the possibility of screening students prior to admission for writing ability so that we can better understand which students will likely need extra communication training if admitted. 2. Faculty will be asked to brainstorm and implement ways to improve each of the three aspects of Communication (oral, written and information literacy) for each of their classes. Best practices will be documented and shared amongst faculty. 3. We will continue to send students needing extra help with their writing to the NDNU Tutoring Center. We will also increase our efforts to liaise with the writing tutors to better understand the causes of our students writing deficiencies. 4. Our ACBSP and WASC assessment processes for Program Learning Outcomes and Institutional Learning Outcomes assessment will serve to focus our efforts on continually assessing and improving students Communications skills. General Assessment Procedures: The following next steps will be implemented during our academic year: assessment results will be discussed broadly with faculty as well as SBM and NDNU administration. Feedback from faculty and administration will be used to suggest and implement further refinements to our pedagogy and curriculum in order to close the loop on above-mentioned issues. 2. MBA faculty and the MBA Program Director will jointly develop a detailed set of rubrics for each of the six Program Learning Outcomes for the MBA program. Our initial year of assessment using the multiple-course method of assessment purposely kept assessment at a more general level so that faculty could (1) best adapt to the new system, and (2) discover the important elements of assessing each of the learning outcomes which will feed into our development of the more detailed rubrics in We will endeavor to reflect upon and assess our own assessment processes. Faculty will be asked to provide feedback such aspects as whether our assessment processes are easy to implement, whether they are helping to improve learning outcomes, whether our rubrics properly align to the outcomes being assessed. 4

5 MA Education: Actions Plans The School of program faculty were invited to attend this retreat, during which the program director distributed the assessment data and facilitated a discussion of the results. Overall, faculty members are happy with students' achievement in the program. Nevertheless, some faculty wondered how to made EDU4703-Educational Assessment and Evaluation relevant to two distinct groups of students who take the course: students in the MA Education program, and students in the Administrative Services Credential Program. Dr. Tolley proposes trying again in Spring 2015 to offer an elective called "Instructional Design for Online Learning" essentially, this could be an updated version of one of the advanced technology classes still on the books. She shared with faculty that this spring, there are more than 500 open jobs in "curriculum development" and "instructional design" within commuting distance of Belmont, jobs that graduates would be well qualified to apply for, especially if they had the skills to design online courses. Faculty members were supportive, so Tolley will pursue this idea further and develop a proposal for a class over the summer. 5