EXECUTIVE SUMMARY. List all of the program s learning outcomes: (regardless of whether or not they are being assessed this year)

Transcription

1 STUDENT LEARNING ASSESSMENT REPORT SUBMITTED BY: JOHN PATRICK MULLINS DATE: SEPTEMBER 2014 BRIEFLY DESCRIBE WHERE AND HOW ARE DATA AND DOCUMENTS USED TO GENERATE THIS REPORT BEING STORED: THE DATA ARE OBTAINED FROM PARTICIPATING FACULTY AND STORED IN DR. MULLINS S OFFICE EXECUTIVE SUMMARY List all of the program s learning outcomes: (regardless of whether or not they are being assessed this year) Learning Outcome Year of Last Assessment Year of Next Planned Assessment Demonstrate satisfactory knowledge of theory for history majors. Demonstrate satisfactory skills in undergraduate research and writing skills necessary for history. OUTCOME THREE: Students will be able to communicate effectively in oral and written forms and apply problem solving and analytical skills to attain professional goals. OUTCOME ONE: Students will be able to comprehend historical events through identification of their causes and consequences and analysis of primary and secondary evidentiary sources by means of close reading of texts within historical context. OUTCOME TWO: Students will be able to research and integrate evidence from primary and secondary sources by synthesizing data into a general interpretation of past events in a logically structured, interpretive paper that applies knowledge of historical theory and method. (INQUIRY) Describe how the program s outcomes support Marymount s Mission, Strategic Plan, and relevant school plan: Marymount s history faculty view academic excellence and preservation of the liberal arts tradition as inherent to the History Program mission as they are central to the Mission of the University. Through its internship program, which places students in some of the best historical and research settings in the region, the program also implements the University mission s prioritization of experiential learning, career preparation, and opportunities for professional growth. The Student Research Conference offers history majors another opportunity for building professional skills. Finally, as is demonstrated in the analysis of the first two outcomes of this report, the work of the program reflects the Scholarship hallmark claimed by the University, encouraging and rewarding intellectual curiosity.

2 The Shared Strategic Priorities described in the 2009 Strategic Plan emphasize Enhancing the Intellectual Experience through rigorous programs that nurture scholarship and inquiry. Learning Outcomes One and Two in this report reflect student achievement in the History Capstone course, Senior Seminar. This course requires seniors to complete a research paper of significant length that involves inquiry into important historical questions. Completion of this course requires research into evidentiary sources including analysis of primary sources and scholarly literature and synthesis of evidence into a general interpretation applying knowledge of theory and method. Students must have acquired the skills and historical knowledge required in history major courses throughout their undergraduate program of study in order to accomplish successfully goals of the Senior Seminar. As mentioned above, the internship offers another avenue for majors to travel to achieve personal and professional goals that are cited as a part of the strategic priorities in the 2009 Strategic Plan. Provide a brief description of the assessment process used, including strengths, challenges and planned improvements: The assessment process focused on two performance areas: the Senior Seminar and the Internship. These are the areas that best reveal the performance of students at the time that they are completing their studies in the History major. This year, as previously, one of the three outcomes (Outcome Three) drew upon student performance in HI 400 Internship, while the two remaining outcomes drew upon student performance in HI 420 Senior Seminar. As noted above, the centrality of historical research to Senior Seminar makes that course peculiarly well suited to assessment of interpretive thinking and critical understanding of history as well as inquiry-based learning, as per Outcomes One and Two. The applied aspect of the assessment process continues to be an examination of student performance ratings in various types of work sites that require them to demonstrate intellectual skills outlined on the university s internship evaluation form for internship site supervisors. Senior Seminar is the only classroom-based course at Marymount restricted entirely to History Majors, and this has been an ongoing challenge to assessment of student learning within the Major. In , the History Program began offering HI 250 Research and Writing each fall semester as a course separate from POL 250, whereas previously the two had been cross-listed for both programs. Although HI 250 is oriented primarily toward History Majors, the History Program cannot count student performance in HI 250 for program assessment purposes, because the course remains open to non-majors. The cancellation of a section of HI 250 proposed for Spring 2013 due to insufficient enrollment has shown that the History Program does not have enough Majors (30-40 each year on average) to sustain more than one section of HI 250 per academic year, hence the need to keep it open to non-majors. Our History Majors would be well served by more classroom courses (other than Senior Seminar) which are restricted to Majors, but the numbers of students majoring in History at Marymount cannot sustain such courses at this time. Over the last few years the History Program has suffered the loss of two faculty members due to resignation, one faculty member due to retirement, and one faculty member due to accidental death. It has been a difficult time for the Program and our Majors. We have had to cope with lack of sufficient full-time faculty to support the assessment process. As Fall 2013, the History Program regained a full complement of four full-time history faculty members, and that was a strength we relished over this last academic year. (Regrettably one of those new hires, Dr. Maccari-Clayton, resigned at the end of the academic year, which

3 poses yet another challenge for ) Dr. Patrick Mullins had taught Senior Seminar exclusively since Spring In the course rotation for agreed upon in Fall 2013, Senior Seminar and HI 250 were put into rotation among all faculty members. The learning of Marymount s History Majors will benefit from the addition of new faculty members in general and more particularly from the teaching of HI 420 and HI 250 from the differing perspectives on content, theory, and method that will be brought to the classroom through rotation of those courses among different faculty members (e.g. early Americanist, modern Americanist, ancient and medieval Europeanist, modern Europeanist). Moreover, the History Internship program will be directed by Dr. Mark Benbow rather than Dr. Patrick Mullins as of Fall 2014, and Dr. Benbow brings a wealth of personal experience and professional connections as a public historian to the identification of internships and the assessment of student learning into those internships. The teaching of HI 420 by Dr. Stuckey in Spring 2015 and of HI 400 by Dr. Benbow in is the main improvement planned for Describe how the program implemented its planned improvements from last year: Outcome Planned Improvement Update (Indicate when, where, and how planned improvement was completed. If planned improvement was not completed, please provide explanation.) please see note below NOTE: The only planned improvement to the History Program specified in the Student Learning Assessment Report was the search for two new full-time faculty members (for details on how this improvement implements objectives from the Program s 2011 Action Plan, please see below for response to recommendation #1). That improvement was accomplished in the academic year with Dr. Mark Benbow and Dr. Marina Maccari-Clayton beginning their terms as full-time faculty in the History Program. The ramifications of this improvement with respect to assessment of student learning within the History Major can be weighed at the conclusion of the academic year, when Dr. Stuckey and new full-time historian Dr. Benbow take over Senior Seminar and the History Internship as per the new course rotation. Provide a response to last year s University Assessment Committee review of the program s learning assessment report: The University Assessment Committee accepted the History Program s assessment report as submitted and gave our program a sterling grade. The History Program deeply appreciates this positive response to our efforts, particularly during a particularly trying academic year. The UAC concluded that the History Program met the criteria in six out of seven categories and partially met the criteria in that seventh category. In its Fall 2013 review of the assessment report, the UAC made two recommendations: 1) Continue to report on the success of action plans for program improvement.

4 With the recruitment of Drs. Benbow and Maccari-Clayton in Fall 2013, the History Program finally fulfilled a two-part objective first outlined in the Action Plan we adopted following the 2011 Program Review, namely: Hire Modern US Historian with Public History subfield and Hire Modern European Historian with non-western subfield. 2) Respond to suggestions made in each of the critical categories in this report. The UAC review included one critical comment, specifically on the requirement in the rubric for Academic Learning Outcome #1, that the student have completed assigned reading. The rubric in the report indicated that 11 out of the 11 students surveyed had exceeded expectations on this requirement. The UAC quite rightly objected that it was impossible to exceed such a modest expectation (!), and on that basis concluded that the program had partially met the criteria for assessment of learning outcomes. This particular requirement has been cut from the rubric for the first learning outcome as inappropriate for assessment.

5 Outcome and Past Assessment Learning Outcome 1: Students will be able to comprehend historical events through identification of their causes and consequences and analysis of primary and secondary evidentiary sources by means of close reading of texts within historical context. Is this outcome being reexamined? Yes No If yes, give a brief summary of previous results (including trends) and any changes made to the program. Assessment Activity Outcome Measures Explain how student learning will be measured and indicate whether it is direct or indirect. Direct Measure: The direct measures for History Majors in HI 420 Senior Seminar is oral and written analysis of assigned reading of primary source texts (documents, images, films, etc.) and scholarly secondary sources (monograph on teaching method, chapters from scholarly collection on theory) Performance Standard Define and explain acceptable level of student performance. Direct Measure: A meets the criteria rating or better on two thirds of the items on the rubric achieved by two thirds of the class is the acceptable level of student performance. Data Collection Discuss the data collected and student population Direct Measures: Results of student performance in oral and written analytical work according to course requirements specified in syllabus and particular class assignments, as provided by sample of nineteen students. Analysis 1) Describe the analysis process. 2) Present the findings of the analysis including the numbers participating and deemed acceptable. If students achieved the acceptable level of performance on this outcome, the instructor assesses their analytical skills adequate to meet program expectations. The instructor has this in mind when choosing common readings and approving research topics. It is hoped that 20% of a student s final grade in this capstone course is considered substantial enough to motivate students to seek excellence in the exercise of their analytical skills. All 19 students enrolled in HI 420 in Spring 2014 met or excelled the criteria for analytical thinking in class discussions and written analytical papers. (revised for simplification in September 2014) Students who meet the criteria on this learning outcome demonstrated skill in the analysis and comprehension of primary and secondary sources, as well as understanding of the content for this course:

6 specifically, how historians study human behavior in the past, the methods they use to acquire evidence of this subject, and the theories of human nature they apply in interpreting that evidence Interpretation of Results Extent this Learning Outcome has been achieved by students: Based on the direct measure (class participation through oral and written analysis of assigned primary and secondary evidentiary sources), the first learning outcome was successfully fulfilled, as nineteen out of nineteen students in HI 420 Senior Seminar in Spring 2014 met or excelled the criteria for this outcome. This year there was no indirect measurement tool. Program strengths and opportunities for improvement relative to assessment of outcome: By the time the students came to this capstone course, they were well prepared in their analytical and writing skills thanks to the HI 250 Research and Writing course and their previous experience in upper-division history courses. Some students excelled at written and oral analysis of primary source documents and secondary sources on theory and method. All students were at least satisfactory in their performance. Student performance on the direct measure in previous Student Learning Outcome Reports suggests that the program is strong overall in the area measured by this outcome, but there remains room for improvement in leading our students to academic excellence. Discuss planned curricular or program improvements for this year based on assessment of outcome: The Program could use improvement in preparing students for understanding of theory and methodology in the history discipline. It is difficult to address this problem before the capstone course, since upper-division courses serves primarily the needs of the Core and are not mainly populated by History Majors, but the bifurcation of HI/POL 250 has enabled the History faculty to provide students an effective introduction to historical methods before Senior Seminar. In the Spring 2014 Senior Seminar, Dr. Mullins assigned one book on teaching methodology and one collection of scholarly articles on the theory and method of the historical discipline, in addition to primary source materials. This curricular change was made to address the concern stated in the 2013 Assessment Report that History Majors needed greater cultivation of their understanding of theory and method. In Spring 2014 Senior Seminar, a majority of HI 420 students rose to this challenge. Since all students met the criteria for the first learning outcome in , no overhaul of the curriculum or program with respect to this outcome is currently under consideration.

7 Rubric for Measuring Outcome One: Fails to meet criteria Exceeds Meets Criteria Criteria Oral and Written Class Participation Student is able to identify causes of the historical event/s discussed in the reading and their interrelationships 8 11 Student understands the historical interpretation contained in the essay/article/book 5 14 The student paper identifies the author s thesis 9 10 The student paper demonstrates comprehension of the content The student paper demonstrates accurate critical analysis of the reading 6 13

8 Outcome and Past Assessment Learning Outcome 2: (Inquiry Outcome) Students will be able to research and integrate evidence from primary and secondary sources by synthesizing data into a general interpretation of past events in a logically structured, interpretive paper that applies knowledge of historical theory and method. Is this outcome being reexamined? Yes No If yes, give a brief summary of previous results (including trends) and any changes made to the program. Assessment Activity Outcome Measures Explain how student learning will be measured and indicate whether it is direct or indirect. Direct Measure: The research prospectus and bibliography, two drafts of the final research paper, and oral class presentation of research required in Senior Seminar HI 420 will provide the data. The measures are identified in the rubric below. Performance Standard Define and explain acceptable level of student performance. Direct Measure: The acceptable level of student performance will be a meets the criteria rating achieved by two thirds of the class on two thirds of the items on the rubric. Data Collection Discuss the data collected and student population The electronic documentation of the research paper, prospectus, and PowerPoint presentation provides data for measurement of the outcome as performed by a sample of nineteen students. Analysis 1) Describe the analysis process. 2) Present the findings of the analysis including the numbers participating and deemed acceptable. Nineteen students engaged in original research of both primary and secondary sources and produced a 15+ page research paper that included a review of the scholarly literature, analysis of primary source documents, and an original analysis of the chosen subject. Toward that end, all students explained their work to the instructor in a research prospectus and bibliography, submitted a first draft of the research paper as well as a second draft that implemented required revisions, and presented their research findings in an oral class presentation accompanied by PowerPoint. Their work was evaluated according to the attached rubric. In Spring 2014, all students in HI 420 met or

9 exceeded the criteria on the first and second drafts, the research prospectus, and the oral presentation of research. Interpretation of Results Extent this Learning Outcome has been achieved by students: Based on the direct measure (research paper drafts, prospectus & bibliography, class oral presentation), nineteen out of nineteen students met or exceeded the criteria for Learning Outcome Two. This year there was no indirect measurement tool. Program strengths and opportunities for improvement relative to assessment of outcome: Because all students are required to take the HI 250 Research and Writing course, and because all upper division courses in history require an analytical paper of some kind, students come to the capstone course with considerable experience in forming a research question, developing a thesis, presenting an argument, research primary and secondary sources, and writing a research paper. As was the case in , reported in the History Program s previous assessment report, there was no indication of a performance gap between four-year Majors and transfer students in HI 420 in the academic year. Discuss planned curricular or program improvements for this year based on assessment of outcome: Since all students met the criteria for this learning outcome, no overhaul of the curriculum or program with respect to this outcome is currently under consideration. Rubric for Measuring Outcome Two: Exceeds Criteria Meets Criteria The paper presents a well formulated appropriate research question The paper applies appropriate primary and/or secondary resources to the investigation of the research question The paper is well organized and well written Fails to Meet Criteria

10 The paper presents appropriate analysis and interpretation of the sources as they pertain to the research question Chicago style is correctly used to document sources in footnotes and bibliography The findings are plausible and well defended. 8 11

11 Outcome and Past Assessment Learning Outcome 3: Students will be able to communicate effectively in oral and written forms and apply problem solving and analytical skills to attain professional goals. Is this outcome being reexamined? Yes No If yes, give a brief summary of previous results (including trends) and any changes made to the program. Assessment Activity Outcome Measures Explain how student learning will be measured and indicate whether it is direct or indirect. The direct measure for the outcome is criteria taken from the Center for Career Services Employer Evaluation of Student Internship Performance Form that every internship site supervisor completes for students enrolled in HI 400 Internship. (Note that the CCS revised its criteria over the academic year, reducing them from 14 in Fall 2013 to 9 in Spring 2014.) The Indirect Measure: The Performance Standard Define and explain acceptable level of student performance. Direct Measure: The acceptable level of student performance will be a good or fair rating on six out of the nine criteria selected from the CIC Employer Evaluation of Internship Form achieved by 80% of the students completing internship in the year. Indirect Measure: The acceptable level of performance will be 80% of students reporting in their Internship Journals, Data Collection Discuss the data collected and student population Submission of the Internship Journal, Summary Reflective Statement, and the completed CCS Student Evaluation Form is an internship requirement stated in the Internship Syllabus which provides the data for assessment of the Indirect Outcome. Analysis 1) Describe the analysis process. 2) Present the findings of the analysis including the numbers participating and deemed acceptable. The nine criteria decided upon by the CCS for evaluation of student performance on internships jibe well with the History Program s standards for applied skills and knowledge from the History discipline in a workplace context and overall professionalism. The CCS provides site supervisors with assessment criteria in a midterm evaluation form, so they are very familiar with university assessment standards before submitting a final evaluation of student performance in the internship. The students journals and reflective papers provide insight into the meaning of these criteria in the individual settings in which students worked as interns within the discipline of History. From Fall 2013 through Summer 2014, thirteen History

12 student s Internship Journal, Summary Reflective Statement (short essay), and the CCS Student Evaluation Form (self-critique). reflective essay, and selfcritique, indicating in what ways it was a learning experience. Majors completed internships (including the completion of student teaching by History and Social Science/Secondary Teaching Certification Majors). In terms of ratings on the nine criteria for internship performance, all thirteen students met the criteria for a rating of excellent or good/fair from the site supervisor; no student received an unsatisfactory evaluation on any criterion. Five students received an overall rating of good, while eight received an overall rating of excellent. None were rated overall as fair or unsatisfactory. Indirect Measure: All students completed Internship Journals and Reflective Statements. All students noted that in the end they were adequately prepared to take on the tasks assigned to them. All noted the courses in the History Program and the skills they acquired there which helped them take on the challenges of their experiential learning in a professional setting. (Revised September 2014 in light of change in CCS procedures) Interpretation of Results Extent this Learning Outcome has been achieved by students: As noted in the narrative above and the tabulations on the rubric below, all students met the criteria for this Learning Outcome. Site supervisors assessed the performance of five out of thirteen students as good overall and eight out of fourteen as excellent overall. No student received an overall rating of fair or unsatisfactory. Most History interns exceeded the university criteria for experiential learning in , continuing a pattern of academic excellence in Outcome Three carrying over from the academic year. This pattern of student performance demonstrates the strength of the History Program with respect to Outcome Three.

13 Program strengths and opportunities for improvement relative to assessment of outcome: The History Program has a reputation among students for being one of the more challenging programs. Perhaps because students do have to persevere through a challenging amount of reading and writing and many courses require oral communication and class presentations students do well when they apply their knowledge in the workplace. This year s employer evaluations and written work indicate that Marymount s History Majors as a whole excel at punctuality, collegiality, independent research, writing skills, and overall professionalism. Discuss planned curricular or program improvements for this year based on assessment of outcome: The data from indicates that the History Program has well prepared its Majors to thrive with experiential learning and to excel at applying their training and habits from undergraduate studies to professional objectives in a real world environment, in fulfillment of Outcome Three. There is not at present a plan for a curricular or program overhaul, nor do we think one is needed. Rubric for measuring Outcome 3: (Source: 2014 Career for Career Services Employer Evaluation Criteria) Exceeds Criteria (Excellent) Meets Criteria (Good/Fair) Fails to Meet Criteria (Unsatisfactory) Attendance and Punctuality Follows Directions Adheres to Deadlines Takes Initiative Without Supervision Oral Communication Written Communication Ability to Accept Criticism Applies Academic Knowledge to Workplace Overall Professionalism Overall Performance rating 8 5 0

14 Rubrics are included within this report. We did not use surveys in Please find attached to this report the syllabi for HI 420 Senior Seminar and HI 400 History Internship from the Spring 2014 semester.