Assessment of Program Learning Outcomes at Boise State University

Transcription

1 Assessment of Program Learning Outcomes at Boise State University Assessment of student learning is central to teaching and learning at multiple levels: the individual courses that faculty teach, the compendium of courses and experiences that comprise academic programs, the Foundational Studies Program that reflects the general education, and the complementary experiences that students have outside the classroom. Assessment is a recursive tool, the primary aim of which is the improvement of teaching and learning at these many levels. Boise State University will assess student learning and experiences across the university, including University Learning Outcomes (ULOs), Program Learning Outcomes (PLOs), and outcomes of students co-curricular learning and experiences. The principles and process outlined in this document pertain specifically to PLOs, that is, the intended learning in students specific programs of study. Responsibility for PLO Assessment Responsibility for assessing student learning in the undergraduate and graduate academic programs lies within the departments/units (heretofore referred to as departments ) in which the programs reside. Assessment of learning is intended to be a formative, faculty-led process focused on the collection, interpretation, and use of information to guide curricular and instructional decisions, improve programs and policies, and ultimately contribute to student success through the achievement of intended learning outcomes. Assessment of student learning is part of Boise State s commitment to learning, a component of our ongoing regional accreditation (NWCCU), and a component of our ongoing planning and continuous improvement processes. The Office of the Provost is responsible for the administration and management of assessment planning, processes, and reporting to meet all of these needs. Key Principles Guiding PLO Assessment Assessment of PLOs at Boise State is informed by several key principles. 1. Assessment produces meaningful and actionable information that programs can use to improve teaching and student learning. 2. Assessment lives closest to the programs in which the learning occurs (i.e., it is a tool to be used by programs rather than an event/occurrence that happens to programs). 3. Assessment-based change is favored by a collaborative, collegial process in which the community of educators engages with evidence of student learning. 4. Assessment efforts are transparent and explicit rather than known only to insiders of the program or to the individual faculty members teaching a given course or set of courses. 5. Assessment reporting is frequent enough to ensure reasonable assurance of learning and continuous improvement yet not so frequent so as to detract from meaningful and actionoriented efforts. 6. Assessment is a regular, ongoing effort rather than an episodic event designed solely to satisfy reporting or external regulators. April

2 Measures Departments and programs will determine the measures best suited to assessing the PLOs. These measures must provide information that can be used to identify curricular and instructional strengths and weaknesses. Direct measures must be used and can be supplemented with indirect measures, such as student surveys, focus groups, etc. Process Departments are charged with development of meaningful and measurable PLOs, which are statements of intended learning focused on what students will be able to do at the conclusion of the program. They differ from objectives in that objectives typically are delivery-centered and may focus on what the instructor will do or what students will experience. At the inception of new programs, the programs will submit to the Office of the Provost a three-year assessment plan and be scheduled into the Periodic Review/Assessment Reporting Cycle (see AP Template, not yet attached). The plan should include the PLOs; a curriculum map illustrating how courses within the program align with these outcomes; and an implementation plan with a timeline identifying when and what will be assessed, how the department will gather assessment data, and how the department will use the information to make improvements. Every three years, departments will provide Program Assessment Reports (PARs) for each of its programs. The reports will include the following elements (see PAR template for specific details): 1. Mission of the program 2. Overview of assessment in the department/program 3. Brief description of curricular, instructional, or programmatic improvements/changes based on the previous assessment report 4. New or updated Curriculum Map 5. Intended PLOs 6. Measures used to assess outcomes 7. Interpretation/summary of key findings 8. Actions taken or planned based on findings 9. Strengths and improvements Reports will be reviewed by a small team of faculty and staff using a PAR Rubric (see Rubric for details). The Office of IR will provide guidance and calibration support to the review team. Feedback will be provided within two months and include comments and a comparative analysis of program and university-wide results generated from the PAR rubric review. Departments are asked to discuss the feedback and next steps and provide a brief follow-up report of the discussion and summary of actions to be taken. Results of the rubric review also will be shared with the appropriate college dean, the Provost, and the Assessment Coordinating Committee. The Assessment Coordinating Committee will annually review progress in assessment and include relevant information in their annual report about the state of assessment at Boise State. April

3 Program Assessment Report (PAR) Template Person completing this report: Date: 1. Mission: What is the mission of your program? (i.e., What does the program do that is unique from other units? What is the larger impact of the program? Who are its stakeholders? How does it align with the mission of the college and university?) 2. Overview: Describe how the department discusses, uses, and shares information about student learning outcomes achievement. (i.e., How does the assessment process work beyond individual courses? Who is involved? Is there a department committee? How do the department s faculty interact around this topic? How often? In what ways do you share findings of assessment within and outside the department with stakeholders?) 3. Reflection: Thinking back on the actions from the last report, what specific curricular, instructional, or programmatic changes have been implemented based on what was learned? Refer to action items and plans from the prior report, as appropriate. 4. New or Updated Curriculum Map: Attach a current curriculum map (see examples) and discuss how the map illustrates support for your mission and goals for student learning. Questions to consider: o Are students provided with multiple learning opportunities to develop the learning outcomes? o Are courses in the major sequenced in the developmental pattern to facilitate student achievement of the learning outcomes? o Do individual courses provide students with opportunities to integrate multiple learning outcomes? o How do learning experiences such as internships or service-learning courses support the development of the knowledge, skills, and dispositions identified as PLOs? o Are students provided with feedback on their progress in mastering the learning outcomes? o How well are the learning outcomes communicated to students in course syllabi? 5. Strengths in PLO assessment: Please describe what is going well in the assessment of this program. What are the highpoints or noteworthy accomplishments? (150 words max) Note: Responses to this question will not be rated with the rubric; they will provide information on the program s successes, be used to identify best practices, and assist in University accreditation reporting. 6. Improving PLO assessment: Please describe any challenges, gaps, or areas for improvement in the assessment of this program that you have already identified or begun to address. (150 words max) April

4 Table 1 Program Assessment Report List the Intended Program Learning Outcomes (one per row, typically 6-8 per program) Learner-centered statements that address: What should students know, be able to do, and value or appreciate as a result of completing the program? Measures Used to Assess Outcomes What evidence is used by the department/ program to determine whether the outcome has been achieved? Direct measure(s) such as portfolios, embedded assignments, lab reports, etc. Indirect measure(s) such as surveys, focus groups, etc. of students, alumni, employers, supervisors, etc. Informal method(s) such as faculty observations, informal reports, discussions, etc. Interpretation of Key Findings What have you discovered about student learning in each of the intended learning outcomes areas? EXAMPLE: EXAMPLE: EXAMPLE: EXAMPLE: 1. Actions Taken or Planned Based on Findings How have or will the findings be used by the faculty to make changes to the curriculum, specific courses, and/or to the pedagogy used in the program? Provide relevant examples. * NOTE: You will report on these action items in your next assessment report April

5 List the Intended Program Learning Outcomes (one per row, typically 6-8 per program) Learner-centered statements that address: What should students know, be able to do, and value or appreciate as a result of completing the program? Measures Used to Assess Outcomes What evidence is used by the department/ program to determine whether the outcome has been achieved? Direct measure(s) such as portfolios, embedded assignments, lab reports, etc. Indirect measure(s) such as surveys, focus groups, etc. of students, alumni, employers, supervisors, etc. Informal method(s) such as faculty observations, informal reports, discussions, etc. Interpretation of Key Findings What have you discovered about student learning in each of the intended learning outcomes areas? Actions Taken or Planned Based on Findings How have or will the findings be used by the faculty to make changes to the curriculum, specific courses, and/or to the pedagogy used in the program? Provide relevant examples. * NOTE: You will report on these action items in your next assessment report. 6. (add rows as necessary) April

6 Table 2 Rubric for Evaluating Program Assessment Reports 1. Program Intended Learning Outcomes * Learner-centered statements of what students will know, be able to do, and value or appreciate as a result of completing the program (e.g., students will [action verb] ) Rubric for Evaluating Program Assessment Reports Score Deficient (0) Beginning (1-3) Developing (4-6) Proficient (7-9) - No evidence of intended learning outcomes. 2. Measures - No evidence of measures used. 3. Key Findings - No findings or analysis presented. - Incomplete, overly detailed, disorganized, or not measurable. - Describe a process or delivery of education (i.e., what the instructor does for students) rather than intended student learning (i.e., what the intended result is to be). - Do not address the breadth of knowledge, skills, or services associated with the program. - Not all outcomes have associated measures. - Few or no direct measures used. - Methods are mismatched, inappropriate, or otherwise do not provide evidence linked to the intended learning outcomes. - Lack of connection between the outcomes, the data gathered, and results reported. - Observable and measurable. - Most outcomes are clearly defined or the meaning is easily discernable. - Language may be vague or need revision. - Most outcomes are written as learnercentered statements - Encompass the mission of the program and/or the central principles of the discipline. - Aligned with program, college, and university mission. - At least one measure per outcome. - Direct and indirect measures are used. - The methods used provide some evidence linked to the intended learning outcomes. - Some findings are reported that address outcomes and evaluate student achievement of them. - Observable and measurable. - Learner-centered statements that use action verbs. - Focus on the cumulative effect of the program of study. - Reasonable number adequate in number to meet mission while still manageable to assess. - Align with program, college, and university mission and goals - Align with professional standards, as appropriate. - Multiple measures for some or all outcomes. - Direct and indirect measures used; emphasis on direct. - Clear alignment between measures and the outcomes (i.e., the data gathered is aligned to the stated outcomes) - Feasible (uses existing strategies when possible); some measures apply to multiple outcomes. - Purposeful; clear how results could be used for program improvement - Described in sufficient detail. - Complete, concise and well-organized. - Aligned with achievement targets as appropriate. - Interpretation of degree to which desired outcomes meet achievement targets. - Compares new findings with past results. - Supporting documentation provided as appropriate. April

7 4. Actions Taken or Planned based on Findings * NOTE: You will report on these action items in your next assessment report. - No actions have been taken or planned. - Limited evidence that findings from assessment have been used to improve the curriculum, individual courses, pedagogy, etc. - No actions are documented or there are too many plans to reasonably manage. - Some evidence that findings from assessment have been used to improve the curriculum, individual courses, pedagogy, etc. - At least one concrete action has been documented or planned with relevant details, timelines, etc. - Multiple actions have been implemented and documented and/or detailed plans for implementation have been provided. - Action plans: clearly follow from assessment results and state directly which finding(s) motivated the action; identify area that needs to be monitored, remediated, or enhanced; define logical next steps; identify those responsible for implementation. - Number of action plans are manageable. April