Learning Assessment Examples Academic Year

Transcription

1 Learning Assessment Examples Academic Year College of Humanities and Fine Arts Art Studio Goal: The department expects all Art Studio students to develop technical skills for making art in one or more art disciplines. Link To Key Strategies: 1 Meeting the Educational Needs of Students and Community 2 Ensuring Student Success Student Learning Outcome: A visual arts student will be able to meet or exceed departmental expectations for demonstrating competency in the fundamentals of art making. Metric: Competency in the fundamentals of art making Source of Data: Rubrics used to assess portfolios Description of Assessment: Students will submit a portfolio of foundation level art (ARTS 103, 104, 111, 112) that is assessed by three art faculty through the use of a departmentally approved rubric in the required Sophomore Review, ARTS 297. The student portfolios must demonstrate competency in the fundamentals of art making. Analysis of Results: For academic year the department continued with an online/tablet assessment procedure for ARTS 297. By moving to a completely digital format we, as a department, have the ability to discern trends both positive and negative among our students, classes, and instructor strategies. The interim foundations coordinator also migrated the foundations vocabulary exam (not included in the Description of Assessment ), originally implemented for academic year , to an online format. Originally beta-tested in Fall 2013, we finally have a system that works for both students and faculty. The new digital procedure has evolved from a grading scale using a rubric with integers (0-5) to a system that divided the values (0,0.5,1,1.5,etc.) to more accurately assess our students' progress. For us, this means our best comparison for results, due to our grading scale adjustment, begins with Spring The foundation courses we evaluate include ARTS 103 (2-D Design), ARTS 104 (3-D Design), ARTS 111 (Drawing I), and ARTS 112 (Drawing II). Numerically, our overall assessment scores are up among all courses evaluated. Comparing the aggregate data from Spring 2014 to Spring 2015, a rise in these scores for all areas is noted. The analysis of our information shows that the aggregate data for Fall 2014 are lower than the data for Spring This could be due to the arrangement of faculty of who evaluated the students during the assessment. While our results are consistently up among courses evaluated our grades seem a bit inflated. The overall average score (out of a 5 point scale) for Fall 2014 is 3.70 and Spring 2015 is This averages at a score of 3.73, or a B. Historically our students have been weak, across all areas of evaluation, in preparatory works and conceptual development. This trend has continued in academic year While the scores are improving, as seen in our assessment, we still see room for improvement. As our numeric data has improved from previous years design and art studio majors have the same artistic foundation (craftsmanship, principles of art, same basic vocabulary, etc.), therefore both majors have the same basic assessment goals.

2 Use of Assessment Results for Continuous Improvement: Academic year has provided us with a more accurate and consistent data sample than previous years. As we continue with this method for data collection, we will be able to better determine the effectiveness of the alignment of our foundation courses, as well as the impact of a dedicated foundations coordinator (hired August 2015). The department continues to be thrilled with the new procedures and with the statistical data we receive from Institutional Research; it is helping our department review the progress of our students and manage the growth of the department. For continuity, our interim foundations coordinator worked with Institutional Research to migrate our ARTS 297 foundations vocabulary test to an online format. By moving this online, the exam can be directly sent to Institutional Research for inclusion in the aggregate data. This new testing process was supervised by the interim foundations coordinator and conducted in one of our computer labs. The online test was not completed by our student population offsite. The results of from the online test were positive overall with the majority of our students passing with a B. 2. College of Education Master of Arts in Teaching (M.A.T.)/Social Studies Goal: M.A.T. Social Studies candidates will be able to demonstrate the ability to plan instruction Link To Key Strategies: 2 Ensuring Student Success Student Learning Outcome: M.A.T. Social Studies candidates will create and implement a successful unit plan during the internship Metric: Ability to plan instruction Source of Data: EDSC 580 Teacher Work Sample Sections: 1-Contextual Factors, 2-Learning Goals, and 4-Design for Instruction Description of Assessment: The Teacher Work Sample (TWS) is a project that M.A.T. Social Studies candidates write and implement with grades 9-12 students during their internship semester; it contains seven teaching processes identified by research and best practice as fundamental to improving student learning. The TWS employs a range of strategies and builds on each student s strengths, needs, and prior experiences. Candidates are required to teach a comprehensive unit involving pre-planning that includes a description of contextual factors, learning goals based on IEP goals/objectives that are aligned with state or district content standards, the creation of an assessment plan designed to measure student performance before (pre-assessment), during (formative assessment) and after (postassessment) instruction, and a plan for instruction. After teaching the unit, candidates analyze student learning, and reflect upon and evaluate their teaching as related to student learning. Each teaching process is followed by a TWS Standard, the Task, a Prompt, and a Rubric (scoring guide) that defines various levels of performance on the standard. The TWS is a school-wide assessment adapted from the Renaissance Group's Teacher Work Sample favored by the Department of Education of the State of South Carolina as a means of determining a candidate s mastery of multiple elements of the teaching process. For the goal of demonstrating the ability to plan, the TWS project contains three elements: Contextual Factors, Learning Goals, and Instructional Design.

3 Section 1 - Contextual Factors The teacher uses information about the learning-teaching context and student individual differences to set learning goals and plan instruction and assessment: Knowledge of community, school, and classroom factors Knowledge of characteristics of students Knowledge of students varied approaches to learning Knowledge of students skills and prior learning Implications for instructional planning and assessment Section 2 - Learning Goals The teacher sets significant, challenging, varied and appropriate learning goals: Significance, Challenge and Variety Clarity Appropriateness for students Alignment with national, state or local standards Section 4 - Design for Instruction The teacher designs instruction for specific learning goals, student characteristics and needs, and learning contexts: Alignment with learning goals Accurate representation of content Lesson and unit structure Use of a variety of instruction, activities, assignments and resources Use of contextual information and data to select appropriate and relevant activities, assignments and resources Use of technology Candidates must pass their TWS with a combined grade of 85% in order to pass their internship. M.A.T. Social Studies program faculty expects a target score of 85% for each section of the TWS as well as for the overall scores. Analysis of Results: Data reported for the past two years indicates no serious concerns in any aspect of the TWS planning sections. In the data, there were only three recorded scores of not met for any of the TWS sections included in this assessment, and the average mean for all sections was a In other words, most candidates generally scored between a 2 and a 3 on all measures, and the majority of candidates excelled in each section. The numbers are even higher for the cohort. There was only one recorded score of a not met for any TWS section included in this assessment, and the average mean for all sections was a Scores for data ranged from (on two measures; Contextual Factors: Knowledge of students skills and prior learning and Learning Goals: Challenge, significance, and variety) to 3.00 (on Learning Goals: appropriateness for students). Relatively high for this cohort were measures pertaining to the content candidates teach: candidates earned an average of on Learning Goals: Alignment with Standards and a on Design for Instruction: Accurate Representation of Content. Scores for data ranged from 2.85 (on two measures; Contextual Factors: Knowledge of students skills and prior learning and Contextual Factors: Implications for

4 instructional planning and assessment) to a 3.00 on multiple measures, including the same measures listed above as especially pertaining to content: Learning Goals: Alignment with Standards and Design for Instruction: Accurate Representation of Content. Use of Assessment Results for Continuous Improvement: Data collected for the cohorts indicate candidates partially met or met expectations for most of the sections of the Teacher Work Sample related to planning. One section of the data (1.5) Contextual Factors: Implications for Instructional planning and Assessment reveals a weakness in that 8% of the candidates received a not met. This displays that candidates needed more practice with interpretation of the assessments and how they align with their instruction. The data shows an improvement with only 5% of the candidates falling in the not met category, and the met indicator rising from 62% to 90%. The same occurred in section 1.3 Knowledge of Characteristics of students where the students improved from one year to the next. In only 54% of the students received a met indicator and in % of the students received the met category. The two categories 1.5 and 1.3 are related in that candidates need to understand their students approaches to learning before they plan for instruction and assess for content. The increase in both of these scores indicates that the program successfully altered instruction in ways that helped students meet these expectations for planning. 3. College of Science Information Systems Goal: By graduation, information systems will be successfully adapting to technical, societal, and environmental changes by building upon strong foundational competencies and continuing lifelong learning in information systems or related areas. Link To Key Strategies: 1 Meeting the Educational Needs of Students and Community 2 Ensuring Student Success Student Learning Outcome: An ability to use current techniques, skills, and tools necessary for computing practice Computer Science. Metric: Embedded Question Source of Data: Courses, Senior Exit Exam Description of Assessment: Faculty Course Assessment Reports (FCAR) - Courses mapped to SLO I: CSCI-370, CSCI-375 [FA13], CSCI-409, CSCI-416[SP14]) Senior Exit Exam - Knowledge-based questions / Section I Senior Exit Exam - Survey-Based questions / Section I Analysis of Results: The most recent Student Outcome score measured was a 2.9, thus this Student Outcome scored slightly lower than the goal of 3.0. Because the goal was not met, the results show a slightly weak performance in student achievement for this Student Outcome. The chart below provides a visual of the overall progress of this Student Outcome over the last three years. Results over the last three years demonstrate stable performance. This demonstrates no movement (either positive or negative) in performance over time.

5 Strengths & Weaknesses: While performance is slightly weak, both strengths and weaknesses have been identified. Strengths: Course data, as reported in the FCARs, indicate that students are successful at this Student Outcome as four of five individual FCARs reported a 3.0 or higher for this Student Outcome. Weaknesses: Though the compiled FCAR score was a 3.2, the other two areas of triangulation measured (both sections of the Senior Exit Exam) showed that the goal was not met in either category. The score for the knowledge-based questions on the exam was a 2.3 and the score for the survey-based questions was a 2.7. The surveybased questions indicate that students are not confident in their ability to perform the skills measured by this Student Outcome. Details of the exam reveal that students have the lowest confidence (1.8 for both coverage of and the ability to perform) skills related to "Configure a server or device (or describe how to) to provide a set of required services with specified constraints." Server configuration is covered in two upper-level systems administration courses: CSCI 415 and CSCI 416, however both are electives. It is possible for an Information Systems student to not have taken either course by graduation. When the IS degree was created, systems administration was a required course. The Senior Exit Exam shows how few graduating students are choosing systems administration as an elective (the course is required in the IT degree program). Use of Assessment Results for Continuous Improvement: Incorporation of Data & Future Actions: Over the last three years, this Student Outcome has yet to reach the goal of 3.0. As part of our assessment cycle, the Information Systems Curriculum Committee will review the assessment scores and identify any areas where changes are necessary. An important consideration will be whether to continue to test students regarding systems analysis or replace the question with one that is more suitable to the general Information Systems population. Use of Prior Data: Continuous improvement strategies are in place, but changes have had neither a positive nor negative impact on this Student Outcome. The scores for all three years of assessment were a 2.9. While changes were made based on prior results, the two changes had no bearing on the single question with the lowest score on the Senior Exit

6 Exam during this cycle. While overall scores did not increase, the changes made to Question 27 (see below) to a more relevant question had a positive impact on the Senior Exit Exam (knowledge-based questions) with an improvement on Question 27 from a 0.3 to a 4.0. Keeping the exam questions up-to-date with various curriculum changes is a challenge in an evolving field. Changes Implemented Since the Last Review: Modify Senior Exit Exam (Academic Year 2013) - The Departmental Assessment Committee Modified Question 27 as the content was no longer included in the curriculum. Modify CSCI 490 (Academic Year 2013) - Modifications were made to the Advanced Software Engineering course (CSCI 490), which included the use of Leap Motion devices to expose students to current tools in computing. This course was offered with the new technology in Fall 2013.