Summary of Program Assessment Activities

Transcription

1 Summary of Program Assessment Activities Each degree-granting academic program has been actively engaged in the assessment of their program learning outcomes over the last year. This engagement began as part of the Smith Family Project, in which each of the - then programs designed assessment plans to be implemented in the AY. These plans culminated in the collection of data that would shed light on the degree to which students were achieving specified learning outcomes in the majors. These plans called for data to be collected on the spring 2006 graduates and analyzed during the fall 2006 semester. Conclusions and implications were to be generated that allowed the programs to use the results and findings from the assessment projects both for program review and program modification purposes. This report summarizes the projects undertaken by each academic program, highlighting the learning outcomes that were assessed, the data collected, the analysis of those data, and the conclusions and implications generated from the assessment projects to date. The individual reports generated by the programs may be viewed by clicking on the links in the Summary Table below or by clicking on the Program Title for each section of the report. Summary Table of Program Assessment Activities Art English Performing Arts Biology ER&RM Political Science Business History Psychology Chemistry Liberal Studies Sociology Computer Science Mathematics Spanish Education Nursing ART The Art program assessed the degree to which students can produce in-depth projects, work in specific media, and demonstrate competency in advanced artistic production. In assessing this outcome the Art program collected and evaluated a large amount of student work. This included: (a) photographic documentation of in-studio working processes and completed projects, (b) electronic documentation, interactive CD ROMs, and DVDs of completed multimedia, video and time-based art projects, (c) Art capstone projects, (d) annual Student Exhibitions and subsequent photo documentation thereof, (e) Art portfolios, and (f) an Art Student Exit Survey completed by graduating seniors in the program. From their analysis of the data, the Art program concluded that the learning outcome has been achieved at a level acceptable both to the faculty and students in the Art program. Through their student exit survey, they also found that some students experience some uncertainty concerning their preparation for employment. They have instituted an advising solution to this problem. Their conclusion is that Overall, the large amount of documentation collected clearly demonstrates proof that the CSUCI Art Program is highly successful in implementing the selected learning outcome: Students produce in-depth projects, working in specific media, demonstrating competency in advanced artistic production. Program Assessment Outcomes Page 1

2 Biology The Biology program assessed the degree to which students can describe life forms and their relationships to environment and ecosystems. This overarching goal was broken down into ten subgoals that were amenable to direct assessment by embedded assessment methods that placed 47 questions on the midterm and final exams in appropriate courses. Across all questions students answered 75% of the questions correctly. From this Biology concluded that the learning goals were met at an acceptable level. They further have planned to assess learning outcomes in a prepost fashion as well. Business The Business program chose to assess student learning outcomes in their capstone courses. The capstone courses is useful to the assessment process because it requires students to integrate all business core subjects (use critical thinking) and apply their individual and collective knowledge in a case study and simulation environment. The course also stresses communication and cooperation (working with others). As such, they developed a rubric to assess students written communication skills on both a group and individual basis. Based on these ratings, Business decided that there are discernable and noticeable analytical and writing deficits. To help Business students gain the writing skills required by the program, Business has developed a four-point plan and is in the process of implementing that plan. Chemistry Chemistry has developed a comprehensive five-step assessment plan of their learning outcomes. This year, the Chemistry program chose to assess the degree to which students are able to explain Geometric Structure as related to chemistry, and discriminate when it can be applied to a chemical problem. Using embedded assessment techniques, they seeded exams in six appropriate courses with different types of questions (multiple choice, short answer) and with assignments (literature review, student survey) designed to provide appropriate data for their evaluation and review. Based on the assessment data collected from students in six Chemistry courses, the Chemistry program determined that students met the outcome at a passing level. In all of the classes, at least 83% of the students demonstrated that they could partially meet the learning outcome. Further, based on this assessment cycle, Chemistry plans on changing their assessment strategy to make it more useful. Computer Science This year, the Computer Science program chose to assess the degree to which students demonstrate critical thinking and problem solving skills by identifying, evaluating, analyzing and presenting fundamental software solutions and their applications. They decided to assess the selected outcome by seeding final examinations in two key courses with questions selected from the AP Examination in Computer Science, GRE in Computer Science, and ETS Major Field Test in Computer Science. The data reveled that students was generally weaker than thy had expected. Consequently, Computer Science has begun to implement a course of action to improve student s performance in this area and in their assessment of student performance in this area. More specifically, they have decided to add a Capstone course in which all majors will be required to complete the project at CSUCI and, as a culminating experience, it will help us better assess the whole of the major. We expect to consult with other programs regarding tools for assessing capstone projects. Program Assessment Outcomes Page 2

3 English The English program has developed a systematic and comprehensive assessment plan of their program and student learning outcomes. This year, the English program chose to assess the degree to which students can examine texts, issues, or problems in the discipline from multiple perspectives (multicultural, interdisciplinary, international, experiential, theoretical and/or educational). English majors keep a portfolio of work produced in each of their required courses and electives. The students work closely with their advisors in developing the portfolio, which is reviewed by the capstone instructor to verify that requirements have been fulfilled for the major, and there is a review of the final portfolio by a committee of at least three English professors. Through the portfolio assessment, the English program concluded that students have achieved this learning outcome. Through the process, the faculty also has decided to modify the assessment strategy to make it more efficient and useful both for student and program assessment purposes. Finally, the English program has been involved in an on-going evaluation of the directed selfplacement program Environmental Science and Resource Management (ESRM) This year, the ESRM program assessed the degree to which students can collect, organize, analyze, interpret and present quantitative and qualitative data. Students enrolled in their capstone course complete an empirical independent project and create a poster summarizing their projects. This year, ESRM and ESRM-affiliated faculty independently scored student presentations (for a total of 38 assessments) at the end of the Spring 2006 semester. Both the printed poster and student responses to reviewers questions were considered in this evaluation and given an aggregate score from 1 (unacceptable) to 5 (excellent). Aggregate program assessments (grand mean=4.64, SD=0.45) averaged between Very Good (4) and Excellent (5). From this, ESRM concluded that students had achieved this learning outcome. Even so, ESRM determined that students greatest shortcomings identified for improvement were aspects of the final technical presentation. In response to these findings faculty have developed a five-point plan to modify the program in ways designed to improve students skills in this area. History This year, the History program assessed two learning outcomes: (1) communication skills in oral and written forms, and (2) skill in historical research, analysis, and presentation. These learning outcomes were assessed through the evaluation of student work in two courses, HIST 280 and HIST 491. HIST 280 The Historian s Craft is a lower division general education course required for history majors designed to instill an appreciation for the historical methodology, writing, and interpretation of the past. HIST 491 is the upper division course required of history majors that focuses on method and theory in the practice of history. Final projects for HIST 491 were collected in , and will be evaluated according to a rubric. The data from those evaluations will be compared to accumulated data consisting of final research projects completed in HIST 280, which will be assessed by comparable criteria. From these two sets of data and rubrics comparative analysis will be conducted in order to assess the academic development of students in the CSUCI History Program. History faculty are meeting this semester to do the final data analysis, draw conclusions, and use the data for program modification as appropriate. Program Assessment Outcomes Page 3

4 Liberal Studies Liberal Studies program efforts this year focused on the following program outcome: Liberal Studies graduates will be able to evaluate effectively oral or written communication for accuracy of content, logic of argument, and clarity of reasoning. Because all CSUCI freshmen must complete a freshman level class in critical thinking, the initial assessment and evaluation of critical thinking and reasoning skills was associated with this class. After a brief examination of the literature Liberal Studies elected to use the California Critical Thinking Skills Test. The test was administered to all students registered in UNIV 110 Critical Thinking in Interdisciplinary Contexts in Spring semester 2006 and Fall semester of 2005 using a pretest-posttest format. The essential finding from these assessments was that there was no significant difference in the students critical thinking and reasoning skills after the16 week critical thinking class. Based on a methodological considerations, Liberal Studies concluded that findings were insufficient to warrant changes in the critical thinking class. However, they feel they have sufficient data to create the mechanisms needed to institutionalize feedback loops in assessment and evaluation programs. The Liberal Studies Implementation Plan for the Assessment of Student Learning Outcomes (Exhibit) proposes that five members of the Liberal Studies Advisory Committee (LSAC) meet for one or two days in the intersession between Fall and Spring semesters to evaluate annual assessment data, and reflect on what they have learned. The product from this meeting will be a series of recommendations directed to the entire LSAC during the course of the Spring semester. This will provide a regular mechanism that ensures that assessment and evaluation data guide decision making in pedagogy and programs. Mathematics For the academic year, the Mathematics Program selected three learning outcomes to assess: (1) demonstrate knowledge of some of the current applications of mathematics in the sciences, industry, and/or education; (2) demonstrate communication skills by expressing mathematical ideas in oral and written form; and, (3) demonstrate a sense of exploration that enables one to pursue lifelong learning. To assess the degree to which students met these three outcomes, Mathematics developed a rubric for evaluating student presentations, distributed them to the entire CSUCI mathematics faculty (including part-time instructors), and solicited evaluations from them for student presentations. These presentations were made at (1) the Mathematics Colloquium in which Math seniors are required to make a presentation, and on June 11, 2006, and the CSU Student Research Competition, held on May 5, 2006, at CSUCI. The presentations were scored along a number of dimensions relevant to the student learning outcome. Statistical tests provided strong evidence that each of the three learning outcomes was met. However, Math also decided that further depth should be sought from student projects. In the future, further depth will be solicited from student projects in Math 492 and 499. Nursing None submitted. Nursing is a new program for the AY. Performing Arts Performing Arts is a new program at CSUCI. The program has not yet determined a primary outcome to assess this year; however, it has determined a number of items to investigate during the coming year to get a better grasp of the program s success and to make the program stronger. Program Assessment Outcomes Page 4

5 These include, but are not limited to a student survey to determine levels of interest and progress within the degree, and direct assessment of the quality of student work from internships and/or performances. Political Science Like Performing Arts, Political Science is a new program at CSUCI, having enrolled its first students in fall For the AY, the Political Science program selected the following learning outcomes to assess: (1) analyze political and policy problems and formulate policy options, and (2) deliver thoughtful and well articulated presentations of research findings. Students in the Capstone course during the spring semester will produce a research project based on a real world policy problem impacting the community. Students will analyze the problem using techniques of social science research, formulate policy options and proposed solutions, and present their findings in the form of a policy memo and a poster. The posters will be assessed based on a rubric designed to measure the two learning outcomes selected for this year. The assessment rubric, which is currently under development, will be modeled after successful rubrics used to measure policy analysis skills as well as the quality of presentation of research findings. The rubric will be pilot tested using the five posters produced by students in first year of the major. During the second year of the major, when a larger cohort will be ready to complete the capstone course, data will be collected for purposes of assessing the identified outcomes, and making program and course modifications based on our findings. Psychology The Psychology program adheres closely to the Learning Objectives and Outcomes outlined in Goals and Objectives for the Undergraduate Psychology Major: Recommendations from a Meeting of California State University Psychology Faculty and Undergraduate Psychology Major Learning Goals and Outcomes: A Report from the American Psychological Association. This year, the program assessed whether students understand and can use major research methods in psychology, including design, data analysis and interpretation. The culminating experience and assignment for the two-semester research methods sequence (PSY 300 Psychological Research and Statistical Methods with Lab I, 3 units; PSY 301 Psychological Research and Statistical Methods with Lab II, 3 units) is a poster presentation of students original empirical research. This poster presentation mirrors those held at both regional (e.g., Western Psychological Association) and national (e.g., American Psychological Association) conferences. The posters present students original, empirical research conducted as a requirement for passing PSY 301. These posters provide a rich source of data that to assess the degree to which Psychology students have met this learning outcome. Psychology faculty independently judged 54 student posters at the annual Psychology Program Poster Presentation using a validated rubric designed especially for rating psychology posters. The results indicate that Psychology students have met the program goal and objective. However, the findings indicated that there was more variation in the quality of the posters than desired. The Psychology program instituted an informal working group meeting weekly to discuss the issue and formulate curricular changes to the research methods courses to help ensure that all Psychology students develop the highest possible set of competencies and skills in conducting, analyzing and interpreting research. Sociology A new major at CSUCI, the Sociology program The Sociology program follows many of the Learning Objectives and Outcomes outlined in Creating an Effective Assessment Plan for the Program Assessment Outcomes Page 5

6 Sociology Major by the American Sociological Association Task Force on Assessing the Undergraduate Sociology Major. For the AY, Sociology plans to assess student understanding of the role of evidence in the social sciences and how to conduct quantitative and/or qualitative sociological research. To do so, Sociology plans to look at specific measurable outcomes via the Capstone project in Sociology. The Capstone is designed to be a culminating experience in the major. Students enroll in Sociology 499 having completed all major substantive and methodological coursework in the major. That is, they will have completed the statistics and quantitative research methods sequence (SOC 303 and 310); Sociological Theory (SOC 420); important substantive courses in major areas of the field, such as stratification (SOC 350), race and ethnicity (SOC 360), and political sociology (SOC 330). The Capstone is designed to demonstrate that the student is capable of completing original research on her own, with appropriate guidance. Using a rubric, Sociology will assess student work and determine how well the learning objective has been met. This year s work will be an important assessment tool for the first group of students completing the research methods sequence. Based on how well students perform across the dimensions here, Sociology may determine to lengthen the research methods course from one to two semesters, bringing the total number of skills-based semesters to three. Since the Sociology program opened to students in fall, 2005, they are clearly in building mode. Assessing what they have gives them an opportunity to avoid pitfalls in the early years of the program. Spanish The Spanish program selected to assess the degree to which students perform at an intermediatehigh to advanced level of language proficiency in speaking and writing in the Spanish language as defined the American Council on the Teaching of Foreign Languages (ACTFL) Guidelines. To do so, data were collected in two courses during the Spring 2006 semester: in SPAN 302 Advanced Spanish: Part Two and in SPAN 499 Capstone in Spanish. SPAN 302 counts as a required course toward the major, and SPAN 499 is required of students completing their Spanish program in May or in December of the same year. Two populations of students were identified: heritage speakers (for whom Spanish is spoken at home by at least one adult family member), and non-heritage speakers (those who have learned Spanish as a second language). These populations were identified because they often experience different challenges. The typical heritage speaker may have difficulty with reading and writing, for example, yet may exhibit strong listening and speaking skills. For speaking, oral proficiency interviews were conducted by a Spanish faculty member with individual students, and these interviews were digitally recorded. The students interviewed were: 6 students, 2 heritage and 4 non-heritage speakers, from SPAN 302; and 4 students, all heritage speakers, from SPAN 499. For writing, essay questions administered on a SPAN 302 in-class exam were collected, a total of 5 students, 1 heritage and 4 non-heritages speakers; and capstone research papers from SPAN 499 were collected from a total of 4 students, 3 heritage and 1 non-heritage speakers. The data suggest an increase in level among the heritage group. No conclusions can be drawn from the non-heritage speaker group since no data are available from the capstone project. This is due to the fact that this was the first Capstone course offered by the Spanish program, and a speaking sample from the one non-heritage speaker was not collected. Non-heritage speakers in SPAN 302 will eventually take the Capstone, at which point comparable results will be available. The ACTFL Guidelines for writing were used to rate the writing samples. The data, also suggest that there is an overall increase in both the heritage and non-heritage groups in speaking from SPAN 302 to the Capstone. However, this suggested increase is based on a small sample of students. Consequently, no conclusions were drawn from the existing data, as the number of students used was small, suggesting findings are not generalizable. In order to draw conclusions and to suggest implications for program modification, the Spanish program needs to collect and analyze more data related to speaking and writing Program Assessment Outcomes Page 6

7 proficiency. The Spanish program will collect more data on speaking and writing proficiency over the course of the academic year, as it continues to identify and develop other means to assess the program s other student learning outcomes. Program Assessment Outcomes Page 7