PROGRAM LEVEL ASSESSMENTS FOR BIOLOGY

Transcription

1 PROGRAM LEVEL ASSESSMENTS FOR BIOLOGY The following is the list of the PROGRAM LEVEL OUTCOMES FOR BIOLOGY that has been assessed through AY thereby resulting in one complete cycle of assessment of all PLOs: (PLO #s 1a and 1b): developed the ability to communicate effectively through oral and written formats PLO #2: acquired an appreciation of the impact of science on society PLO #3: identified, explored, and developed a career and continuing education plan PLO #4: developed critical thinking skills and laboratory techniques so as to be capable of designing, carrying out, and interpreting scientific experiments PLO#5: developed a thorough understanding of and be able to describe the relationship between structure and function at the molecular, cellular, and organismal level Assessment Plan for PLOs in Biology: What student work will be assessed for each outcome? Outcome #1 Written Communication: One of the longer papers from NSC 2432, Scientific Writing, will be assessed. Outcome #1 Oral Communication: The oral presentations from BIO 2215, Genetics, BIO 2252, Cell & Molecular Biology, or BIO 2138, Environmental Toxicology, will be assessed. Outcome #2: One assignment focusing on the socio-economic, political, or ethical impact of scientific information/ experimental results will be assessed in BIO 2108, Contemporary Environmental Issues, BIO 2200, Developmental Biology, or BIO 2215, Genetics. Outcome #3: Career planning is discussed in advising sessions, in departmental seminars and workshops, and in a special topics career development course focused on health care careers. This learning outcome will be assessed through a survey to be taken by students, as well as through a rubric applied to a written assignment.

2 Outcome #4: To assess this learning outcome, we will be evaluating the process involved in experimentation (identifying the problem, choosing appropriate methodology, gathering and interpreting results) as well as the final presentation. Students in the Cell and Molecular Biology lab, BIO 2252L, those in Developmental Biology Lab, as well as the students in the Research course, BIO 2884, will be required to complete such a project. Outcome #5: Students will show their understanding of the relationship between structure and function at the molecular, cellular, and organismal level through various assignments in BIO 2210, Human Physiology, including exams and a paper. Who will assess the student work? The Department will conduct a day long meeting after finals in May for the purpose of assessing student progress on learning outcomes. All full-time faculty will be expected to attend and part- time faculty will be invited and encouraged to participate. All part-time faculty members involved in assessment will be expected to attend. How will you assess the student work? Rubrics have been developed for assessing written and oral communication and critical thinking through the University-wide Educational Effectiveness Task Force and the Assessment Council. We have assessed outcomes #2 and #5 with rubrics that were developed by the faculty in the Department. We will be using a simple 1 5 scale for each category. Outcome #3 will be assessed by using an online survey consisting of 5 to 7 questions as well as through a rubric applied to a written assignment.. How will your department determine whether students have performed well enough? How will you come to consensus about next steps? In our May, day-long meeting on assessment, we will discuss whether we are satisfied with the results. On the rubrics used for assessment, we will consider success to be a minimum average score of 3/5 over all of the categories. For learning outcome

3 #3, career planning, we will consider success to be 75 % of graduating seniors having thoroughly explored career options and having a plan. PLO ASSESSMENT RESULTS The full reports for each PLO may be found in Appendix: ASSESSMENT. The results are summarized here. PLO#1a: WRITEN COMMUNICATION AY 2009: An earlier assessment was done for Written communication (Spring 2009) and the report is attached below: Learning Outcome Assessed Students will communicate effectively through oral and written formats. Note that this assessment focused on written communication only. Method of Assessment An assignment from Writing for the Sciences course (NSC 2432; Spring 2009) was used to assess the written communication program level learning outcome. This course satisfies the upper division writing requirement for students in the sciences. It is our intention that most students who major in Biology or Biochemistry will take this course as juniors or seniors. Assignment Details Students were asked to write a page persuasive essay on a scientific topic of their choice. This assignment was chosen for program-level learning outcome assessment because it represents the type of writing and reading most of our graduates will encounter. Contentious issues such as stem cell research, global climate change, and the link between vaccines and autism (all topics chosen by students for this assignment) are usually debated in the public media and over the internet, often without much basis in the actual science being discussed. This assignment asked students to delve deeper into these issues, using some primary literature (scientific research) as evidence for their arguments. To prepare for this assignment, students were introduced to scientific persuasive essays, including reading and analyzing an essay of this type as well as building skills needed for

4 research methods, evaluation of sources, and mechanics of constructing an argument. Students were provided a handout detailing criteria used for assessment that followed the lecture introduction to this writing style. An in-class peer review session (instructorfacilitated) was held to provide students an opportunity to learn by commenting on a classmate's draft. Students also had one individual meeting with the instructor prior to submitting the assignment to discuss details of their own paper. Students were encouraged to visit the NDNU writing center for assistance. Students submitted their assignment and received instructor feedback via a rubric and extensive written comments. Students were then able to revise the paper based on that feedback. This revised assignment was used for the evaluation for this report. While the rubric used for this program-level learning outcomes was not the one used during the course for this assignment, they examine similar criteria. Assessment Details We used the Written Communication rubric adopted by the Educational Effectiveness Task Force (EETF) at NDNU to assess the written communication learning outcome. This assessment was performed by the course instructor at the end of the semester (separate from the in-course grading of this assignment). Seven students were enrolled in the course; six student papers were assessed. The seventh student paper was not available for assessment. The rubric provides for both quantitative data as well as qualitative data in the form of evaluator comments. Examples of weak, adequate, and exemplary work are available. Discussion by Program Faculty Natural Sciences faculty discussed this evaluation in a Department meeting and in conversations in May Areas of discussion are outlined below How to we deal with different standards of individual faculty for these assessments? What did we learn about students' ability to write effectively? Is student learning good enough? What are strengths and weaknesses? What changes can be implemented to improve student learning in this area? Report Authors Isabelle Haithcox, Ph.D., Chair, Dept. of Natural Sciences Melissa Book McAlexander, Ph.D., Dept of Natural Sciences.

5 Data Summary NSC 2432 Writing for the Sciences, Spring 2009 Assignment assessed: Revised Persuasive Essay Assessment instrument: NDNU EETF Written Communication Rubric Evaluation Criteria Student Scores n=6 Average score Low Score High Score Evaluator Comments Cogency Weakest students struggle to write coherent paragraphs with topic sentences/conclusions and a logical body. Middle scores reflect broader organizational weakness in the paper. Analytical Complexity Supporting Evidence Diction and Clarity Persuasive argument lends well to analytical complexity most students dug deep into the complexities Depth and complexity of evidence varied. Weakest writers skimmed the surface, while strongest writers used a variety of sources and synthesized arguments Struggles here go beyond the ideal of scientific language. Many struggle with diction in the non-scientific parts of their writing as well. Most were able to write in appropriate tone/style, but had difficulties with word choice and clarity. Mechanics Weakest students struggle with grammar basics (s/v agreement). Most have few errors in usage. Documentation Many students could not correctly apply documentation guidelines from Council of Scientific Editors (CSE) that were provided. The data from the PLO were not disaggregated based on the different programs within the department as this course was taken by both Biology and Biochemistry majors. In addition due to the small sample size this data was used for generating discussions within the department faculty as to the areas of writing that needed to be reinforced among our students.

6 PLO Outcome # 1: AY (Repeated) This PLO was reassessed in alignment with the ILO assessment done for all students enrolled in the General Microbiology Course (BIO2275) during the Fall semester of For results please see earlier section on ILO-1. PLO Outcome #1b ORAL COMMUNICATION AY The oral presentations from BIO 2215, Genetics, were assessed. BIO2215 (Eugenia Panomitros) Results from the assessments are shown below: Student ID Introduction Body Conclusion Language Voice Physical Presence Total Score The rubric used was the rubric that was used in all institutional assessment and this data needs to be analyzed and reports will be submitted shortly. Discussions will be held among all faculty regarding assessment results in the Summer of 2013 after the end of the semester.

7 PLO Outcome #3: Career planning is discussed in advising sessions, in departmental seminars and workshops, and in a special topics career development course focused on health care careers. This learning outcome will be assessed through a survey to be taken by students. CAR 2986 (Now BIO2886) (Lisa Bjerknes) Assessment Report for Program-Level Learning Outcome #3 in Biology 1. Learning Outcome Assessed Outcome #3: Career planning is discussed in advising sessions, in departmental seminars and workshops, and in a special topics career development course focused on health care careers. This learning outcome will be assessed through a survey to be taken by students and/or a rubric applied to a written assignment. 2. Describe Assessment Methods briefly but clearly. a. Method # 1 A rubric was applied to assess student papers about career mission and career strategic and action plans. 3. Describe assessment results e.g., means percentages, tables, graphs, inferential statistics (if possible), etc. a. Please report at least one set of quantitative scores, for use in normative, longitudinal and/or disaggregate comparisons. Fall 2011 SLO Outcome #3, Career Planning Assignment Scores Assignment: Create a career mission statement, strategic plan and action plan. Conclusion: With 79% of students submitting excellent Career Plans, I conclude that the majority of students have met SLO #3. Lack of participation seemed to be the overriding issue for the other students. Hence, this will be an area of focus next semester. Total Point Value of Assesstment: 10 Overall Student ID Mission Stmt Strategic Plan Action Plan Project Average

8 Class Average % of students scoring 93% or higher 79% 4. Conclusion: With 79% of students submitting excellent Career Plans, I conclude that the majority of students have met SLO #3. 5. Discuss program modifications suggested by results and faculty discussion. Lack of participation seemed to be the overriding issue for the other students. Hence, this will be an area of focus for the future. For example, I will have students turn in drafts of the assignment for comment at least a week prior to the final draft due date. This will be performed to improve student writing and participation in the assignment. Outcome #5: Students will show their understanding of the relationship between structure and function at the molecular, cellular, and at the organism level through

9 various assignments in BIO 2210, Human Physiology, including exams and a paper. Bio2210 (Lisa Bjerknes) PLO Outcome #4 Students will have developed critical thinking skills and laboratory techniques so as to be capable of designing, carrying out, and interpreting scientific experiments. The course used for this assessment: Bio2252L, the Cellular and Molecular Biology Lab offered in Spring of 2013 and assessment done by Monica GuhaMajumdar Student demographics: 5 Biochemistry majors and 5 Biology majors (all at the junior and senior level), along with 3 Post Baccalaureate Students. Background and Assignment: Throughout the entire 2013 Spring semester, students were taught various basic methodology and techniques that are associated with the current field of molecular biology. Each lab period consisted of learning a few basic techniques and then applying them towards certain specific applications/ real life problems. Once the projects were completed, students were required to turn in a written lab report with standard formatting that is observed in all scientific lab reports. This includes an Introduction, Materials and Methods, Results and finally a discussion section. The Reports are also expected to have citations from peer reviewed journals and textbooks that are properly referenced throughout the lab report. The final lab project was one of those that had a real life problem that needed to be solved using a compilation of techniques that were taught in the earlier sections of the class. The final lab research project included design and implementation using critical thinking. Assessment and Discussion: I used the NDNU Critical Thinking Rubric from the critical thinking ILO assessment to assess the final lab report written regarding the research projects. This rubric assesses five different criteria and a 5-point scale is used, with a score of 5 signifying highly developed, while a score of 1 signifies below standard. The total scores ranged from a high of 25 to a low of 6 out of a possible 25 points (100% to 24%). Overall, the students did well in determining the question to be answered and planning and implementing experiments to test the relevant question (Explanation of Issues). Where some of them ran into trouble was in dealing with unexpected results or problems and also data analysis using statistical and mathematical methods. The students with the lowest scores were those that did not know how to properly analyze and evaluate the data they collected.

10 The students with the highest scores were able to plan and run their experiments effectively, analyze their data thoroughly and draw conclusions well. It was also satisfying to see that 4 out of the 12 students had scores above 20 (greater that 80%), but unfortunately many of them were in the Biochemistry majors. Therefore it can be inferred that students in Biology major may need to be monitored more closely and taught the relevant background on data analysis and presentation of results. Although that was an important component of all of the projects throughout the semester, the students in most projects were working in groups and turning in their reports as a group so they did not necessarily all participate fully in the analysis. DATA FROM CRITICAL THINKING ASSESSMENT-PLO 4 Biology Student ID Explanat ion of issues Influenc e and Context Student' s Position Eviden ce Conclusi ons TOTAL SCORE Future Program Modifications and Faculty Discussions: Based on Faculty discussions with three of the four Full time Natural Sciences Faculty on May 2013, the following action Items were generated: a. Need to emphasize Critical Thinking early on in the same fashion in Labs in General Biology (Freshmen and Sophomores) and Developmental Biology Labs (Sophomores and Juniors) using the same rubrics in order for students to be acquainted with the expectations). b. Since this was a lab research report, we felt that we need to adjust certain aspects of the Rubric such that it is more applicable for lab research reports. Similar observations were also made regarding PLO4 from Biochemistry.

11 c. Finally it was decided that we need to coordinate with the Math Faculty to coordinate specific data analysis type problems to be incorporated into their syllabi as well. PLO Outcome #5 for Biology 1. Learning Outcome Assessed Outcome #5: Students will show their understanding of the relationship between structure and function at the molecular, cellular, and at the organism level through various assignments in BIO 2210, Human Physiology, including exams and a paper. 2. Describe Assessment Methods briefly but clearly. a. Method # 1 Multiple choice exam questions. 3. Describe assessment results e.g., means percentages, tables, graphs, inferential statistics (if possible), etc. Spring, 2012 Assessment Outcomes for SLO Outcome #5: Relationship between sturcture and function Conclusions: With 2/3rds of students answering all 3 questions correctly and 100% of students answering at least 2/3rds of the questions correctly, I conclude that a large majority of students have demonstrated an understanding of the relationship between structure and function of the assessed material. Assessment was exam IV questions 2, 21, and Structures that function to increase surface area along the GI tract include: 21. The enteric nervous system of the digestive system: 42. A kidney stone that blocks the flow of urine in the ureter will result in:

12 Student ID Q2 Q21 Q42 Overall Score out of 3 points % students with correct Q2 answer 86% % of students with correct Q21 answer 86% % of students with correct Q42 answer 95% % of students with all correct answers 67% % of students correctly answering at 2 of 3 questions 100% 4. Conclusions: With 2/3rds of students answering all 3 questions correctly and 100% of students answering at least 2/3rds of the questions correctly, I conclude that a large majority of students have demonstrated an understanding of the relationship between structure and function of the assessed material.

13 5. Discuss program modifications suggested by results and faculty discussion. I would like 100% of students to answer the questions correctly. To achieve this, I place more emphasis on providing multiple examples of the relationship between structure and function in both lecture and lab, as well as start the course with an overview of how structure and function relate with multiple examples. OVERALL CONCLUSIONS WITH REGARDS TO PLO ASSESSMENT The first cycle of assessments were completed in the AY 2013 and we are evaluating all our results through several departmental meetings to discuss the results as well as charter future plans of action with relation to student learning outcomes. In addition, our program faculty have already used assessment data to modify their curricula, and we have already begun conversations about how our curriculum maps to these learning outcomes and where coverage can be strengthened. We have also been exploring new ways to improve the assessment process such as creating assignments that perfectly align with rubrics as well as assess multiple courses for the same PLO so as to address issues such as inter rater reliability and comprehensive assessment of student learning. As we complete the first PLO cycle, it is the right time to consider revisions to these learning outcomes to help focus on things specific to the Biology program. Student Success: One of the ways to gauge student success is to track their career paths after they graduate from the University. Currently, this data is not readily available in our Student Alumni database and we have identified this as one of our immediate needs that will be addressed in the near future. For this process, we have started to combine all the data from different faculty who have had saved student and letters and had informally tracked the career paths of several of our students (PLEASE SEE FOLDER- ALUMNI LISTS). A brief look into the alumni lists show that some of our students have been successful in entering Medical Schools, dental Programs among the medical professions. In addition a few of our students have also gone to graduate school (M.S. and Ph.D. programs) while a few of them (more in the recent past) have opted for the M.S. in Clinical Sciences Program. Some of our students have chosen to work in Biotechnology and a few have also chosen careers in elementary and high school education. We are currently working to establish a brief alumni survey that will be sent to all our recent graduates as well as working toward establishing guidelines that will help us formally track our alumni so as to give us a better understanding of where our

14 students go after graduation. This would also allow us to incorporate changes in our curriculum based on student needs and also help them to face the challenging landscape of the job market. Based on the increased number of students in the past years who have shown an interest in applying to the M.S. program in Clinical Sciences Program, we are discussing the need to introduce a few more diverse electives that are required by the Program but are currently not offered in our curriculum (e.g., Hematology and Clinical Microbiology).