Learning Assurance Report. for the. Biotechnology Degree Program. in the. Department of Biological and Physical Sciences

Transcription

1 Learning Assurance Report for the Biotechnology Degree Program in the Department of Biological and Physical Sciences School of Sciences and Mathematics March 2006 Prepared by Dorothy Don Davis

2 Table of Contents for Learning Assurance Report* Biotechnology Degree Program Matrices of General and Specific Learning Outcomes 3-5 Articulating Student Learning Outcomes 6,7 Linking Outcomes to Program Requirements includes a list of all requirements, electives, and their prerequisites Articulating Expected, Hypothesized and Actual Findings for the Evidence Articulating the Plan and Time Table for Collection of AOL Using Findings of Assurance of Learning for Quality Enhancement 26 Appendix 1 Graduate Exit Survey Appendix 2---Embedded Exam Questions Appendix 3---Embedded Laboratory Reports

3 LEARNING OUTCOME I for BIOTECHNOLOGY EXPECTED RESULTS The Major will: A S S E S S M E N T M E T H O D S Laboratory/internship notebooks & reports Laboratory practical proficiency Major papers written Individual and group presentations Student exams, quizzes Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as appropriate supporting areas of mathematics, and computer technology Explain the major components, processes and mechanisms of cellular and molecular biology and genetics BIOL2107, BIOL3300, BIOL3340 Discuss the impact of gene modification on agricultural, environmental, and medical sciences Apply commonly used techniques in contemporary biotechnology at the cellular and molecular levels BIOL2101, BIOL3300, BIOL3340 Interrelate principles from cellular and molecular areas to biotechnology BIOL3340 Propose and use appropriate mathematical and chemicalphysical principles BIOL2107, BIOL3340 BIOL3340 BIOL3340 BIOL3340 BIOL3340 BIOL3380, BIOL4486 BIOL3380, BIOL4486 BTEC3400, BIOL4486 BIOL4486 BIOL4486 BTEC3400, BIOL4486 BIOL2107, BIOL3300, BIOL3340, BIOL3380, BTEC3301 BIOL3340 BIOL3380, BTEC3301 BIOL2101, BIOL2107, BIOL3300, BIOL3340, BTEC3301, BTEC3400 Apply appropriate computer software for statistical analyses, database mining, and data presentation. BIOL2101 Math 2107 Cite appropriate regulations associated with biotechnology industry and laboratories BTEC3400 BIOL3380 BTEC3400 BTEC3400, BIOL4486 BIOL3340, BTEC3301, BTEC3400 BIOL4486 BIOL4486 BTEC3400 BIOL2107 BIOL3380, BIOL3340 BIOL2101, BTEC3301 BTEC3301, BTEC3400 3

4 LEARNING OUTCOME II for BIOTECHNOLOGY Students should be able to conduct and interpret a good scientific investigation. SKILLS EXPECTED RESULTS The Major will: A S S E S S M E N T M E T H O D S Laboratory/internship notebooks & reports Laboratory practical proficiency Major papers written Individual and group presentations Student exams, quizzes Design a scientific investigation BIOL2107, BIOL3300, BIOL3340 Select and apply reference sources Employ and interpret probabilities and statistics BIOL2101, BIOL3300, BIOL3340, Math 2107 Formulate and evaluate scientific arguments BIOL3340 Synthesize and present scientific information BIOL2107, BIOL3340 BIOL3340 BIOL3340 BIOL3340 BIOL3340 BIOL3380, BIOL4486 BIOL3380, BIOL4486 BTEC3400, BIOL4486 BIOL4486 BIOL4486 BTEC3400, BIOL4486 BIOL2107, BIOL3300, BIOL3340, BIOL3380, BTEC3301 BIOL3340 BIOL3380, BTEC3301 BIOL2101, BIOL2107, BIOL3300, BIOL3340, BTEC3301, BTEC3400 Apply principles from cellular and molecular areas to biotechnology BIOL2101 Cite appropriate regulations BTEC3400 BIOL3380 BTEC3400 BTEC3400, BIOL4486 BIOL3340, BTEC3301, BTEC3400 BIOL4486 BIOL4486 BTEC3400 BIOL2107 BIOL3380, BIOL3340 BIOL2101, BTEC3301 BTEC3301, BTEC3400 4

5 5

6 LEARNING OUTCOME III for BIOTECHNOLOGY ATTITUDES Graduates should be able to assess and choose value systems to guide them in the practice of science and to critique ethical concerns raised by science and technological advances, recognizing that science is a human endeavor that influences and is influenced by the social context in which it is practiced. EXPECTED RESULTS The Major will: A S S E S S M E N T Laboratory/internship Notebooks & Reports Laboratory practical proficiency Major papers written Recognize that scientific ideas change over time BIOL3340 Collaborate in problem solving BIOL3340 BIOL3380 BTEC3301 Rely exclusively on empirical data when dealing with scientific issues BIOL3340 BIOL3340 BIOL3340 BIOL3380 BIOL3380 BTEC3400 Judge the impact of biotechnology on agricultural, environmental, and medical sciences by applying scientific knowledge BIOL3340 BTEC3400 BIOL3340 BTEC3400 Identify and employ major arguments relevant to ethical issues arising from our understanding of living systems and the practice of biotechnology BIOL2107 BIOL2107 BIOL3340 BTEC3400 BTEC3400 Confirm the importance of quality control and quality assurance in biotechnology BTEC3400 BTEC3400 BIOL4486 BIOL4486 BIOL4486 BIOL4486 BIOL4486 M E T H O D Individual and group presentations BIOL4486 BIOL4486 BTEC3400 BIOL4486 BTEC3400 BIOL4486 BTEC3301, BTEC3400, BIOL4486 BTEC3301 BTEC3400 6

7 Student Learning Outcomes for Biotechnology I) Learning Outcome (Knowledge) Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as a broad familiarity with appropriate supporting areas of science, mathematics, and computer technology. A. Explain the major components, processes and mechanisms of cellular and molecular biology and genetics. (K) B. Examine and discuss the impact of gene modification on agricultural, environmental, and medical sciences. (K) C. Choose and apply appropriate methods and techniques used in contemporary biotechnology at the cellular and molecular levels. (K) (S) D. Interrelate principles from cellular and molecular areas to biotechnology. (K) E. Propose and use appropriate mathematical and chemical-physical principles and their correct applications. (K) (S) F. Apply appropriate computer software for statistical analyses, database mining, and data presentation. (K) (S) G. Cite appropriate regulations associated with biotechnology industry and laboratories. (K) II) Learning Outcome (Skills) Graduates should be able to demonstrate an operational understanding of science and how it may be used to explain natural phenomena. Students should recognize science as an epistemological method and be able to conduct and interpret a good scientific investigation. A. Design a scientific investigation that includes identification of variables and formulation of a testable hypothesis. (S) B. Select and apply reference sources when researching what is known about a problem. (K) (S) C. Employ and interpret probabilities and statistics in the gathering and analysis of data. (S) D. Formulate and evaluate scientific arguments. (S) (A) E. Synthesize and present scientific information. (S) F. Apply principles from cellular and molecular areas to biotechnology. (S) G. Utilize appropriate mathematical skills and chemical-physical principles. (S) H. Employ computer software for statistical analyses, database mining, and data presentation. (S) I. Perform and interpret commonly used methods and techniques in quality control and quality assurance. (S) 7

8 III) Learning Outcome (Attitudes) -- Graduates should be able to demonstrate an appreciation of science as a human endeavor that influences and is influenced by the social context in which it is practiced. They should assess and choose value systems to guide them in the practice of science and to critique ethical concerns raised by science and technological advances. A. Recognize that scientific ideas change over time. (A) B. Collaborate in problem solving. (A) C. Rely exclusively on empirical data when dealing with scientific issues. (A) D. Judge the impact of biotechnology on agricultural, environmental, and medical sciences based on scientific knowledge. (A) (K) E. Identify and employ the major arguments relevant to ethical issues that arise from our understanding of living systems and the practice of biotechnology. (A) (K) F. Confirm the importance of quality control and quality assurance in biotechnology. (A) 8

9 SEMESTER REQUIREMENTS and Prerequisites for Bachelor of Science in Biotechnology Course Number and Title Prerequisite Course Number and Title GENERAL EDUCATION COURSES (directly related to major) MATH 1113 Precalculus MATH 0098 (if required) Intermediate Algebra (higher math may be substituted) MATH 1113 Precalculus MATH 1190 Calculus I CHEM 1211/1211L General Chemistry I and High school chemistry or CHEM 1151 Survey of Chemistry Lab CHEM 1212/1212L General Chemistry II and CHEM1211/CHEM1211L Lab LOWER DIVISION MAJOR REQUIREMENTS BIOL 2101 Intro Culture & Methods Biology BIOL 2107 Biological Principles I BIOL 2108 Biological Principles II MATH2107 Biostatistics BIOL2101 Biological Principles I or CHEM 1211/1211L General chemistry 1 and lab BIOL 2107 Biological Principles I MATH 1101 Mathematical Modeling or MATH 1113 Precalculus PHYS 1111/1111L Introductory Physics or MATH 1113 Precalculus or PHYS 2211/2211 Lab Calculus-based Physics I MATH 1190 Calculus I & Lab UPPER DIVISION MAJOR REQUIREMENTS CHEMISTRY CHEM 3361/3361Lab Organic Chemistry I & CHEM 1211/1211L General Chemistry I and Lab Lab CHEM 3362/3362Lab Organic Chemistry II & CHEM 3361/3361Lab Organic Chemistry I & Lab Lab BIOLOGY BIOL 3300 Genetics BIOL 2108 Biological Principles II or CHEM 1212/1212L General Chemistry II and Lab 9

10 BIOL 3340/3340 Lab Microbiology & Lab BTEC 3301 Intro to Biotechnology BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II BIOL 3300 Genetics Course Number and Title Prerequisite Course Number and Title BIOL 4486 Bioethics BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II MAJOR TRACKS (Students Choose One) A. General Biotechnology Track CHEM 2800/2800L Quantitative Analyt Chem CHEM 3501/3501L Biochem I BIOL 4410 Cell & Molecular Biology BIOL 4431/443L Human Physiology or BIOL 4420/4420L Plant Physiology BTEC Biotechnology Lab Modules Electives: Biology 3327 Medical Genetics Biology 4460 Medical Microbiology Biology 4465 Immunology Biology 4475 Virology or appropriate Seminar, Special Topics or Advanced Topics Courses CHEM 1212/1212L General Chemistry II and Lab CHEM 3362/3362L Organic Chemistry II & Lab BIOL 3300 Genetics BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II and CHEM 3361/3361Lab Organic Chemistry I & Lab BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II BIOL2101 Biological Principles I and BTEC 3301 Intro to Biotechnology BIOL 3300 Genetics BIOL 3340/3340 Lab Microbiology & Lab BIOL 3300 Genetics BIOL 3300 Genetics variable B. CYTOGENETIC TECHNOLOGY TRACK Biology 3327/3327Lab Medical Genetics BIOL 3300 Genetics 10

11 Biology 4431/4431 Lab Human Physiology & Lab BTEC Biotechnology Lab Modules Electives in Biotechnology OR BIOL 3398 Cytogenetics Lab Electives: CHEM 3501/3501L Biochemistry 1 BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II and CHEM 3361/3361Lab Organic Chemistry I & Lab BIOL2101 Biological Principles I and BTEC 3301 Intro to Biotechnology Special review and approval of major area review committee CHEM 3362/3362L Organic Chemistry II & Lab Course Number and Title Prerequisite Course Number and Title BIOL 4410 or appropriate Seminar, Special Topics or Advanced Topics courses BIOL 3300 Genetics and CHEM 3361/3361Lab Organic Chemistry I & Lab C. MOLECULAR DIAGNOSTICS TRACK BIOL 3300 Genetics BIOL 3327 Medical Genetics BIOL4431/4431L Human Physiology & Lab BTEC Biotechnology Lab Modules BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II and CHEM 3361/3361Lab Organic Chemistry I & Lab BIOL2101 Biological Principles I and BTEC 3301 Intro to Biotechnology Special review and approval of major area review committee Electives in Biotechnology OR Biology 3398 Electives: CHEM 3501/3501L Biochemistry I Biology 4460 Medical Microbiology Biology 4465 Immunology Biology 4475 Virology or appropriate Seminar, Special Topics or Special Topics or Advanced Topics Courses CHEM 3362/3362L Organic Chemistry II & Lab BIOL 3340/3340 Lab Microbiology & Lab BIOL 3300 Genetics BIOL 3300 Genetics variable D. FORENSIC MOLECULAR BIOLOGY TRACK CRJU 1101 Foundations of Criminal Justice 11

12 CRJU 3320 Criminal Investigation CRJU 1101 Foundations of Criminal Justice CHEM 2800/2800L Quant. Analyt Chem Biology 4431/4431L Human Physiology & Lab BTEC Biotechnology Lab Modules BTEC 4455 Case Studies in Forensic Science Electives from select group CHEM 1212/1212L General Chemistry II and Lab BIOL 2107 Biological Principles I and BIOL 2108 Biological Principles II and CHEM 3361/3361Lab Organic Chemistry I & Lab BIOL2101 Biological Principles I and BTEC 3301 Intro to Biotechnology BIOL 3300 Genetics Variable 12

13 SECTION I- ARTICULATING STUDENT LEARNING OUTCOMES BIOTECHNOLOGY Department of Biological and Physical Sciences School of Sciences and Mathematics I. Overall Summary of the Strength of the Student Learning Outcomes Rating and Response: Exemplary Analysis of the Learning Outcomes plan gives evidence that the strength of student learning outcomes is exemplary. These outcomes are based on the outcomes for the biology department that have been identified and refined over the course of many years with input from the entire faculty. These outcomes are designed to identify knowledge, skill and attitudes in the courses taught. Specific outcomes are clearly delineated from, but related to, the general learning outcomes. These outcomes are evidenced throughout courses that build upon each other in using knowledge, skill and attitudes and progressing from lower-level to higher-level thinking both within the course and within the sequence of courses. Evidence for this exemplary rating is that the biotechnology majors from our program with a concentration in cytogenetics have a 95% pass rate over the past 5 years on the National Credentialing Agency for Laboratory Personnel, Inc. required exam for cytogenetic technologists, exceeding the national average pass percentage for the first time candidates in cytogenetics of 79.5 (part I) and 93 (part II). IA. Knowledge/Skill/Attitude Balance and Student Centered Rating and Response: Exemplary Student learning outcomes have been identified for the Biotechnology degree and include knowledge, skill and attitudinal facets. This is evidenced by the learning outcomes charts, with a separate chart for each type of outcome (Knowledge, Skills, Attitudes) that includes the general outcomes as well as the specific outcomes. These outcomes focus on student achievement rather than course content. The outcomes require the active involvement and participation of the student in the learning process at all levels identified knowledge, skill, and attitudes, and are thus student centered. For example, the broadest of the knowledge based outcomes is: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as a broad familiarity with appropriate supporting areas of science, mathematics, and computer technology. A specific outcome within this category is: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics. The broadest of the skills based outcomes is: Graduates should be able to demonstrate an operational understanding of science and how it may be used to explain natural phenomena. A specific outcome is: Design a scientific investigation that includes identification of variables and formulation of a testable hypothesis. The broadest of the attitudinal based outcomes is: Graduates should be able to demonstrate an appreciation of science as a human endeavor that influences and is influenced by the social context in which it is practiced. They should have developed value systems to guide them in the practice of science and an awareness of ethical concerns raised by science and technological advances. A general outcome identified within this category is: Judge the impact of biotechnology on agricultural, environmental, and medical sciences based on scientific knowledge.

14 IB. SLO Differentiation Rating and Response: Exemplary The specific student learning outcomes are focused and measure the general student learning outcomes to which they are associated. They are measurable, are focused on and closely related to the general learning outcomes with which they are associated, and able to be assessed. For example, the general attitudinal learning outcome is: Graduates should be able to demonstrate an appreciation of science as a human endeavor that influences and is influenced by the social context in which it is practiced. They should have developed value systems to guide them in the practice of science and an awareness of ethical concerns raised by science and technological advances. Some specific learning outcomes directly related to the general learning outcome are: collaborate in problem solving,: make judgments about the impact of biotechnology on agricultural, environmental, and medical sciences based on scientific information: confirm the importance of quality control and quality assurance in biotechnology. These may be assessed by means of: Laboratory/internship notebooks and reports, individual and group presentations, directed study or team research. IC. Compliance with Disciplinary Conventions Rating and Response: Very good The National Accrediting Agency for Clinical Laboratory Sciences has accredited our program at Kennesaw State University as one of only three accredited institutions in the United States offering a program in Cytogenetics Technology. Graduates earn a Bachelor of Science (BS) degree in Biotechnology with a concentration in cytogenetics and are eligible to sit for the NCA (National Credentialing Agency) exam in clinical cytogenetics and earn certification as clinical laboratory specialists in cytogenetics [CLSp(CG)]. All learning outcomes were based upon the suggested national requirements. As previously stated, the biotechnology majors from our program with a concentration in cytogenetics have a 95% pass rate over the past 5 years on the National Credentialing Agency for Laboratory Personnel, Inc. required exam for cytogenetic technologists, exceeding the national average pass percentage for the first time candidates in cytogenetics of 79.5 (part I) and 93 (part II). Because the Biotechnology degree program is a new program, we have developed and will continue to develop our program and our learning outcomes with the object of meeting the requirements so that our students can become accredited in other areas in addition to cytogenetics. This is a work in progress. ID. Building Upon Prior Learning Rating and Response: Exemplary All of the general learning outcomes expand upon the general education program s five goals to: 1)develop productive habits of mind, 2)develop effective communication skills, 3)expand knowledge and understanding, 4)expand creative capabilities, and 5)exhibit understanding of the impact of ethical and aesthetic values. In the knowledge category: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as a broad familiarity with appropriate supporting areas of science, mathematics, and 14

15 computer technology; in the skills category: Graduates should be able to demonstrate an operational understanding of science and how it may be used to explain natural phenomena. Students should understand science as an epistemological method and be able to conduct and interpret a good scientific investigation; and in the attitudes category: Graduates should be able to demonstrate an appreciation of science as a human endeavor that influences and is influenced by the social context in which it is practiced. They should have developed value systems to guide them in the practice of science and an awareness of ethical concerns raised by science and technological advances. IE. Lower-Order and Higher-Order Thinking Rating and Response: Exemplary Many of the specific learning outcomes focus on higher-order thinking. For example: two specific outcomes within the knowledge category are: Be able to inter-relate principles from cellular and molecular areas to biotechnology, and understand the impact of gene modification on agricultural, environmental, and medical sciences. Within the skills category two specific outcomes that evince higher-order thinking are: Use and interpret probabilities and statistics in the gathering and analysis of data; and Interrelate principles from cellular and molecular areas to biotechnology. Within the attitudes category: Utilize scientific information in making judgments about the impact of biotechnology on agricultural, environmental, and medical sciences, and identify the major arguments relevant to ethical issues that arise from our understanding of living systems and the practice of biotechnology. This category requires students to integrate biological knowledge with social values to appreciate and understand diverse viewpoints. SECTION II: CONNECTING OUTCOMES TO THE PROGRAM REQUIREMENTS II. Overall Strength in Connecting Outcomes to Program Requirements Rating and Response: Exemplary As evidenced by the attached matrix sheets, all lower level and upper level required courses as well as elective courses are linked to both general and specific learning outcomes in Knowledge, Skills, and Attitudes. IIA. Linkage of All Program Requirements to SLOs Rating and Response: Exemplary All courses (program requirements) are linked to specific student learning outcomes. The matrix sheets linking program requirements to specific student learning outcomes provides this evidence. IIB. Reinforcing SLO Attainment through Multiple Program Requirements Rating and Response: Exemplary As shown in the attached matrices for each required course, all of the specific student learning outcomes are linked to multiple program requirements. Additionally the elective block of courses is linked to specific student learning outcomes. All courses in that block of elective courses are linked to these specific student learning outcomes. IID. Core Upper Division (or Graduate) Course Requirements Rating and Response: Exemplary 15

16 The upper division course requirements ( BIOL 3300 Genetics, BIOL 3340/3340L Microbiology & Lab, BTEC 3301 Intro to Biotechnology, BIOL3380 Evolution, BIOL 4486 Bioethics, BTECH 3400 Quality Control/Quality Assurance) are linked to multiple specific student learning outcomes, as evidenced by the attached charts for each. IIE. Elective Upper Division (or Graduate) Requirements of the Program Rating and Response: Exemplary All upper division program requirements in which there are course options (See extensive attached list of requisite and prerequisite courses for the program) are linked to multiple specific student learning outcomes. IIF. Building Upon General Education (or the Baccalaureate Education) Outcomes Rating and Response: Exemplary There are specific student learning outcomes in all three areas: Knowledge, Skills, Attitudes, that build on the general education experience. In the knowledge category: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as a broad familiarity with appropriate supporting areas of science, mathematics, and computer technology. Here a specific course has recently been added in response to assessment -- a jointly taught between Mathematics and Biology course in Biostatistics (Math 2101) in the skills category: Graduates should be able to demonstrate an operational understanding of science and how it may be used to explain natural phenomena. Students should understand science as an epistemological method and be able to conduct and interpret a good scientific investigation; and in the attitudes category: Graduates should be able to demonstrate an appreciation of science as a human endeavor that influences and is influenced by the social context in which it is practiced. They should have developed value systems to guide them in the practice of science and an awareness of ethical concerns raised by science and technological advances IIG. Required Practicums & Internships Rating and Response: Exemplary There is no required capstone experience for the biotechnology program. However graduates from the biotechnology program with a concentration in cytogenetics do have a required internship of one semester of work at a registered cytogenetics laboratory. These students have a 95% pass rate over the past 5 years on the National Credentialing Agency for Laboratory Personnel, Inc. required exam for cytogenetic technologists, exceeding the national average pass percentage for the first time candidates in cytogenetics of 79.5 (part I) and 93 (part II). IIH Capstone Experience Rating: N/A Response: The Program does not have a specifically defined Capstone Experience. The biotechnology degree program lacks a common capstone course or exit exam, so assessment efforts will be focused on the last common courses taken by biotechnology majors. These courses are BIOL 3300 Genetics and BIOL 3380 Evolution. This approach was chosen after discussions with the AOL consultation team and evaluation of the available options. We will begin by evaluating one General Student Learning Outcome and the Specific Student Learning Outcomes within it that are appropriate for BIOL 3300 Genetics in Fall BIOL 3300 and BIOL 3380 will then be evaluated in alternating Fall semesters. 16

17 SECTION III: LINKING STUDENT LEARNING OUTCOMES TO METHODS THAT COLLECT EVIDENCE OF AOL General SLO: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as appropriate supporting areas of mathematics, and computer technology. Specific SLOs: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics [K]. Apply commonly used techniques in contemporary biotechnology at the cellular and molecular levels [K,S]. Identify all of the methods that collect evidence of AOL to be used and the instruments to be employed in those methods to determine student achievement on this SSLO. Three methods will be employed: (a.) A questionnaire for graduating seniors that asks them to evaluate the program s attainment of the three GLOs and associated SLOs. Students will rate the program s success for each on a Likert-scale and provide written responses to open-ended questions concerning the degree program. The survey may be provided in written form or as an online form, depending on the results of preliminary tests in Spring (b.) 10 multiple-choice exam questions will be embedded within the course. They could be 10 questions on a comprehensive final or scattered throughout the course on different exams. The questions will be developed by the course instructor in consultation with the AOL Biology coordinator. (c.) Two embedded course assignments (e.g., lab reports) will be evaluated for 20% of the students in the course, selected at random. A rubric for scoring the assignments will be developed and each submission evaluated by the instructor, the Biology AOL coordinator, and the Biotechnology AOL coordinator. Comment on the strengths and weaknesses of the methods and instruments on focus, reliability/validity, information type, timeliness/cost, student motivation, and varied nature. (a.) Measurement Focus: The embedded multiple choice questions will predominantly measure lower-level thinking processes while the embedded written assignments will address predominantly higher-order thinking. The written assignments will enable assessment of the skill component of SLO #2. Both will be appropriate for assessing the two SLOs being evaluated. 17

18 (b.) Reliability and Validity: While the chosen methods are not as reliable or valid as a standardized exam (which is difficult to implement in this degree program see below), they are nonetheless strong. As the embedded assignments contribute towards the course grade, student motivation to perform well on them will be high. Student motivation to complete the graduate survey will undoubtedly be lower, although we are investigating different methods of distribution and collection to maximize student response rate. The reliability of the scoring of the embedded written assignments will be improved by the use of a standard rubric and multiple evaluators. (c.) Type of Information Provided by the Method that Collects Evidence of Assurance of Learning: Direct measurements include the embedded test questions and writing assignments. An indirect measure is the graduate survey. The evaluation plan has a mix of these two types of information. The exam questions and Likert-scale rating questions on the graduate survey will provide quantitative data. Qualitative data will be provided by the embedded written assignments and the open-ended questions on the graduate survey. (d.) Timeliness and Cost: Costs will not be a major factor. The only costs that may be incurred are those associated with providing return postage for graduate surveys, if it is decided to provide them in written form. Time issues were carefully considered when planning assessment methods given existing strains on faculty time. Embedded assignments were utilized, as their evaluation can simultaneously be used for course evaluation and AOL program evaluation. Evaluation of the multiple choice questions can be easily accomplished with the Biology and Physics Departmental scanner and associated computer software. The time commitments associated with the embedded written assignments have been reduced by subsampling within the course assignments and within the students in the class. Further, by having the AOL coordinators for Biology and Biotechnology coordinate their efforts, we are able to simultaneously evaluate both degree programs in these two common courses. The use of the coordinators of these two AOL initiatives as additional evaluators also reduces the need for using additional faculty members as outside evaluators. (e.) Student Motivation: As stated above, student motivation on the embedded assignments will be high as they contribute towards the final course grade. Motivation on the graduate survey will be lower, but maximized based on preliminary work examining student response rates and attitudes about different methods of survey presentation. (f.) Use of Multiple Methods that Collect Evidence of Assurance of Learning: The assessment methods employed are diverse in the variety of criteria evaluated. See the individual sections above for further information. 18

19 SECTION IV: ARTICULATING EXPECTED AND HYPOTHESIZED FINDINGS FOR THE EVIDENCE General SLO: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as appropriate supporting areas of mathematics, and computer technology. Specific SLOs: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics [K]. Apply mathematical skills and chemical-physical principles when appropriate [K,S]. List all of the methods to be used and the instruments to be employed to collect evidence of assurance of learning for this (these) specific outcome(s). See the response for Linking Student Learning Outcomes to Methods that Collect Evidence of AOL for a listing of the three methods to be employed in assessing the Biology degree program. List the expected or hypothesized findings for the evidence to be collected for each of those methods and instruments. This portion of the assessment program of the Biology degree program was easily the most difficult. If we were able to secure a common exit exam for all of our majors that took into account their varying specialties, this would be an option. We do not currently require an exit exam of our students. While exit exams for the various specialties within Biology are available (e.g., medical school entrance exams, biology GRE) there is not a common exam that would be appropriate for all Biology majors from the degree program. While my research has failed to turn up such an exam at this point, I will be continuing my search to see if such a possibility exists. If a suitable exam could be secured, the faculty in the program would need to determine if such an exam was appropriate, how to administer it, the consequences of passage/failure, the associated costs and how to pay them, and a host of other logistical issues. Such discussions would take considerable time and this is therefore not an option for immediate implementation. If instituted, such an exam would provide: criterion-referenced standards for comparison (if such were provided or could be secured); norm-referenced comparisons to students at public institutions in Georgia, at institutions in the southeast, and the national average; best-practice comparisons with top institutions; and value-added comparisons that contrast scores of entering freshmen with graduating seniors. If an exam was deemed valuable but an appropriate instrument could not be secured at the national level, a locally-developed exam could be developed. While this would not provide criterion-referenced, norm-referenced, or best-practice standards, it would allow for value-added comparisons within the program. Discussions within the faculty of the degree program will need to occur if this possibility in investigated further. Given this situation, we will proceed by examining standardized test options and limit ourselves in the short term to longitudinal standards of comparisons in each of the two assessed courses (Genetics and Evolution). The scores on the embedded test questions and assignments will be compared across years, as will student responses on the graduate exit survey. This will allow us to determine if our students are improving in their attainment of the desired objectives over time. Accordingly, we will work to ensure that the exam questions and embedded writing assignments are similar (if not identical) each time the course is evaluated and administered in a similar 19

20 manner (all on comprehensive final or on different exams). We will also use the same rubric for evaluation in each year. This will allow us make reasonable comparisons over time. SECTION V: ARTICULATING THE PLAN AND TIMETABLE FOR COLLECTING EVIDENCE OF AOL General SLO: Graduates should be able to demonstrate a broad knowledge of the central themes, principles, and technologies or biology, as well as a broad familiarity with appropriate supporting areas of science, mathematics, and computer technology. Specific SLOs: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics [K]. Apply commonly used techniques in contemporary biotechnology at the cellular and molecular levels [K,S]. List all of the methods to be used and instruments to be employed to collect evidence of assurance of learning for this specific outcome. Three methods will be employed: a.a questionnaire for graduating seniors that asks them to evaluate the program s attainment of the three GLOs and associated SLOs. Students will rate the program s success for each on a Likert-scale and provide written responses to open-ended questions concerning the degree program. The survey may be provided in written form or as an online form, depending on the results of preliminary tests in Spring b. 10 multiple-choice exam questions will be embedded within the course. They could be 10 questions on a comprehensive final or scattered throughout the course on different exams. The questions will be developed by the course instructor in consultation with the AOL Biology coordinator. c. Two embedded course assignments (e.g., lab reports) will be evaluated for 20% of the students in the course, selected at random. A rubric for scoring the assignments will be developed and each submission evaluated by the instructor, the Biology AOL coordinator, and the Biotechnology AOL coordinator. Identify when, who, where, and how evidence will be collected for the methods and instruments used for this specific outcome. When will assurance of learning evidence be collected? As stated previously, BIOL 3300 (Genetics) and BIOL 3380 (Evolution) will be assessed in alternating years, beginning in Fall The graduate exit survey will be developed and pilottested in Spring 2005, and fully implemented in Fall In each evaluation of the course, additional general and specific learning outcomes will be assessed. In Fall 2007 (the second evaluation of BIOL 3300), assessment of the specific learning outcomes in GLO #1 will be incorporated, and in Fall 2009 (the third evaluation of BIOL 3300) the specific learning outcomes in GLO #3 will be included. The same pattern will be used for evaluations of BIOL Who will collect assurance of learning evidence? 20

21 The AOL initiatives for the Biology degree program will be directed by a single AOL coordinator. The coordinator will work with the instructors of evaluated courses on all assessment efforts and organize the distribution and collection of the graduate exit surveys. This scheme will provide centralized responsibility but ensure a diversity of ideas on evaluation by bringing in two to three additional faculty members with assessment experience in a consultation role. Course releases, funded by the AOL committee at the University level, will be necessary to see that the work is completed in a timely, comprehensive manner given the heavy existing teaching and research commitments for faculty in our Department. The coordinator for the Biology degree program will work closely with the responsible parties for the Biotechnology degree program to overlap assessment initiatives whenever possible to reduce the impacts of the AOL process on faculty workload. Where will assurance of learning evidence be collected? Exam questions and written assignments will be evaluated within the assessed courses by the course instructor and the coordinators of the Biology and Biotechnology programs. Students will have high motivation on the embedded assignments as they will contribute to the course grade, and student focus groups and research on survey distribution (marketers and demographers would seemingly have a wealth of information on such topics) will attempt to determine the delivery method for graduate exit surveys that result in the greatest response. How will assurance of learning evidence be collected? Instrument: The embedded exam questions and the rubric for scoring embedded writing assignments will be developed in Spring The graduate exit survey will build upon an existing graduate survey and be constructed and pilot-tested in Spring Efforts will be taken, using supplied resources, to see that the materials developed optimize validity and reliability. Participants: The population to be sampled will be the students in the assessed courses and graduating seniors. The students in the courses will be sub-sampled as described above. All of the graduating seniors will receive a survey to maximize the potential number of returns. Procedure: Care will be taken to develop testing conditions in the assessed courses that will be similar across semesters to allow for longitudinal comparisons. As the embedded assignments are part of the normal course work, special instructions to students are not necessary. The procedures for distributing and collecting the graduate exit surveys will be determined in Spring Modifications may or may not be made to procedures based on the results of this testing. Training: There are no assessment methods that require special training. Pilot Testing: As mentioned previously, extensive pilot-testing will be involved in the development of the graduate exit survey in Spring As the embedded assignments are not unusual or novel course elements, extensive pilot testing is not necessary. Nonetheless, we will critically evaluate the embedded exam questions, the rubric used to score the embedded written assignments, and the level of subsampling during our first assessment of BIOL 3300 in Fall Any needed changes could be identified at this point, modified in Spring 2006, and instituted the following Fall semester. SECTION VI: COLLECTING, ANALYZING, AND INTERPRETING EVIDENCE FOR AOL General SLO: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as appropriate supporting areas of mathematics, and computer technology. 21

22 Specific SLOs: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics [K]. Apply commonly used techniques in contemporary biotechnology at the cellular and molecular levels [K,S]. List all of the methods to be used and instruments to be employed to collect evidence of assurance of learning for this specific outcome. Three methods will be employed: a.a questionnaire for graduating seniors that asks them to evaluate the program s attainment of the three GLOs and associated SLOs. Students will rate the program s success for each on a Likert-scale and provide written responses to open-ended questions concerning the degree program. The survey may be provided in written form or as an online form, depending on the results of preliminary tests in Spring b. 10 multiple-choice exam questions will be embedded within the course. They could be 10 questions on a comprehensive final or scattered throughout the course on different exams. The questions will be developed by the course instructor in consultation with the AOL Biology coordinator. c. Two embedded course assignments (e.g., lab reports) will be evaluated for 20% of the students in the course, selected at random. A rubric for scoring the assignments will be developed and each submission evaluated by the instructor, the Biology AOL coordinator, and the Biotechnology AOL coordinator. Comment on the evidence collection, evaluation, and interpretation processes that occurred for the methods and instruments used for this specific outcome. Data: Questionnaire for Graduating Seniors Distribution and Return Rates Survey data was gathered for both the Spring and Fall semesters of Surveys were given to faculty members for distribution to students in their courses in the last few weeks of the regular semester or mailed to listed addresses for students without courses in biology in the semester in question. Return rates for surveys were 30% for Spring 2005 and 34% for Fall These values are slightly lower than traditional rates of return for graduate surveys in the biology program, and indicate additional efforts are needed to increase the rate of return. In the next round of assessments, the surveys will be distributed earlier in the student s final semester to avoid potential overlaps with the increased workloads associated with the end of classes and hopefully boost returns. Results and Analysis A copy of the survey sheet and the raw data and written responses for these surveys are provided in Appendix I. To interpret the results of the questionnaire quantitatively, a response of Strongly Agree was assigned a value of 1.0, Agree a value of 2.0, and Disagree a value of 3.0. The mean response for each survey question was calculated and is presented in Figure 1. 22

23 Figure 1. AOL Graduate Survey Results Spring 2005 and Fall Mean Value Spr05 F Question # Some trends were apparent despite the limited data. Students largely feel that the program is meeting its stated objectives. The mean scores for all of the questions fell between 1.0 and 2.0 (between Strongly Agree and Agree ) and only 5 of 34 points fell above a value of 1.5. Ratings of Disagree were exceedingly rare only one in Spring 2005 and two in Fall There was substantial temporal variation to student responses, with questions scoring substantially different in the two semesters. Such variations are expected given the small sample sizes of these two survey administrations and should moderate over time as additional data is gathered. Several objectives received consistently positive marks (low values), such as Find and use reference sources when researching what is known about a problem, Make and evaluate scientific arguments, Appreciate the dynamic nature of scientific knowledge, and Adopt a collaborative approach to problem solving. Some areas of close attention are questions that scored lower, such as Use and interpret probabilities and statistics in the gathering and analysis of data, Utilize appropriate computer software for statistical analyses and data presentation, and Apply mathematical skills and chemical-physical principles when appropriate. The common mathematical nature of these objectives indicates the need for greater attention to statistical and mathematical principles in program courses, and close monitoring of future graduate surveys for similar trends. Data: Embedded Course Questions Methods Ten common exam questions were embedded in the exams of each section of Biol 3300 in Fall The number of students answering each question correctly was calculated for each section and the mean of the two sections calculated. The questions were developed and agreed upon by the two instructors of Biol 3300 in Fall 2005 and will be used in AOL assessments of this course in future years. Student performance on these questions will be evaluated longitudinally to track trends. 23

24 Results and Analysis The percentage of students correctly answering each question in each section and the overall mean are provided below. The ten questions are provided in Appendix 2, along with a listing of the SLOs addressed by each question. Table 1. Percentage of students responding correctly to embedded questions. Question Instructor X Instructor Y Mean Key points: The mean success rate for the ten questions was 77%, with a range of 53 93%. This mean value, along with the scores for each question, will be compared longitudinally to track student progress. Several questions showed considerable variation in success between the two sections. Examples include #5, 6, 7, and 8. Performance levels were not consistent in the two sections, as the section that scored high on one question scored lower on others. These results suggest areas where improvements can be made in the individual sections to better standardize course content between sections. The exam questions dealing with mathematical principles (#6-8) had mean success rates similar to the other questions but showed considerable variation between sections. This again suggests avenues for individual instructors to stress mathematical principles and increase student performance in this area. 24

25 The concepts addressed in low-scoring questions like #5 and 7 provide opportunities for minor course modifications to improve student performance in these areas. Data: Embedded Laboratory Assignments Methods Student laboratory reports for two assignments were sub-sampled and 20% of the class was independently evaluated by the course instructor, Biology AOL coordinator, and Biotech AOL coordinator using a common rubric. The rubric was developed by the faculty teaching Biol 3300 in Fall The rubric was provided to all parties well in advance of its use and training was not deemed necessary, as both AOL coordinators had used similar rubrics when evaluating laboratory assignments in their own classes. Student performance on these assignments will be compared longitudinally to track student progress. Results and Analysis The mean percentage for each assignment for each section is listed below. The individual scores for each sub-sampled student are provided in Appendix 3. Table 2. Mean student scores (percentage) by laboratory assignment and rater. Laboratory Instructor Biology AOL Biotech AOL Maize Instructor X Maize Instructor Y Fruit Fly Instructor X Fruit Fly Instructor Y Key points: The mean 25

26 SECTION VI: USING FINDINGS OF ASSURANCE OF LEARNING FOR QUALITY ENHANCEMENT General SLO: Graduates should be able to demonstrate a broad knowledge of the central themes, principles and technologies of biology as well as appropriate supporting areas of mathematics, and computer technology. Specific SLOs: Explain the major components, processes and mechanisms of cellular and molecular biology and genetics [K]. Apply commonly used techniques in contemporary biotechnology at the cellular and molecular levels [K,S]. When the expected findings were not found in the evidence for each of the methods and instruments employed for this specific outcome, identify changes planned to reach the expected student performance level. From graduate exit surveys there was a student perception of more help needed with mathematics, statistics, and interpretation of results. Analysis of results from exam questions seemed to back this up. In response, the Department of Biological and Physical Sciences in collaboration with the Mathematics Department is now offering a required course -- Biostatistics for all biology majors. Also, we have recently instituted a required course for all Biology majors, BIOL2101, a course that introduces basic biological principles and techniques, as well as applied mathematical principles and techniques as a way of reinforcing basic knowledge needed for success in the major and for increasing student retention. 26

27 APPENDIX 1: GRADUATE EXIT SURVEY Graduate Exit Survey Department of Biological and Physical Sciences Revised Spring 2005 As a soon-to-be graduate of the Degree Program in Biology at Kennesaw State University, the faculty of the Department of Biological and Physical Sciences would like to ask you to please provide us with feedback to help guide future efforts in the program. The survey takes only a few minutes to complete and we would greatly appreciate you taking the time to assist us. A scantron sheet has been included along with this question sheet for all responses. Please enter the following pieces of information on that sheet for the listed field: NAME (Last, First, M.I.) Please enter and bubble in your name in the space provided. If you wish to remain anonymous, feel free to leave this section blank. BIRTHDATE Do not enter your birthday. Please enter the month and year (last two digits) of your graduation from the program in the MO. and YR. fields. Leave the DAY field blank. Survey Questions: The degree program in Biology aims to provide graduates with knowledge, skills, and attitudes that will enrich their growth as scientists and prepare them for employment or continued education. The program is guided by three broad objectives, each with a set of related specific objectives. For each of the specific objectives, please state how well you felt the program was in fulfilling these objectives using the scale below. Please bubble in the appropriate letter from the scale below for each of the questions. Enter your response on the scantron sheet according to the number next to each objective. (a.) Strongly Stated objective clearly met, no improvement needed. agree (b.) Agree Stated objective met, but improvements could be made. (c.) Disagree Stated objective not met, improvements are necessary. Broad Objective #1: Graduates should be able to demonstrate an operational understanding of science and how it may be used to explain natural phenomena. Students should understand science as an epistemological method and be able to conduct and interpret a good scientific investigation. (1.) Design a scientific investigation which includes identification of variables and formulation of a testable hypothesis (2.) Find and use reference sources when researching what is known about a problem (3.) Use and interpret probabilities and statistics in the gathering and analysis of data (4.) Make and evaluate scientific arguments (5.) Synthesize and present scientific information (6.) Utilize empirical data when dealing with science issues 27