Expanding Assessment of Analy3cal Skills among Biology Majors: From Introductory labs to Upper Division Elec3ves

Transcription

1 Expanding Assessment of Analy3cal Skills among Biology Majors: From Introductory labs to Upper Division Elec3ves Presented by Kathleen McAuley PI: Serena Moseman- Val3erra, Ph.D. Department of Biological Sciences

2 Who we are Kathleen McAuley Undergraduate Assistant URI 2012, B.S. Secondary Educa3on and B.A. Biology Sydney Bridge Climb, Spring 2011 Serena Moseman- Val3erra New Assistant Prof. in Biological Sciences Instruc3ng BIO 102 in Spring 2013 Very sorry to miss the conference! A Team photo from the Na3onal Academies Summer Ins3tute, June Faculty, Instructors, and Administrators from Boston College, Yale, Ohio State, U Wisconsin- Madison

3 General Goals I. To iden3fy common learning objec3ves for BIO 101 (Introductory Biology, semester 1) II. To establish an approach for assessing data analysis skills at both introductory and upper- levels. BIO 101 laboratory sec3ons BIO 262 (Ecology)

4 I. Overview of Ac3vi3es Fall 2011 For Common BIO 101 Learning Objec3ves: Mee3ng(S) with four BIO 101 Instructors Introduced idea of a formal, common assessment Collected recent final exams from each instructor (or midterms from new instructors) Analyzed prior final exams to iden3fy common learning goals: - Categorized each exam ques3on in terms of a learning objec3ve - Compared each ques3on to an extensive list of possible objec3ves from a colleague in the Undergraduate Educa3on in Biology Scien3fic Teaching Summer Ins3tute - Tallied ques3ons in each category - Iden3fied areas of overlap among the 4 instructors Found 6 common objec3ves! (in next slide)

5 Learning Objec3ves in BIO 101 Assessment (Fall 2011) 1) Dis3nguish between a living and non- living organism 2) Predict direc3on of water movement in diffusion across a membrane, given solute concentra3ons on either side 3) Iden3fy similari3es and differences of prokaryo3c and eukaryo3c cells (either classifying examples or iden3fying in terms of cell proper3es) 4) Iden3fy the macromolecules that make up cells and the rela3ve abundance of those components in a cell 5) Recognize the steps involved in an electron transport chain 6) Dis3nguish between the outcomes of mitosis and meiosis How were these used? 1. List was shared with instructors, feedback solicited 2. For Fall 2011 finals: Example ques3ons for each of the 6 learning objec3ves were wrieen and offered for instructors (most instructors wrote their own ques3ons) **Enabled a post- assessment of student performance in each sec3on of BIO 101. (Built into the Final Exam).

6 Common Objec7ve Number 100 Post- Assessment for 6 Common Objec7ves Average Percent Correct Higbie Instructor 1 Wetherbee Instructor 2 Van Instructor Sciver 3 Cary Instructor Test 4 1 Test A Instructor 4 Test B Cary Test

7 100 Student performance shown by objec3ve (Instructor 1) 90 % correct Objec3ve 1 Objec3ve 2 Objec3ve 3 Objec3ve 4 Objec3ve 5 Objec3ve

8 Student Common performance Objec7ve shown Percents by Wetherbee objec3ve (Instructor 2) Objec3ve 1 Objec3ve 2 Objec3ve 3 Objec3ve 4 Objec3ve 5 Objec3ve

9 Student performance shown by objec3ve Common Objec7ve (Instructor Percents 3) Van Sciver Objec3ve 1 Objec3ve 2 Objec3ve 3 Objec3ve 4 Objec3ve 5 Objec3ve

10 Student performance shown by objec3ve Common Objec7ve Percents Cary Test 1 (Instructor 4, Test A) Objec3ve 1 Objec3ve 2 Objec3ve 3 Objec3ve 4 Objec3ve 5 Objec3ve

11 100 Student Common performance Objec7ve shown Percents by objec3ve Cary Test 2 (Instructor 4, Test A) Objec3ve 1 Objec3ve 2 Objec3ve 3 Objec3ve 4 Objec3ve 5 Objec3ve

12 Total number of student responses Instructor 1: 73 Instructor 2: 115 Instructor 3: 71 Instructor 4 (Test 1): 54 Instructor 4 (Test 2): 55

13 What these data suggest: Students generally weakest with objec3ves 1*,3*,and 5 (*Scores < 50% except for 1 of the 4 instructors) Students strongest with objec3ve 6 (Scores close to, > 70%) High variability between instructors 1) Dis3nguish between a living and non- living organism 2) Predict direc3on of water movement in diffusion across a membrane, given solute concentra3ons on either side 3) Iden3fy similari3es and differences of prokaryo3c and eukaryo3c cells (either classifying examples or iden3fying in terms of cell proper3es) 4) Iden3fy the macromolecules that make up cells and the rela3ve abundance of those components in a cell 5) Recognize the steps involved in an electron transport chain 6) Dis3nguish between the outcomes of mitosis and meiosis

14 Other observa3ons, sugges3ons 1. Instructors described minimal communica3on about the course and scheduling a common mee3ng 3me (for 4 instructors) was impossible. - Is a mee3ng possible before school starts? 2. All instructors independently suggested that the amount of content in BIO 101 was too large (some specified this in comparison to BIO 102) Priori3es are needed for content, coordinate with 102 instructors. (Work in progress)

15 Bloom's Levels of Understanding: Exams target levels 1 and 2 6. Evalua3on: think cri3cally about and defend a posi3on Judge, Jus)fy, Defend, Cri)cize, Evaluate 5. Synthesis: transform, combine ideas to create something new Develop, Create, Propose, Design, Invent 4. Analysis: break down concepts into parts Compare, Contrast, Dis)nguish 3. Applica3on: apply comprehension to unfamiliar situa3ons Apply, Use, Compute, Solve, Predict 2. Comprehension: demonstrate understanding of ideas, concepts Explain, Summarize, Interpret, Describe, Diagram 1. Factual Knowledge: remember and recall factual informa3on Define, List, State, Name, Cite - Ok for introductory course? (But are students bored? prepared? Challenged?) Ideas: - Freeman text indicates Blooms levels for each ques3on. - Pre- lecture quizzes online (Mastering Biology) can address low Bloom s level content well, enabling lecture 3me to focus on applica3ons and higher levels of understanding. (This does take a lot of planning, thought, and 3me.) Adapted from Allen, D. and Tanner, K., Cell Biol. Educ. 1: (2002)

16 II. Overview of Ac3vi3es Spring 2012 Introductory Level Discussed goals with Course Coordinator for BIO 101 labs New Forma3ve Assessment: Students plot their grade in class vs. 3me Students should be able to build a graph I want students to be able to interpret data on a graph Intermediate Level Discussed goals with Instructors (and TA) for Bio Used experiences from Fall semester to design a pre- assessment for Spring New Pre- Assessment: Students answer ques3ons about graphs, interpret experimental results New Post Assessment in prepara3on.

17 Example of ploqng ones grades assignment Opportunity for self- reflec3on Extra credit opportunity: (forma3ve assessment= Instructors and TAs get feedback before there is a penalty to student grades)

18 Administered in recita3on within the second week of class Pre- Assessment 262 (Objec3ve: Students can interpret data) Format 6 ques3ons: Mul3ple Choice and Short Response Five types of data selected (based on instructor s emphasis in the course): (1) Popula3on Growth (2) Keeling Curve (3) Rarefac3on Curve (species richness) (4) Compe33on/Preda3on Graphs (5) Bar graphs from manipula3ve experiment Sources for ques3ons: - Assigned SimU text (online program) - Teaching Issues and Experiments in Ecology (TIEE) - Customized from literature

19 40 Anxiety Levels A B C D E 0

20 Pre- assessment BIO students total Student performance was stronger than expected. Generally students performed poorly on a compe33on graph (required Ecology specific- knowledge). Post Assessment (by end of this semester) will measure improvement. Five types of data selected (based on instructor s emphasis in the course): (1) Popula3on Growth (2) Keeling Curve (3) Rarefac3on Curve (species richness) (4) Compe33on/Preda3on Graphs (5) Bar graphs from manipula3ve experiment Correct Responses Overall Percent Correct Number of Correct Responses Ques7on Number 36 1

21 Next Steps BIO 101 lecture 1. Revise Common learning objec3ves (priori3es) based on all faculty and instructor input. - An all- faculty survey is being used to iden3fy priori3es for higher level courses. 2. Build a common on- line pre- assessment for Fall Compile a set of similar final exam ques3ons across sec3ons for post assessment.

22 Next Steps BIO 101 lab BIO Con3nue graphing assessment, revise based on effec3veness. Examine results of post assessment, compare to pre- assessment Con3nue pre- and post- tests in future semesters (with instructor modifica3on). Present format to faculty as a possible model for other upper level elec3ve courses. Sustaining Assessment Ac3vi3es Assemble a team of 3 faculty in Summer 2014 to apply for the Undergraduate Educa3on in Biology Conference on Scien3fic Teaching.