Using Curriculum to Change How Teachers Teach Science and Students Learn Science

Transcription

1 Using Curriculum to Change How Teachers Teach Science and Students Learn Science A Paper Set Prepared by Susan M. Kowalski, Janet Carlson, Pamela Van Scotter, Brooke N. Bourdélat-Parks, Stephen R. Getty, Betty Stennett, and Paul Beardsley NARST Annual Conference Indianapolis, IN 25 March 2012

2 Flow of the Paper Set A Research-based Development Process Features of the Curriculum Teacher Practice Associated with Curriculum Student Learning Associated with the Curriculum

3 The Challenge 1 Middle school is a critical time for Inspiring student interest in science Establishing foundational understandings in science (Hanson, 2004; Beghetto, 2007) To inspire interest and promote understanding, we need Effective teaching practices Coherent, rigorous, focused instructional materials

4 The Challenge 2 Preponderance of materials are fragmented (AAAS, 2001; Schmidt, 2001) Two-thirds of middle school science teachers have a degree in a field other than science (Fulp, 2002) Teachers teaching out-of-field rely heavily on materials (Ball & Feiman-Nemser, 1988) Problems are most severe in low-income schools

5 The Challenge 3 Teachers need access to materials that support development of teacher content knowledge and pedagogical content knowledge. Achievement gaps by demographic subgroups persist nationwide The U.S. requires a scientifically literate citizenry to face global challenges

6 The Challenge 4 Science instruction must evolve to incorporate current knowledge about learning. Rigor, coherence, and focus of middle school science curricula need to be improved.

7 Our Perspective Effective curricula can be a valuable means to improve student interest and achievement in science. Taking Science to School, NRC, 2007

8 Curriculum (Matters) Research Development Design

9 Our Work to Address the Challenge Develop instructional materials that are beneficial to both teachers and students Part of a US Department of Education IES-funded Goal 2 development and innovation (Grant # R305A080422) Materials iteratively developed and tested (pilot study plus two feasibility tests) Materials include seven key features

10 Theoretical Framework for Approach to Learning Our curriculum development process draws on years of research on cognitive development and how humans learn Vygotsky (1962) and Piaget (1975) Summary of recent findings well-articulated more recently in the meta-analysis How People Learn (Bransford, Brown, & Cocking, 2000).

11 Theoretical Framework for Approach to Learning Key findings summarized in How People Learn include the following: Students come to class with their own conceptions about how the natural world works. Students need a strong foundation of knowledge upon which to build their understanding of new ideas. Students benefit from a metacognitive approach to learning where they are responsible for monitoring their own progress.

12 Features of the Curriculum Materials For students 1. Rigorous, coherent, and focused 2. The BSCS 5E Instructional Model 3. Comprehensive assessment package 4. Metacognitive strategies 5. Literacy strategies 6. Collaborative learning For teachers 7. Highly educative

13 Scope & Sequence Content Standards Science as inquiry Core concepts (physical) Core concepts (life) Core concepts (Earth- space) Mul0- disciplinary Unit Science & Society 6 th Grade Form and Func0on/ Evolu0on and equilibrium Science as a way of knowing Properties and changes in properties of matter Structure and function in living systems (cells) Regulation and behavior (structure and function) Structure of Earth systems Integrating chapter Natural hazards Risks and benefits Abilities of technological design 7 th Grade Constancy and Change / Evolu0on and equilibrium Science as a way of knowing Motions and forces Integrating chapter Reproduction and heredity Diversity and adaptations of organisms Inheritance and evolution of behavior Earth s history Integrating chapter Science and technology in society Understandings about science and technology 8 th Grade Systems and Subsystems / Energy and Ma@er Science as a way of knowing Develop explanations using evidence Communicate scientific procedures and explanations Energy in energy out Transfer of energy Living energy Structure and function in living systems Regulation and behavior Populations and ecosystems Energy in earth systems Earth in the solar system Energy and your body Personal health Body systems

14 8 th Grade Systems and Subsystems /Energy and Science as Inquiry Science as a way of knowing Develop explanations using evidence Communicate scientific procedures and explanations Physical Science Energy in energy out Transfer of energy Life Science Living energy Structure and function in living systems Regulation and behavior Populations and ecosystems Earth-space Science Energy in earth systems Earth in the solar system Science & Society Energy and your body Personal health Body systems

15 Development Process Development team included scientists, teachers, and science educators Held a design conference Shared conceptual flow graphics for units and chapters BSCS 5E Instructional Model guided the development Engage, Explore, Explain, Elaborate, and Evaluate

16 Theoretical Framework Curriculum Four strands from Taking Science to School (NRC, 2007) guided our process. This report recommends that students 1. know, use, and interpret scientific explanations of the natural world, 2. generate and evaluate scientific evidence and explanations, 3. understand the nature and development of scientific knowledge, and 4. participate productively in scientific practices and discourse.

17 Development Process Use the Understanding by Design approach (Wiggins and McTighe, 2005) Backward Design Stage 1: What do we want students to learn? Stage 2: What will serve as evidence that they have learned? Stage 3: What should the learning sequence be? Begin the development with the Evaluate Then develop the Engage, Explore, Explain, and Elaborate for each chapter Iterative process

18 Review and Test Internal reviews were conducted by the team, and external reviews were conducted by content experts. First test of feasibility included 25 teachers Selected for diversity across Geographic location Urban, Suburban, Rural Student population

19 Formative Stage of Research Collected formative data Student pre-posttests Teacher surveys Classroom observations External reviews of teacher and student materials by experts Used rubrics designed to evaluate the extent to which the materials reflected each of the intended features

20 Iterative Revision Process The team revised the materials as necessary to attend to the external reviews of the materials, the observation data, and the student learning results following the first feasibility test. The team used the same backward design process and internal review process. These iterative steps resulted in 6 iterations of curriculum revision.

21 Second Test of Feasibility Tested materials around the US in second test of feasibility 24 field test teachers Approximately 2000 students Attended to diversity Hosted a three-day orientation for teachers Presenting results from this test

22 Characteristics of Participating Teachers & Students and their Schools 12 states 17% urban; 25% rural; 58% suburban districts 13% private schools; 87% public 2-100% of students from underrepresented groups (average: 50%) 2-37 years of teaching experience (average: 14 yrs)

23 Methodology Mixed Methods Student data modified Attitudes Toward Science Inventory (Weinburgh & Steele, 2000) Pre and Posttests (content and confidence) Sample student notebooks Curriculum data external expert reviews of features Teacher data Usability of each feature Observations RTOP (Sawada et al., 2002) BSCS Teacher Fidelity of Implementation Rubric (BSCS, 2009) BSCS Student Fidelity of Implementation rubric (BSCS, 2009) Collaboratives for Excellence in Teaching Preparation (CETP) 5-minute observation rubric (Lawrenz et al., 2007)

24 Ongoing Research Digital version of the program will be available at no monetary charge for use during the school year. Go to elearn.bscs.org to register and see the curriculum.

25 What is the Evidence of Usability and Feasibility of the Curriculum Features?

26 Features of the Curriculum Materials For students 1. Rigorous, coherent, and focused 2. The BSCS 5E Instructional Model 3. Comprehensive assessment package 4. Metacognitive strategies 5. Literacy strategies 6. Collaborative learning For teachers 7. Highly educative

27 Data Collection Key Feature External Review Classroom Observations Teacher Surveys Rigorous, coherent, and focused X X The BSCS 5E Instructional Model X X Comprehensive assessment package X X Metacognition strategies X X Literacy strategies X X X Collaborative learning X X X

28 Dominant theme of inquiry Feature 1: Enhancing Rigor, Coherence, and Focus Unifying concepts of energy and systems Chapter Organizers help students and teachers see where they are in the learning process by making connections between ideas explicit

29 Feature 1: Enhancing Rigor, Coherence, and Focus Teacher Surveys Not at all coherent Somewhat coherent Coherent Highly coherent External reviewers (combined results) Not at all coherent Low coherence Moderate coherence High coherence

30 Feature 2: The BSCS 5E Instructional Model Engage Evaluate Explore Elaborate Explain

31 Feature 2: The BSCS 5E Instructional Model External reviewers 0% 25% 50% 75% 100% Use of 5Es Data points represent the 4 units

32 Feature 2: The BSCS 5E Instructional Model Teacher Fidelity of Implementa0on to BSCS 5E Instruc0onal Model Percent of Possible Points 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% PS1 PS2 PS3 PS4 PS5 PS6 LS1 LS1a LS2 LS2a LS3 LS4 LS5 LS5a ES1 ES2 ES3 ES3a ES4 ES4a ES5 ES5a ES6 ES7 ES7a SS1 SS1a SS2 SS2a SS3 SS3a SS4 SS5 SS5a SS6 SS6a Teacher ID

33 Feature 2: The BSCS 5E Instructional Model Student Fidelity of Implementa0on to BSCS 5E Instruc0onal Model Percent of Possible Points 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% PS1 PS2 PS3 PS4 PS5 PS6 LS1 LS1a LS2 LS2a LS3 LS4 LS5 LS5a ES1 ES2 ES3 ES3a ES4 ES4a ES5 ES5a ES6 ES7 ES7a SS1 SS1a SS2 SS2a SS3 SS3a SS4 SS5 SS5a SS6 SS6a Teacher ID

34 Feature 3: Comprehensive Assessment Package

35 Feature 3: Comprehensive Assessment Package Teachers surveys Not at all effective Somewhat effective Effective Highly effective External reviewers 92% of elements Not incorporated Incorporated Very well incorporated

36 Feature 4: Metacognitive Strategies

37 Feature 4: Metacognitive Strategies

38 Feature 4: Metacognitive Strategies Teacher surveys Not at all effective Somewhat effective Effective Highly effective In a few cases, classroom observations showed that students tended to reflect back on thinking to the extent that teachers encouraged them to do so.

39 Feature 5: Literacy Strategies

40 Feature 5: Literacy Strategies

41 Feature 5: Literacy Strategies Teacher surveys Not at all effective Somewhat effective Effective Highly effective External reviews Results for almost all elements of Science and Society unit Not at all effective Somewhat effective Highly effective

42 CCCR Strategy Feature 6: Collaborative Learning Consider-Contribute-Consult-Revise Strategy Use and reflect on teamwork skills

43 Feature 6: Collaborative Learning Teacher surveys Not at all effective Somewhat effective Effective Highly effective External reviews Element not present Element present Element present and clearly supported

44 Classroom observations Use of collaborative learning Feature 6: Collaborative Learning Feature not evident Feature evident When collaborative learning was seen, other elements rated Feature not evident Feature evident

45 Discussion Based on teacher surveys, external reviews, and classroom observations Each of the 6 key features related to students were found to be usable and feasible for students and teachers In several chapters, revisions were made after the second field test to further strengthen the key elements

46 What do the Data Say about Teacher Practice Associated with Curriculum?

47 Theoretical Framework for Approach to Teaching Teachers play an interpretive role in bringing curricula to life for their students Select elements for inclusion Emphasize or deemphasize curricular elements

48 Theoretical Framework A complex teacher-curriculum relationship exists (Remillard, 2005) Contextually based Dependent on both the teacher and the curriculum Tightly interconnected with other teacher practices

49 Theoretical Framework Curriculum materials could contribute to professional practice if they were created with closer attention to processes of curricular enactment. (Ball & Cohen, 1996, p. 7, emphasis in original)

50 Theoretical Framework To support teachers, curricular materials should be educative. That is, they should be designed to promote teacher learning as well as student learning. Nine design heuristics of educative science materials guided the development of teacher support materials (Davis & Krajcik, 2005). Nine heuristics articulate roles for materials in three major areas (total of 24 elements) PCK for Science Topics PCK for Scientific Inquiry Subject Matter Knowledge

51 Educative Materials Heuristics and Elements of Curriculum Materials Educative Materials Major Area PCK for Science Topics Element in Materials Information on common student conceptions Information on pedagogical strategies Explanations and instructions for process and procedure steps Suggestions for specific opportunities for individual and group assessment Outcomes and Indicators of Success Samples of student work

52 Educative Materials Heuristics and Elements of Curriculum Materials Educative Materials Major Area PCK for Scientific Inquiry Elements in Materials Descriptions of the nature of each E and rationale for the instructional model Extensive use of focus questions within units Multiple activities for students to ask and answer their own questions with support for teachers on how to guide students toward appropriate questions Rationale for the importance of having students design their own investigations Multiple opportunities for students to use an explanation template and guidance to teachers for fading scaffolds Multiple opportunities for students to collaborate with guidance for both teachers and students on how to share ideas successfully.

53 Educative Materials Heuristics and Elements of Curriculum Materials Educative Materials Major Area Subject Matter Knowledge Element in Materials Comprehensive teacher background information on science content Teacher background materials extend beyond the level of understanding required by students Teacher background materials illustrate the relationships between key ideas and everyday phenomena

54 Example Design Heuristic 4 (includes 2 elements) Instructional materials should support teachers in anticipating, understanding, and dealing with students ideas about science Curriculum materials should help teachers recognize the importance of students ideas and help teachers identify likely student ideas within a topic Curriculum materials should help teachers gain insight into how they might be able to deal with the ideas in their teaching, for example, by giving suggestions of thought experiments likely to promote the development of more scientific ideas.

55 Design Heuristic 4 in Science and Society Unit, Chapter 5 Chapter rated comprehensive and thorough by an external evaluator Evidence: In the Engage phase, the Process and Procedure section (p. 12) highlights a particular idea that students might have: Some may say that they don t want to know anything about diabetes. The TE provides a suggestion for how to address this idea: Encourage them to think of questions a person who just found out they had diabetes might want to know.

56 Data Collection in Service of Iterative Development External reviews of teacher materials Review rubric aligned with nine heuristics Each of 24 elements scored on a scale Little or none Some Comprehensive and thorough

57 Teacher surveys Data Collection in Service of Iterative Development Administered after every activity Included items related to both teacher support materials and student materials Included extensive comments on enactment Included extensive comments on any changes teachers made and why (sequence, omissions, augmentations, etc.)

58 Teacher Observations RTOP (Sawada et al., 2002) Data Collection in Service of Iterative Development BSCS Fidelity of Implementation Rubric (BSCS, 2009) Student Tests (Pre and Post)

59 Results First, a caveat The data presented here resulted from a development study. Because the purpose of the data collection was to inform revisions to the materials, the data are not of sufficient scope to make broad generalizations. For example, there was no comparison group in this study; therefore, we cannot make claims about the benefit of these materials over others.

60 Results Successful incorporation of MOST educative features: Two elements out of 24 needed enhancement: The curriculum materials should help teachers adapt and use approaches for collecting and analyzing data across multiple topic areas even when the data being collected seem fairly different (e.g. plant growth as opposed to weather conditions). Curriculum materials should help teachers recognize the importance of having students design their own investigations.

61 Examples of Changes to Materials Based on Review Collecting and Analyzing Data Teacher s Guide helps teachers adapt and use vector notation as a means of collecting and analyzing data by 1. introducing the technique as students investigate planetary velocities around a central star and 2. showing teachers how to adapt the technique as students investigate the relative strengths of gravitational forces of attraction between planetary objects.

62 Examples of Changes to Materials Based on Review Importance of Having Students Design Investigations Revised a Teacher s Guide for helping students design investigations. The introductory paragraphs of this guide provide extensive rationale for the importance of providing experiment design opportunities to students.

63 Results Evidence of teacher use of reform-based practice: Mean RTOP score (100 possible) This study: 63.3 Nationally for MS science: 50 (Sawada et al., 2002) Finding aligns with that of another BSCS study: HS materials Random assignment External researchers conducting observations BSCS teachers had RTOP scores > 2 standard deviations above control teachers

64 Results Mean Fidelity of Implementation (FoI) score: 88% Significant correlation between FoI and RTOP: r =.423; p =.040 Fidelity ~ RTOP Association with student learning??? Hierarchical linear modeling (students nested within teachers) FoI neared significance (p =.056) in predicting mean student posttest (adjusted by pretest and student demographics) RTOP non-significant (p =.456)

65 Results in short Use of the materials is positively associated with reform-based teaching practices and is also associated with improved student achievement.

66 What is the Evidence of Student Learning?

67 Proficiencies of Science Students should be able to know, use, and interpret scientific explanations of the natural world; generate and evaluate scientific evidence and explanations; understand the nature and development of scientific knowledge; and Participate productively in scientific practices and discourse (NRC, 2007)

68 Methods Pre- and Posttests Content Questions Confidence Questions Student Notebooks Classroom Observations CETP 5-minute observation protocol (Lawrenz et al., 2007) Fidelity of Implementation of the BSCS 5E Instructional Model: Students in the Classroom [Student FoI] (BSCS, 2009)

69 Content Questions Unit Strand 1: Know, use, and interpret scientific information Mean Difference (SE) Content Gain p Effect Size (95% CI) Earth Science 5.23 (.19) (1.01 to 1.23) Physical Science 3.65 (.24) (0.66 to 0.91) Life Science 3.84 (.25) (0.51 to 0.79) Science and Society 5.93 (.23) (1.14 to 1.43)

70 Achievement Gains by Demographic Group Subject Student Group N p Value Effect Size Earth Science Gender Boys 329 < Girls 380 < Race/Ethnicity White/Asian 330 < Underrepresented 379 < ELL Status English Native 631 < ELL 78 < FRL Status No FRL 415 < FRL 294 <

71 Achievement Gains by Demographic Group Subject Student Group N p Value Effect Size Life Science Gender Boys 182 < Girls 225 < Race/Ethnicity White/Asian 266 < Underrepresented 141 < ELL Status English Native 346 < ELL 61 < FRL Status No FRL 335 < FRL 72 <

72 Achievement Gains by Demographic Group Subject Student Group N p Value Effect Size Physical Science Gender Boys 269 < Girls 248 < Race/Ethnicity White/Asian 256 < Underrepresented 261 < ELL Status English Native 419 < ELL 98 < FRL Status No FRL 327 < FRL 190 <

73 Achievement Gains by Demographic Group Subject Student Group N p Value Effect Size Science and Society Gender Boys 199 < Girls 257 < Race/Ethnicity White/Asian 158 < Underrepresented 298 < ELL Status English Native 315 < ELL 141 < FRL Status No FRL 234 < FRL 222 <

74 Putting Achievement Gains in Context Lynch and colleagues (2005) conducted a study comparing highly rated reform materials (Chemistry That Applies, State of Michigan, 1993) to business as usual. Group N p Value Effect Size Chemistry that Applies Business as Usual 1087 p < p <

75 Confidence Questions Strand 1: Know, use, and interpret scientific information Unit Confidence Gain Mean Difference (SE) p Effect Size (95% CI) Earth Science 10.8 (.52) (0.66 to 0.88) Physical Science 3.10 (.50) (0.14 to 0.41) Life Science 11.5 (.95) (0.49 to 1.04) Science and Society (.62) (0.65 to 0.92)

76 Example of Change in Content and Confidence Q. Suppose you can measure the total amount of energy in different feeding levels. Which level would have the LEAST amount of energy? A. Plants and other producers B. Herbivores that eat plants C. Carnivores that eat herbivores D. Secondary carnivores that eat other carnivores How confident are you that you answered the question correctly?

77 Changes in Understanding and Confidence Frequency of selecting A (Misconception) Frequency of selecting D (Correct Answer) Which answer had the highest confidence rating? Pretest Misconception (answer A) Posttest Correct answer (answer D)

78 Science Notebooks Element of Explanation (N = 75) Generate and Evaluate Scientific Evidence and Explanations Mean SD Claim Evidence Reasoning Score of 2 indicates that responses were often accurate but incomplete.

79 Context for Understanding Scientific Evidence and Explanation Scores Ruiz-Primo and colleagues (2010) scored students explanations in notebooks Completeness Complete explanations with claim, evidence, and reasoning Percent of Students 18.1% Provide only claim and evidence 12.0% Provide only claim 40.3% Provide only data 9.7%

80 Understanding the Nature of Scientific Evidence and Explanations Science as Inquiry Assessment Significant gains from pretest to posttest (p <.001) Effect size d = 0.1 Science notebooks Students were able to generate accurate but incomplete explanations from evidence

81 Productive Participation in Scientific Discourse Classroom Observations: CETP 5-minute observation protocol What cognitive demands were placed on students during each 5-minute segment of time? Passively receiving information? Applying knowledge? Generating Explanations? > 60% of class time To what extent were students on task? Majority of students on task > 90% of class time.

82 Productive Participation in Scientific Discourse Student Fidelity of Implementation Participation closely aligned with developers intentions Example: materials provided clear protocols for sharing work with other students and revising work based on that feedback

83 Implications Development We offer a model of research-based curriculum development. We offer a model of how to incorporate key features of curricula.

84 Implications Student Learning Research-based curricula can play an important role supporting student science proficiency Students can learn challenging content; Students can generate scientific explanations; Students can understand how scientific knowledge develops; and Students can participate productively in scientific discourse.

85 Implications Teachers Curriculum materials can be beneficial for both teachers and students. Educative materials may enhance teacher practice and student learning.

86 Dissemination Model Ongoing R&D model for materials through online curriculum dissemination beginning Fall 2012 Materials available free of monetary charge in exchange for student pre/post data and teacher usability data elearn.bscs.org

87 Future Research Efficacy trial to increase our confidence in making causal claims between use of the instructional materials and their effects on teacher practice. Further development work (for grades 6-7)

88 Thanks to the Team! Janet Carlson (PI) Pam Van Scotter (Co-PI) Susan M. Kowalski (project lead-- research) Brooke N. Bourdelát-Parks (developer) Betty Stennett (project lead--development) Stephen R. Getty (developer) Paul Beardsley (developer)

89 Thanks! Research supported by the Institute for Education Sciences (IES) Grant Number R305A This presentation and the associated paper will be available at Monday afternoon