E x e c u t i v e S u m m a r y Improving science content and inquiry standards mastery of fourth-grade students: A study of the effectiveness of National Geographic Science Chicago Public Schools 2010 2011 This study was conducted by National Geographic School Publishing in partnership with two Chicago Public Schools: Charles Gares Dawes Elementary School and John F. Eberhart Elementary School
Improving science content and inquiry standards mastery of fourth-grade students: A study of the effectiveness of National Geographic Science Background and Purpose National Geographic School Publishing (NGSP) recognizes the importance of gathering scientific research evidence to support educators in making informed decisions about instructional programs. As such, NGSP partnered with two Chicago, Illinois Public Schools (CPS) sites during the 2010 2011 school year to gather continued evidence of effectiveness for National Geographic Science (), a comprehensive core science program for elementary students. is a research-based program that immerses students in the nature of science and science inquiry, unlocks the Big Ideas in science for all learners, and builds scientific and content literacy. makes real-life connections to scientists and explorers that encourage students to think like scientists as they master rigorous standards-based content for life, earth, and physical science. Students experience the ways scientists think, learn, and work as they deepen their scientific understanding through use of a science notebook and leveled inquiry investigations. The purpose of this study was to evaluate the impact of on fourth-grade students mastery of science content and inquiry standards. Fourth-grade classes from two CPS sites, Charles Gates Dawes Elementary School (Dawes) and John F. Eberhart Elementary School (Eberhart), participated in this study. The two schools, each located on the southwest side of Chicago, were selected based on their similar teacher and student demographics, high-performance standards, and because science achievement is considered an important area for continued growth at both sites. Overall, the results of this study strongly suggest that enhances students mastery of science content and inquiry standards. Findings from this study contribute to the growing body of evidence in support of as a core science instructional program for elementary students. Study Design During the 2010 2011 school year, a study using a treatment/control quasi-experimental design was conducted to evaluate the impact of in helping students master science content and build inquiry skills. Fourth-grade classes at Dawes served as the treatment group and used as their core science instructional program, while fourthgrade classes at Eberhart served as the control group and continued to use the existing core textbook and supplemental resources for science instruction, and did not use. Both schools taught science for approximately 40 minutes daily for the entire school year. All study teachers completed periodic implementation logs related to science instruction in their classroom. This study focused on the science achievement of fourth-grade students since that grade level is tested by the state of Illinois in compliance with federal No Child Left Behind (NCLB) guidelines. To determine the impact of on students mastery of science standards, results of the Measures of Academic Progress (MAP ) for Science assessment and the Illinois Standards Achievement Test (ISAT)/Science were compared across the two schools. Student Performance Results For this study, student achievement data was collected in accordance with the Illinois Assessment Framework and the schools existing assessment plan for measuring students mastery of science standards. Resulting data sets were analyzed and compared to gauge the impact of. Measures of Academic Progress for Science MAP for Science, published by Northwest Evaluation Association (NWEA), is a computer adaptive test that gauges students current performance and identifies growth targets for science. MAP for Science is nationally normed and is aligned with both national and Illinois Science Standards. General Science items cover specific concepts and content in the domains of Life, Earth, and Physical Science. Concepts and Processes items cover concepts and skills related to scientific inquiry and the nature of science. MAP results are reported as equal interval scaled-scores on the RIT Scale (Rasch unit) that serve as an accurate predictor of students performance on high-stakes assessments.
Overall Science Performance One way of comparing students performance on the MAP for Science is to look at the percentage of students that met or exceeded typical growth, according to NWEA national norms. Target RIT scores for the spring test were set for each student based upon his/ her fall and winter MAP scores. Overall, 7% more students than control group students (NG Science =60.6%, = 53.6%) met or exceeded their science growth targets after just one school year s instruction (Figure 1). Performance by Science Standard A more powerful way of comparing MAP results is Figure 1. Percent of students that met or exceeded MAP for Science growth targets (fall 2010 to spring 2011) to examine how many students achieved at the High performance level, or above the 66th percentile, which is considered an on-track benchmark for college-readiness. On the spring 2011 MAP for Science, a substantially higher percentage of than control group students performed in the High category in all science areas (Figure 2). Across the General Science domains, 9.4% more students were above the 66th percentile for Life Science, 15.4% more students were above the 66th percentile for Physical Science, and 14.8% more students were above the 66th percentile for Earth Science. In the Concepts and Processes areas, 7.8% more students were above the 66th percentile for Scientific Inquiry, and 9.4% more students were above the 66th percentile for Nature of Science. 60% 58% 56% 54% 52% 60.6% Group 53.6% Group 30% 29.1% 30.9% 25% 20% 15% 24.7% 15.3% 13.7% 23.9% 16.1% 16.1% 23.9% 14.5% 10% Life Science Physical Science Earth Science Science Inquiry Nature of Science Schools Schools Figure 2. Percentage of students achieving at the High performance level (above 66th percentile) for each tested science standard
Illinois Standards Achievement Test / Science The ISAT/Science was administrated in March, 2011, to all fourth-grade students in the state, including those in this study, to measure achievement in standards for Life Science; Physical Science; Earth Science; Scientific Inquiry; and Science, Technology, and Society (Nature of Science). Overall Science Performance ISAT results are reported as percentile ranks that can be compared across schools (Figure 3). NG Science students total ISAT/Science percentile scores were substantially higher, an average of 8.4 points, than control group students (=50.8, =42.4). Exceeds Standards Performance ISAT also reports scaled-scores which are assigned to performance level categories, including Meets Standards and Exceeds Standards. Schools strive to increase the percentage of students exceeding standards (scale score range of 237 361) each year, since this performance level is an indicator of being on-track for college-readiness. At the control school, the percentage of fourth-grade students exceeding standards on the ISAT/Science remained about the same from 2010 (3.0%) to 2011 (3.6%). For the group, the percentage of fourth-grade students exceeding standards on the ISAT/Science showed a marked increase, from 2.5% in 2010 to 13.4% in 2011, representing 10.9% growth (Figure 4). When considering the percentage of students exceeding standards, it becomes clearly evident that after just one school year using the program, NG Science students demonstrated heightened mastery of fourth-grade science standards. 48% 46% 44% 42% 40% 50.8% Group 42.4% Group Figure 3. Average percentile scores by study group on the 2011 ISAT/Science 10.0% 5.0% 0.0% 2.5% (Before ) 2010 13.4% 13.4% (After ) 2011 Group Schools 3.0% (No ) 2010 3.6% 2011 (No ) Group Schools Figure 4. Percentage of students exceeding standards on the 2010 ISAT/Science compared with 2011 ISAT/Science
Performance by Science Standard To more fully understand the differences between study groups in students science achievement, ISAT/Science results were examined by standard areas tested: Life Science, Physical Science, Earth Science, Inquiry, and Science, Technology, and Society. students outperformed control group students by a sizable percentage in all science areas: 9% higher for Life Science, 7% higher for Physical Science, 8% higher for Earth Science, and 8% higher for Inquiry (Figure 5). The largest difference was in Science, Technology, and Society, which assesses how well students understand the nature of science the process skills of science, the work that scientists do, and the impact science has on society. After one school year, students in classes performed 11% higher than students who did not use the program. 65% 64% 65% 60% 55% 58% 57% 59% 51% 58% 54% 45% Life Science Physical Science Earth Science Science Inquiry Schools Schools Science, Technology, and Society Figure 5. Average percent correct scores for each science standard tested on the ISAT/ Science, spring 2011 Science Implementation and Instruction Teachers at both study schools completed online logs during the school year related to science instructional practices, curricular materials used with students, and perceptions about students engagement and science learning. When comparing science instruction at the two study sites, some similarities and differences stand out. Teachers at both sites taught science between 150 200 hours each week. materials included a core text and inquiry book, along with corresponding materials for hands-on activities that supported content standards. group science instructional materials included a core text that was used along with supplemental materials. teachers noted limited availability of hands-on materials and structured inquiry lessons that aligned with their text s content. Fourth-grade students using completed at least one hands-on inquiry activity each week, while control group students typically experienced about one hands-on activity per month. All students used a science notebook to record their notes, observations, and data analysis, while only one control group class used a designated science notebook for similar tasks. Differences between the two study groups are notable in terms of students science learning and engagement during science instruction. teachers reported that the curriculum and materials were effective in supporting students mastery of content, vocabulary, content literacy, nature of science, and scientific inquiry. group teachers reported that their materials were effective in supporting students mastery of content, vocabulary, and content literacy learning. However, they indicated that the existing materials were only slightly/not effective in helping students learn scientific inquiry and the nature of science. A substantial difference (19%) in student engagement with science was noted between study schools. teachers reported that 75% of their students demonstrated either average or high
student engagement during instruction with, while control group teachers indicated that 56% of their students demonstrated average or high engagement during science instruction. 28% 47% 25% Group 15% 41% 44% 0% 20% 40% 60% 80% 100% High Average Low Figure 6. Student engagement during science instruction Teachers at both sites were also asked to gauge the program s impact on science instruction. A control group teacher explained, I really enjoy teaching science and would like to use a science series that better prepares my students for state assessments. I feel that I do not have the proper tools I need to be 100% effective. A fourth-grade teacher noted, incorporated activities for differentiated instruction that allowed me to reach all learners and consequently be more effective in teaching the Big Ideas of science that aligned with our state standards. Summary Overall, use of with fourth-grade students was found to enhance science learning and support science instruction. On two different science assessments, students consistently demonstrated higher mastery of fourthgrade science standards compared to students in the control group for all tested areas: Life Science, Physical Science, Earth Science, Scientific Inquiry, and Nature of Science. On the MAP for Science, not only did more than control group students achieve their target growth scores, but far more students in the group performed in the High category (above the 66th percentile) for all areas tested. On the high-stakes ISAT/Science state standards test, students using outperformed their peers in the control group by average percentile score and percent correct by standard, for all standards tested. At the school, there was also a substantial increase (10.9%) in the percentage of fourth-grade students exceeding standards, while the percentage at the control school remained unchanged. The results of this study provide substantial evidence of the effectiveness of as a comprehensive core science program that builds students mastery of grade-level science content and inquiry standards. National Geographic School Publishing 888.915.3276 NGSP.com DEC/11