Mathematics and Science Performance of Primary School Students in the Netherlands

Transcription

1 Educational Research and Evaluation /99/ $ , Vol. 5, No. 2, pp Swets & Zeitlinger Mathematics and Science Performance of Primary School Students in the Netherlands Anja Knuver University of Twente, Enschede, the Netherlands ABSTRACT In this contribution the results of Dutch students of grades 3 and 4 in primary education on the TIMSS mathematics and science test are discussed. Despite severe criticism from subject matter experts on the test, the Netherlands score very high in the international ranking list. The nature of the criticism and the features of Dutch mathematics and science education are described to serve as a background for interpreting these surprising results. INTRODUCTION The Netherlands is one of the 26 countries in which, besides to the first 2 years of secondary education, primary education was also involved in TIMSS. The research question, derived from the general TIMSS research question (Robitaille & Garden, 1996) addressed here is: How do Dutch primary school students from grades 3 and 4 perform on an international mathematics and science test, as compared to students in other countries? And which educational characteristics can help explain these results? In this article some of the Dutch achievement results are presented in relation to those of other countries. England will receive special attention because it is the only other country in this issue involved in primary education. First some information on the mathematics and science curriculum in Dutch primary education will be provided (see also Kuiper & Knuver, 1997), as well as the opinion of Dutch experts about the TIMSS achievement tests (intended curriculum). This information together with teacher judgements (implemented curriculum) will be used as background in interpreting the Dutch results (attained curriculum). Address correspondence to: Anja Knuver, Advies- en Begeleidingscentrum voor het Onderwijs in Groningen (ABCG) P.O. Box 3101, 9701 DC Groningen, the Netherlands. Tel Fax: evaluatiegroep@abcg.rug.nl Manuscript submitted: March 30, 1998 Accepted for publication: January 19, 1999

2 MATHEMATICS AND SCIENCE PERFORMANCE 215 MATHEMATICS AND SCIENCE IN DUTCH PRIMARY EDUCATION Mathematics Education In primary education, a realistic approach to mathematics education has gained popularity over the mechanistic approach. In the Dutch national goals for mathematics education (core objectives stated in 1993) this realistic approach is reflected. In the mechanistic approach, on the one hand, learning content was atomised in meaningless small parts and students were offered mainly fixed solution procedures and were trained by individual exercises. Realistic mathematics education, on the other hand, uses context problems, visual models, individual constructions, interactive education, and other methods. In realistic mathematics education the teacher has a very important role, relating problem solving strategies to the standard solution by visual schemes and models. In such a model, the teacher must have a detailed knowledge of students problem-solving strategies. Most teachers in the Netherlands (about 80%) use textbooks (partly) based on realistic mathematics education for their mathematics lessons. However, research reveals that it is very difficult for them to implement all the didactic characteristics of this approach (Gravemeijer et al., 1993). Science Education For science there are curriculum changes going on in primary education as well (NOB, 1992). The modern science curriculum focuses on a broad and integrated conception of the content of elementary science, including biology, physics, chemistry, earth science, environmental, and technical education. School surroundings as a source for science teaching, sensory experiences and hands-on activities with concrete objects and organisms are all emphasized. The core objectives for science are based upon these modern ideas. The changes in the science curriculum imply a changing role for the teacher: The teacher is more the one who facilitates student learning rather than an instructor. In some primary schools science education is approached in a more traditional way than proposed in the core objectives. DUTCH EXPERTS CRITICISE TIMSS ACHIEVEMENT TESTS In developing the TIMSS achievement test for mathematics and science, two pilot tests were conducted in order to reach the aim of an equally unfair test for all participating countries (Garden & Orpwood, 1996). During the development of the TIMSS achievement tests for mathematics and sci-

3 216 ANJA KNUVER ence, Dutch experts on both subjects criticised the nature of the items. For mathematics the experts judged only about 30% of the items from the pilot studies appropriate to administer to students of grades 3 and 4 (Kraemer, 1994). The most important points of criticism were: (i) the content of the test matches only partially the content of the Dutch mathematics curriculum for grades 3 and 4, the TIMSS target grades in primary education; and (ii) most of the items do not fit in with the modern realistic mathematics curriculum in which context problems, visual models, individual constructions, and interactive education are important characteristics. For science the criticism was less severe, although only about half of the items from the pilot tests were judged to be appropriate for the target grades. Important points of criticism were: (i) many items ask a lot of proficiency in reading and schematising from the students; and (ii) quite a few items show technical deficiencies: distracters are not always relevant while some obvious distracters are missing. APPROPRIATENESS OF THE TIMSS ACHIEVEMENT TEST Test Curriculum Matching Analysis In the TIMSS main run revised pilot achievement tests were administered to the students. One or two subject matter experts (people known for their acquaintance with the intended curriculum) in all participating countries judged the appropriateness of the content of these items by means of a Test Curriculum Matching Analysis (TCMA). An item was determined appropriate if it was pertinent to the intended curriculum for more than 50% of the students at each of the two grades being tested. Given the criticism of Dutch experts it is interesting to interpret the achievement of the Dutch students in the light of the judgements of the experts. Teacher Judgements In the Netherlands, though unfortunately not in other countries, all items have also been judged by the teachers of the target grades. That is, all teachers received one of the eight achievement item booklets and checked whether the content of the item concerned had been taught to their students. The teacher had to judge each item on the following question: Imagine you have to construct an achievement test which fits in well with the education you gave your students in math and science: which of the items in the booklet enclosed are appropriate to include? The Dutch expert and teacher judgements for the 102 mathematics items and the 97 science items are shown in Table 1.

4 MATHEMATICS AND SCIENCE PERFORMANCE 217 Table 1. Expert and Teacher Judgements for TIMSS Mathematics and Science Items. Subject % Items Appropriate Experts Teachers Grade 3 Grade 4 Grade 3 Grade 4 Mathematics Whole Numbers Fractions and proportionality Measurement, Estimation, and Number Sense Data Representation, Analysis & Probability Geometry Patterns, relations, functions Science Earth Science Life Science Physical Science Environmental Issues and the Nature of Science From Table 1 it can be seen that especially for grade 3 the bulk of the items for mathematics as well as those for science is thought by the experts not to be appropriate. For grade 4 this goes for about half of the items for both subjects. Teachers are more positive about the TIMSS items than the experts. An example of an item teachers and specialists do not agree on is shown in Figure 1. This mathematics item is not appropriate to administer to Dutch primary students according to experts because probability is not in the Dutch curriculum for primary education. It is not certain whether teachers discuss items on probability in their mathematics lessons, but no textbook indicates that they should. Therefore, the most obvious explanation for this result is that for this item teachers estimated the probability for their students to make this item correct and did not indicate whether the content had been taught. Research from De Haan (1992) shows that there is a relation (r = about.50) between estimated difficulty and teacher judgement. An example of an item judged appropriate by both experts and teachers is example item 2 (Figure 2). In the Netherlands one of the goals in the intended science curriculum is that students develop a responsible attitude toward a healthy living environment. An item like example item 2 fits in well in this curriculum. Experts as well as teachers consequently judge it an appropriate item.

5 218 ANJA KNUVER There is only one red marble in each of these bags. 10 Marbles 100 Marbles 1000 Marbles Without looking in the bags, you are to pick a marble out of one of the bags. Which bag would give you the greatest chance of picking the red marble? A. The bag with 10 marbles. B. The bag with 100 marbles. C. The bag with 1000 marbles. D. All bags would give the same chance experts teachers grade 3 grade 4 grade 3 grade 4 77% 80% Fig. 1. Example Item 1. Some things were buried in wet ground. Several years later they were dug up. Which thing is MOST likely to have stayed the same? A. An egg shell B. A plastic cup C. A piece of paper D. An orange shell experts teachers grade 3 grade 4 grade 3 grade % 100% Fig. 2. Example Item 2.

6 MATHEMATICS AND SCIENCE PERFORMANCE 219 SAMPLE AND RESPONSE RATES IN THE NETHERLANDS As can be seen in the international reports for population 1 (Martin et al., 1997; Mullis et al., 1997) the low response rate of the Netherlands is exceptional as compared to other countries. Even after substituting replacement schools, the response rate is no higher than about 60% (both for primary and secondary education). Such a low response rate is quite normal in Dutch educational research. Schools are being confronted with a lot of requests for participating in research, while at the same time they have extended tasks in educational reform which keep them very busy. Moreover, schools are free to decide whether they want to participate in a study. They cannot be obliged by either the Inspectorate of Education or the Ministry of Education, as is the case in some other countries. In total for primary education 5,359 students from 141 schools were involved in TIMSS. Analyses on the background characteristics of schools and students shows that the distribution of background characteristics for the sample is the same as for the population. Hence, we conclude that there is some evidence that the sample is typical for Dutch school children. Results: Student Achievement From the international tables (Martin et al., 1997; Mullis et al., 1997) it can be seen that Dutch students do very well in mathematics as well as in science. In Tables 2 and 3 the results for both subjects are shown for grade 4, the upper target grade. In the tables shown here, the international tables have been adapted in such a way that all 26 participating countries are ranked according to their mean overall achievement score, as well as the countries that are put in a separate category in the international report (for example because of low response rates). From the tables it can be seen that the Dutch students are not only performing very well for both mathematics and science, but also that the range is small, which means that there are very few students who perform poorly. Of course, this also implies that there are not many Dutch students who perform extremely well. One of the explanations for such a small range might be that about 7% of the students with special needs attend special schools. These schools are not included in the sample for the Netherlands. The TIMSS International Study Centre allowed at most 10% of the students from the population to be excluded from the sample in participating countries.

7 220 ANJA KNUVER Table 2. Mathematics Achievement of Grade 4 Students. Mathematics If we look at student performance in England, we can see that for mathematics England does a lot worse than the Netherlands in grade 4. In grade 3 the differences are less extreme. In fact, the two countries are performing at the same level for mathematics in this grade. It seems that in the Netherlands grade 4 is very important for mathematics learning. This can also be concluded when achievement differences between grades are studied. The Netherlands shows the largest achievement differences of all countries from grade 3 to grade 4. From grade 4 to the second year of secondary education, increases in mathematics performance are, except for the USA, no smaller in any other country than the Netherlands.

8 MATHEMATICS AND SCIENCE PERFORMANCE 221 Table 3. Science Achievement Grade 4 Students. Science For science, students in England and the Netherlands perform at the same level. For the Netherlands achievement differences between grades are average. EXPLANATION OF THE RESULTS Science For science the Dutch results are not extreme. On the one hand, experts were not too happy with the fit of the content of the items to the Dutch

9 222 ANJA KNUVER What is the BEST reason for including fruits and leafy vegetables in a healthy diet? A. They have a high water content. B. They are the best source of protein. C. They are rich in minerals and vitamins. D. They are the best source of carbohydrates. experts teachers students g3 g4 g3 g4 g3 g4 89% 100% 91% 93% Fig. 3. Example Item 3. curriculum, but on the other hand the Dutch curriculum is not clear-cut and in practice differs from school to school. The TIMSS test also included a lot of items that could be learned from other sources than education in school. In the Netherlands television offers a lot of science programs for children. Therefore, it is not easy to interpret the performance of the Dutch students in science as a result of education. Example item 3 shows an item Dutch students do very well (Figure 3). Item 3 is judged by subject matter experts to be not appropriate for grade 3 and 4 students according to the curriculum. Teachers are positive and students do very well on it. In fact, Dutch students do better on this item than students in any other country in the world. The international average percentages correct are 58% and 65% for grades 3 and 4, and 72% and 75% correct for the first and second year of secondary education. The explanation for these findings must be that Dutch children are aware of the importance of taking minerals and vitamins to stay healthy and that eating fruit and vegetables is a way of doing so. This information is available to young children not only in the science lessons, but especially through television, school rules (no sweets, only fruits are allowed), journals, books, and so on. Mathematics Especially in grade 4 Dutch students perform extremely well on a test which does not fit the Dutch mathematics curriculum according to experts; only 50% of the items for grade 4 is thought to be in line with the Dutch mathematics curriculum. However, teachers do not criticise the mathematics test as severely as the experts do. These at first sight contradictory findings could be explained mainly in two different ways.

10 MATHEMATICS AND SCIENCE PERFORMANCE 223 First it could be true that Dutch teachers do not teach according to the Dutch intended curriculum for mathematics, in other words: the intended and the implemented curriculum are not in accordance with each other. This could imply that a more traditional curriculum is taught, which largely overlaps with the items in the TIMSS-test. A second possible explanation is that the realistic Dutch mathematics curriculum implies a way of teaching and organisation of the mathematics lessons, that students learn to have an open approach to all kinds of mathematics problems, which does not depend on the exact content they learned before test administration. This could lead to the situation that Dutch students not only perform well on items which suit the content and shape of the Dutch curriculum, but also on strange items. As often is the case, the truth is most likely to lie midway. From studies on Dutch mathematics education we learned that, although modern realistic textbooks based on the intended curriculum (core goals) are being used in most Dutch primary schools (Deinum & Harskamp, 1995; Gravemeijer et al., 1993; Inspectie van het Onderwijs, 1997), teachers seem to have difficulty in using the interactive approach which is the essential characteristic of working with these books. As a result, students who use modern textbooks do not perform better on standardised tests, which measure performance on core goals, than students who use traditional textbooks. To investigate whether Dutch students would have performed better on a test fitting the intended curriculum for mathematics well, national option items were answered by grade 4 students. Example item 4 shows an extreme example of an item on proportions, which was in the Dutch national option test for TIMSS pop1, and thus judged to be appropriate for grade 4 by experts. Teachers doubt whether this item fits to the curriculum they teach their students in grade 4: only 18 x 24 Think of a multiplying problem with the same solution. You may not use the same figures. x experts teachers students grade 4 grade 4 grade % 20% Fig. 4. Example Item 4.

11 224 ANJA KNUVER 56% of the teachers judge it appropriate. Students do not seem to master this kind of item: only 20% finds a correct solution. The conclusion must be that the intended curriculum is not always in line with the implemented curriculum, that is classroom practice. An example of the open mind of Dutch students towards content they never learned is shown by example item 1 on the bags with marbles. Dutch students do very well for this item while probability occurs neither in modern nor in traditional textbooks. This is also the reason for experts to find it an inappropriate item. About 80% of the teachers are positive about the item. We assume that this has more to do with positive expectations of the capacities of their students than with the content of their implemented curriculum. Dutch students make this item better than students in any other country in the world: the international mean is 51% correct for grade 4 and 78% for grade 8; 74% of the Dutch grade 4 students make this item correct. CONCLUSIONS Probable explanations for the excellent results of Dutch students on the TIMSS-tests as mentioned in the previous section are suggestions that need further analysing of the existing data to estimate whether they make sense. Next to these suggestions other characteristics of Dutch mathematics education should be taken into account. Some characteristics in which the Netherlands differ from other countries are for example: In most schools whole class teaching is the dominating procedure in most mathematics lessons. This is the same for a lot of other countries, but for example not for England. In most schools 1 hour per day is spent on mathematics. In most countries this is less (about the same in England). About 50% of the students are never assigned any homework. In all other countries most students do get homework although they do not have to work on it very long. Only in countries like Hong Kong, Korea, and Singapore homework asks a lot of time. In most other countries the majority of teachers in grades 3 and 4 are female. In the Netherlands only 35% of the teachers are female. Apart from characteristics of education and teachers, also students in the Netherlands have some features which could play a role in explaining their high performance on the international tests. Some striking characteristics are:

12 MATHEMATICS AND SCIENCE PERFORMANCE 225 Dutch students have a no nonsense attitude concerning what is necessary to be good in math. They are of the opinion that good luck and memorising do not help; only natural talent and lots of hard work is not useless according to Dutch students. As in other high performing countries like Japan and Korea, Dutch students have less positive attitudes than students in countries with lower mean achievement scores. Girls do less well than boys in the Netherlands: girls achievement scores are lower and their attitudes towards mathematics are less positive. Only the differences in achievement for science are not extreme compared to other countries. For mathematics almost no other countries show differences as large as in the Netherlands for grade 4. Advanced correlational analyses like performed by Bos and Kuiper (1999) are necessary to gain insight into the relation between these typically Dutch educational characteristics and the outstanding performance of Dutch students on a bad fitting test. REFERENCES Bos, K., & Kuiper, W. (1999). Modelling TIMSS data in a European comparative perspective: Exploring influencing factors on achievement in mathematics in grade 8. Educational Research and Evaluation, 5 (2), De Haan, D. (1992). Measuring test-curriculum overlap. Enschede: University of Twente. Deinum, J.F., & Harskamp, E. (1995). Rekenmethoden en rekenprestaties in de jaren negentig [Mathematics textbooks and mathematics achievement in the nineties]. Groningen: RION. Garden, R.A., & Orpwood, G. (1996). TIMSS test development. In M.O. Martin & D.L Kelly (Eds.), Third International Mathematics and Science Study, Technical report Vol 1, (pp. 2 1 to 2 19). Boston, MA: Boston College. Gravemeijer, K.M., Heuvel-Panhuizen, G. van den, Donselaar, N. van, Ruesink, L., Streefland, W., Vermeulen, E., Woerd, E. te, & Ploeg, D. van der (1993). Methoden in het reken-wiskundeonderwijs, een rijke context voor vergelijkend onderzoek [Mathematics textbooks, a rich context for a comparative study]. Utrecht: Freudenthal Instituut. Inspectie van het Onderwijs (1997). Rekenen en wiskunde in het primair onderwijs [Mathematics in primary education]. Kraemer, J.M. (1994). TIMSS-toets niet geschikt om de kwaliteit van het Nederlandse reken/wiskunde-onderwijs te beoordelen [TIMSS test not appropriate for judging the quality of the Dutch mathematics education]. Tijdschrift voor Nascholing en Onderzoek van het Reken/wiskunde-onderwijs, 13(2), Kuiper, W., & Knuver, A. (1997). The Netherlands. In D.F. Robitaille (Ed.), National contexts for mathematics and science education: An encyclopaedia of the educational systems participating in TIMSS (pp ). Vancouver: Pacific Educational Press.

13 226 ANJA KNUVER Martin, M.O., Mullis, I.V.S., Beaton, A.E., Gonzales, E.J., Smith, T.A., & Kelly, D.L. (1997). Science achievement in the primary school years. IEA s Third International Mathematics and Science Study. Boston: Boston College. Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzales, E.J., Kelly, D.L., & Smith, T.A. (1997). Mathematics achievement in the primary school years. IEA s Third International Mathematics and Science Study. Boston: Boston College. NOB (1992). Natural science education. A broad outline. Enschede: National Institute for Curriculum Development (SLO). Robitaille, D.F., & Garden, R.A. (Eds.). (1996). Research questions and design. TIMSS Monograph No. 1. Vancouver: Pacific Educational Press.