Transition from Primary to Secondary Schools in Malta: A Report

Transcription

1

2 2

3 Transition from Primary to Secondary Schools in Malta A Review Report commissioned by the Hon. Dr. Louis Galea, Minister of Education, Youth and Employment, originally presented in July Updated version presented to the Hon. Dolores Cristina, Minister of Education, Culture, Youth and Sport, in September 2008 Grace Grima (Chairperson) Leonard Grech Fr. Charles Mallia Bernie Mizzi Peter Vassallo Frank Ventura Ministry of Education, Culture, Youth and Sports July

4 Transition from Primary to Secondary Schools in Malta: A Review Ministry of Education, Culture, Youth and Sport, 2008 This publication, or part thereof, cannot be reproduced in any format without the permission of the copyright holder. MALTA. Ministry of Education, Culture, Youth and Sport Transition from Primary to Secondary : a review / Grace Grima [et al.]. Floriana : Ministry of Education, Culture, Youth and Sport, 2008 p. : ill. ; cm. ISBN: Primary education Malta Reports. 2. Secondary education Malta Reports. 3. Transition education Malta Reports. I. Grima, Grace Melitensia: MZV5 LC: LB1626.M DDC: dc22 ISBN: Cover design: Ġorġ Mallia 4

5 TABLE OF CONTENTS 1.0 Introduction Purpose of the Review Methodology Structure of the Report Background Lessons from the Past The Rationale of the 11+ Examination Introduction The Purpose of Primary Education: the Preparatory Tradition The Establishment of the Junior Lyceums Developments after Review of the Rationale of the Junior Lyceums The Common Entrance Examination The Independent Schools Conclusions The Impact Review Introduction Impact on Children Impact on Families Impact on Teachers Impact on our Educational System Impact on Certification and Life Chances Quality Review Introduction The Quality of JL and CE Examinations: General Characteristics The Quality of the 11+ Examinations: A Quantitative Analysis Evaluation of the JL and CE Examination Papers by Local Subject Specialists Cambridge International Examinations (CIE) Evaluation of the May 2006 JL and CE Examinations An Analysis of the English and Mathematics 2005 Syllabi by CIE Conclusion 151 5

6 5.0 The Way Forward: Towards a Developmental Approach to Primary Education Introduction The Guiding Principles The Developmental Approach in Education The Assessment System An Action Plan for Assessment in the State Primary Schools An Action Plan for Assessment in the State Secondary Schools Curriculum, Pedagogy and Support Systems Church and Independent Schools External Monitoring International Testing Programmes Concluding Observations 186 Bibliography Appendices 1 Discussions on Transition From Primary to Secondary Perspectives on the 11+ examination J. Fenech Teacher Questionnaire for the 11+ Examination Review G. Grima and F. Ventura An Evaluation of the Junior Lyceum and Private Schools Association Entrance Examination Papers, English 2004, 2005, and 2006 D. Spiteri Submissions to the 11+ Review Committee with respect to the Analysis on the Junior Lyceum and Common Entrance Examination Papers for Maltese and their attendant Syllabi - Sandro Spiteri a Submissions to the 11+ Review Committee with respect to the Analysis on the Junior Lyceum and Common Entrance Examination Papers for Maltese and their attendant Syllabi - Sandro Spiteri An Analysis of the Content and Cognitive Demands of the Maltese 11+ Mathematics Examinations C. Axiak a Coding of Mathematics Items - C. Axiak Report on the Religion Examination Papers for Admission into State Junior Lyceums and Church Secondary Schools A. Gellel The Junior Lyceum & Common Entrance Examinations: A Content Analysis of the Social Studies Syllabi and Examination Papers G. Baldacchino English, Evaluation of the Junior Lyceum and Common Entrance Papers 2006, Evaluation of the 2005 Syllabus J. Richardson Mathematics, Evaluation of the Junior Lyceum and Common Entrance Papers 2006 and Junior Lyceum specimen paper 2007, Evaluation of the Primary Mathematics Syllabus (published 2005) N. Rushton FES Programmes Record of Achievement Measures to improve students learning in Secondary Schools L. Grech

7 1.0 Introduction Part I 1.1 Purpose of the Review This Review stems from the Strategic Plan of the National Curriculum (2001) which presents three educational assessment targets that call for a review of the assessment process used in the transition from primary to secondary education. More specifically, the targets refer to the Junior Lyceum and Common Entrance examinations. Target T02.09 (p. 37) highlights the need to identify the changes required in these examinations to ensure that they conform to the new curriculum policies on teaching and assessment methods. T04.10 (p.36) focuses on the need to explore alternatives to the examinations. T04.13 (p.39) relates to decisions that need to be made on (a) benchmarking and (b) regarding policy on entrance examinations. The latter target follows on from T04.2 (p.28), which addresses external assessment (at the end of the third year of the primary school and the second year of the secondary school) and the creation of structures to research and make plans for benchmarking More recently, the policy document For all Children to Succeed (FACTS) (2005) stated that we need to work on how to eliminate the rather staccato if not abrupt manner by which students move from primary to secondary schooling in the state system (pp.25-26). In May 2006, the Minister of Education, Youth and Employment set up this Review Committee with the purpose of studying the reality surrounding the transition from Primary to Secondary in different types of school and to map out a plan of action for the way forward which will attempt to achieve the above mentioned inter-related targets. 1.2 Methodology The methodology of this review entitled Transition from Primary to Secondary Schools in Malta: A Review included the following: focused interviews with pupils, parents, teachers, Heads of School, College Co-ordinators, officials in the Education Division responsible for the syllabus, curriculum, examinations and processing of special requirements, officials responsible for the Common Entrance examination, policy makers, members of the Faculty of Education, the Private Schools Association and the School Councils Association, representatives of the different political parties and the Malta Union of Teachers as well as high Government educational officials and officials from the Archbishop s Curia (refer to Appendix 1). 7

8 1.2.2 The review also utilized findings of research studies, mostly undergraduate, carried out on the entrance examinations and other reports and publications where such issues are discussed. The input of local curriculum and subject specialists was sought in order to complement the quantitative analysis carried out on various aspects of the examinations by the committee. In two subjects, English and Mathematics, input was also sought from foreign subject specialists working in assessment and examinations at this level. The views of an international scholar on Church school education were also sought in the quest to gather as much valid data as possible in the twelve months available for this review process. 1.3 Structure of the Report The review report is divided into five sections as follows Background In this section, which is included in this chapter, a brief overview of the current situation surrounding the Junior Lyceum and Common Entrance examinations is given. Information is given on the numbers of children in the three educational sectors in Year 6 and Form I in order to quantify the proportion of pupils affected by these entrance examinations. This section also provides information on the number of candidates who apply for these examinations and the outcomes achieved. The aim of this section is to put the task of this review in context Review of the Rationale In this section, a study is carried out of the rationale of the 11+ examination and the educational context in which it operates. This task is carried out to make explicit the rationale of primary education in Malta and the selective mechanisms that encompass State and non-state schooling. The purpose of this part of the review is to determine the extent to which the rationale behind current practices in Malta and Gozo is still valid when also considering current transition and selective mechanisms in other European educational systems and beyond Impact Review The review proceeds with a study of the short and medium term impact on the different stakeholders and on teaching and learning, especially in the upper years of primary and throughout secondary education. The issues of accessibility of the examination and special requirements also form part of this review. Aspects of the longer-term impact are investigated by means of the SEC examination certification obtained by students who followed different education pathways in their secondary 8

9 education and the prospects of these students as reported in the annual tracer studies Quality Review This chapter of the review includes an analysis of various aspects of the quality of the entrance examinations. The aim of the analysis is to determine the soundness of these entrance examinations and their appropriateness in functioning as selective mechanisms. In particular, aspects of validity, reliability and fairness of the examinations are studied using various measures, including technical statistical analyses and evaluations carried out by curriculum and subject experts The Way Forward With the results of previous sections in hand, the Review Committee presents a plan of action for the way forward. A number of principles including those embraced in the NMC guide this plan. The plan is also guided by the developmental approach to primary education, which puts the learner at the centre of the educational experience. The plan focuses mainly on assessment but it also includes curricular and pedagogical implications, which go hand in hand. In this plan, national benchmarking becomes a necessary regulative feature in the Maltese education landscape in a context where standards need to be monitored and teachers and schools need to be supported. Part II 1.4 Background In Malta, there are three educational sectors: the State, the Church and the Independent (refer to Table 1.1). In Gozo, there are no Independent schools to date. The largest sector is the State sector in which all schools are non-continuous schools, that is, pupils need to transfer from a primary school to a different secondary school. All State primary schools are co-educational schools whereas all State secondary schools are single-sex schools. State primary schools are found in the different towns and villages in Malta and Gozo. Currently, there are two types of State secondary schools: Junior Lyceums and Area Secondary schools. In the State system pupils are admitted into the Junior Lyceum in their catchment area if they pass the Junior Lyceum (JL) entrance examination. Pupils who do not pass the JL examination or who do not sit for this examination are admitted to the Area Secondary school in their area. 9

10 Table 1.1 Number of Schools * in Malta and Gozo for State Primary Schools 61 State Secondary Schools 35 Church Primary Schools 20 Church Secondary Schools 21 Independent Primary Schools 13 Independent Secondary Schools The Church sector is the second largest educational sector in Malta and Gozo. It provides free education, although parents are encouraged to support the schools by giving an annual donation. There are both continuous and non-continuous schools in this sector. Church schools are mainly single-sex schools both at primary and secondary level although there are several co-educational non-continuous primary schools. In the Church sector, the issue of non-continuity affects the majority of boys secondary schools since most of the girls schools are continuous schools. Admission into most boys Church secondary schools is dependent on the pupils relative performance on the Common Entrance (CE) examination and the number of vacancies available The Independent sector has both continuous and non-continuous schools which are all fee-paying. The non-continuous schools are all primary schools. There are no schools in the Independent sector which only cater for secondary school students. There are single-sex and co-educational schools at both primary and secondary levels in this sector. Admission into an Independent school is noncompetitive The issue of transition from primary to secondary school is one that preoccupies many parents, children and educators in Malta and Gozo. Currently gaining access into a State Junior Lyceum or a boys Church Secondary school is dependent on high stakes examinations. There are two clusters of examinations based on the same set of syllabi issued by the Department for Curriculum Management of the Education Division: the Junior Lyceum (JL) entrance examination is organised by the State, specifically by the Educational Assessment Unit and the Common Entrance (CE) examination is organised by the Private Schools Association. * Here a school is defined as an educational institution which has a complete cycle of primary classes or secondary classes. Consequently State Primary Schools A and Primary Schools B have been considered as one school forming a complete cycle otherwise the number of State Primary Schools is 75. On the other hand Church and Independent schools which have both primary and secondary cycle have been considered as two schools. 10

11 It is necessary to point out that the CE examination is not used for entry into all boys Church secondary schools. A small number of schools still opt to set their own entrance examination. Others have an entry point at the primary level and pupils move on to the secondary sector of the school automatically as is the case in most girls Church schools. A small number of girls Church schools also set their own examination for entry at Secondary level with the exception of the only girls Church school in Gozo which makes use of the CE examination commonly taken by boys The reality is such that at the end of primary school, the majority of girls in State schools sit for one set of examinations: the JL examination, though a small number sit for two sets: the JL examination as well as the examination set by particular Church schools (in Malta) or the CE examination (in Gozo). The majority of boys in Malta and Gozo also sit for the JL examination. At the same time, around half of the boys cohort in Malta and Gozo also sit for the CE examination and some even sit for a third entrance examination set by particular Church schools. It is accurate to say that the majority of primary school children in Malta and Gozo experience highstakes examinations at the end of their primary cycle of schooling. The only exceptions are the children who attend Church and Independent schools that have both the primary and the secondary cycles of schooling The following set of tables provides a better understanding of the numbers of pupils in continuous and non-continuous schools in the different education sectors. Tables 1.2 and 1.3 illustrate the distribution of Year 6 primary pupils whereas Tables 1.4 and 1.5 follow the same cohorts the following year and give a similar breakdown using Form 1 statistics. These tables indicate the proportions of children in the different types of schools and how many are affected by the staccato system of transition. Table 1.2 shows that at primary level, non-continuity is a reality in the three educational sectors in Malta and Gozo. Table 1.3 shows that, on average, threefourths of all primary school children in Malta and Gozo attend non-continuous schools. Therefore, the issue of staccato transition affects the majority of Maltese and Gozitan families and is of major significance in the local educational scene. Tables 1.4 and 1.5 show the distribution of secondary school students in continuous and non-continuous schools in the different educational sectors. 11

12 Table 1.2: Primary Year 6 Pupils in Different Educational Sectors Schools 2002/ / / /2006 M F Total M F Total M F Total M F Total School Cohort State Church Continuous Church Non-Cont Independent Cont Independent Non-Cont Table 1.3: Primary Year 6 Pupils in Continuous and Non-Continuous Schools Schools 2002/ / / /2006 M F Total M F Total M F Total M F Total School Cohort In Continuous Schools % In Non-Cont. Schools % Table 1.4: Secondary Form 1 Students in Different Educational Sectors Schools 2003/ / / /2007 M F Total M F Total M F Total M F Total School Cohort State Sector Total State Junior Lyceum State Area Secondary Church Continuous Church Non-Cont Independent Cont Table 1.5: Secondary Form I Students in Continuous and Non-Continuous Schools Schools 2003/ / / /2007 M F Total M F Total M F Total M F Total School Cohort In Continuous Schools % In Non-Cont. Schools %

13 1.4.7 Tables 1.6 and 1.7 below provide information related to the JL and CE examinations for the period It is evident that the number of candidates for both examinations is on the decrease. This trend reflects the progressively declining birth rate in Malta and Gozo. Table 1.6 shows that in the past two years, the JL pass rate reached 60%. On average, there is no significant difference between the pass rate for boys and girls. The figures in Table 1.7 indicate that just over half the number of boys who sat for the JL examination sat for the CE examination over the period of the past five years. Table 1.6 also shows that the current vacancies in boys Church secondary schools allow around one third of the candidates to be offered a place in one of the Church schools. Once the results are published and the Church school vacancies are filled, the number of entries into Junior Lyceums is determined. From amongst boys who do not make it to a boys Church school of their choice, and from amongst boys and girls who do not make it in the JL entrance examination, a few opt to pursue their studies in the Independent sector. The great majority of those who are not successful in either entrance examination go to their Area Secondary school. Table 1.6: Number of Applicants and Pass Rates in the JL Entrance Examination M F Total M F Total M F Total M F Total M F Total Malta Gozo Total Passes %Passes Table 1.7: Number of Applicants and Entries in the CE Examination Malta Gozo Total Vacancies % Entries M F Total M F Total M F Total M F Total M F Total Lessons from the Past The present review cannot ignore previous reforms that took place in the late 1960s and early 1970s that affected the transition from primary to secondary 13

14 education. There are lessons to be learnt from the nature of the reforms and their implementation. It is therefore imperative that a very brief outline of the main reforms relevant to the current review is given and that important lessons learnt from that experience are elicited. More information about the period under review from different perspectives can be found in Zammit Mangion (1992), Sultana (1992), Scerri (2000) and Zammit Marmarà (2001) Transition from Primary to Secondary pre Between 1965 and 1969, candidates who passed the Secondary Entrance examination were admitted either to the Government grammar schools of which there were six; two for boys and four for girls, or to the Government Secondary Technical schools, of which there were four; three for boys and one for girls. Other students, including unsuccessful candidates could join one of the 27 feepaying private schools, most of which were owned and administered by religious institutions. A few unsuccessful candidates could also join one of a small number of Government craft and industrial training centres In 1967, the UNESCO expert, L.J. Lewis, who spent the summer of 1967 examining Malta s educational system, noted that at that time the demand for secondary education was intense and he predicted that it would increase (Lewis, 1967, para. 18). In support of this statement, Lewis gave figures of the numbers of candidates sitting for the secondary school entrance examinations from 1965 to 1967 and the number of passes. Table 1.8 below reproduces this data along with other data for 1968 to 1970 published in the Education reports of the respective years (Reports on the Working of Government Departments: Education). These data do not include the numbers of pupils who did not sit for the examination. Presumably, these pupils either opted to find employment after reaching the compulsory school leaving age of 14 or joined a fee-paying private secondary school. Table 1.8: Secondary Entrance Examination Summary Results Year Candidates Successful Unsuccessful % Passes

15 Professor Lewis (1967) reported that in , there was a total of 3755 pupils in Government grammar schools, 3715 pupils in various secondary technical schools and in craft centres, and 4700 pupils attending private secondary schools. By 1969, the enrolment in these schools had increased but certainly not enough to meet the pressing demand. At the beginning of 1969, the total number of pupils in Government grammar schools was 5050 (1658 boys and 3392 girls) and in Government secondary technical schools it was 4168 (2631 boys and 1537 girls). In 1970 Cameron, who had succeeded Lewis as UNESCO consultant to the Maltese Government, noted that in the 19 private secondary schools that belonged to the Malta Private Schools Association there was a total of 4208 pupils (2366 boys and 1842 girls) (Cameron, 1970). He also commented on the gender differences between pupils in the different school types; 2:1 girls to boys in Government grammar schools, 1.7:1 boys to girls in secondary technical schools and 1.3:1 boys to girls in private secondary schools Both Lewis and Cameron commented on the secondary school curriculum and on the teachers qualifications to teach at secondary level. 1 In all schools, the curriculum was geared to prepare pupils for the GCE O-level examinations, irrespective of whether the schools were designated as grammar, technical or private, and the more passes obtained the better; private tuition was used to offer additional subjects. The results showed that a number of pupils managed to obtain a minimal level of success in a large number of subjects Transition from Primary to Secondary from 1970 to the Present In 1970, transition from primary to secondary took on another meaning as secondary education was extended to all pupils. The secondary school entrance examination took place as usual and the 1,968 successful pupils were admitted to the Government grammar and secondary technical schools as in previous years. The other pupils who completed Standard IV and Standard V of the primary school were admitted into 30 new secondary schools; 14 for boys and 16 for girls. These schools began to function on 1 October 1970 and began to cater for the secondary education of 5460 boys and 5659 girls. With this innovation, 1 In 1967, out of approximately 876 teachers in government and private secondary schools, 45 held both a university degree and some form of professional teaching qualification, 111 held a university degree but no teaching qualification, while 324 held a primary school teachers qualification and 396 had various secondary level academic qualifications but no teaching qualification. 15

16 secondary education for all came into operation though there was a clear distinction between the selective schools and the new schools This innovation required considerable planning and preparation, which took place in four phases: (i) building of the statistical infrastructure for planning; (ii) planning the type of secondary schools, duration of courses, and revising the primary and secondary schools syllabi; (iii) selecting and converting 30 primary schools to serve as secondary schools, which meant providing equipment and allocating human resources, estimating transport and other costs; (iv) discussing and agreeing with the private secondary schools association on financial assistance, scholarships and the assessment system, and discussions with the teachers unions The reform continued in subsequent years with a number of reforms, starting with the abolition of all entry tests and examinations in 1972 so that the transition from primary to secondary in State schools became completely nonselective. The Government secondary school system was reorganized on an area basis, and in October 1972, there were 16 area schools for boys and 22 area schools for girls, with populations of boys and girls respectively An innovation in the secondary school curriculum occurred in with the launch of a new system of technical education at craft level for students over the age of 14 who wanted to learn a trade. In the first year, 640 boys were enrolled in five new trade schools in Malta and 64 were enrolled in Gozo, while another 15 students took the course in Agriculture. Plans were made to accommodate about 2000 students, including girls, in trade schools in the following year In , the Education report noted that all schools had remedial classes in the lower forms, the system of continuous assessment was maintained and the end-of-year examinations were re-introduced. New trade schools were opened and the population in these schools increased to 1756 boys and 129 girls in nine schools. The demand for places in these schools required the introduction of a new selection system for entry into trade schools and technical institutes. 16

17 In the same year, following the promulgation of the 1974 Education Act, the school leaving age was raised to 15 years and later to 16 years as from 1 September This regulation brought the school leaving age in Malta in line with that in various European countries. However, the 14-year-old students rebelled against it when they found that they had to stay on at school for another two years. The Education Report for noted that the the larger incidence of troublesome students, evidently resentful of the raising of the school-leaving age, coupled with the extra responsibilities and clerical work placed upon school administrations, strained some schools during the year. The response to these challenges was the introduction of formal annual examinations as from June/July 1975 in an attempt to motivate students in their studies and control their behaviour. Further changes in the curriculum and the grouping of pupils took place over the next few years The reforms had other repercussions. The number of pupils admitted to fee-paying private schools increased and the distribution of motivated and unmotivated pupils between Government and private schools became unbalanced. The establishment of the Junior Lyceums in 1981 as schools for high fliers can be seen as a measure to redress this balance. With the Junior Lyceums came the JL entrance examination, which in practice meant a return to the selective transition from primary to secondary of the late 1960s based on academic ability. Similarly, the Church secondary boys schools needed to find a non-discriminatory way of admitting pupils to the limited number of places available as the number of pupils seeking admission kept increasing. The result was the Common Entrance examination, which was held for the first time in In effect, with the JL and CE examinations, the transition from primary to secondary became even more selective and competitive for a large proportion of boys. Sections to above describe the current picture of how all pupils in the various types of school experience transition from primary to secondary Besides the changes in the mode of transition from primary to secondary, the closing of the Trade Schools in the late 1990s practically meant the loss of the vocational and technical track in secondary education and the return to the academic track as the only type of education at secondary level Achievements The main aim of the reforms of the late 1960s and the 1970s was the opening of secondary education for all pupils. The main thrust was on the academic track and less so on vocational education. One way of gauging the achievement of 17

18 this aim is by analysing the registrations by students for GCE Ordinary level examinations set by UK examination boards in the popular subjects -English Language, Mathematics and Physics- which have been offered for more than 50 years GCE O-level results obtained by Maltese pupils can be traced back to 1951, which is the first year when these examinations were offered. In that year the results obtained by pupils in the Lyceum for boys and the Girls Secondary Schools (Malta) were as shown in Table 1.9 (Vassallo, 1955). It is clear that only a select few sat for the examinations and none of the girls took Physics. The pass rates were 53.7% in English Language, 86.0% in Mathematics and 24.0% in Physics. Table 1.9 Results obtained by the Lyceum and the Girls Secondary Schools (Malta) in the Oxford GCE O-level examinations held in Summer 1951 Sat Passed Subject Boys Girls Total Boys Girls Total English Language Mathematics Physics Almost twenty years later, in Summer 1970, the number of students sitting and passing the GCE O-level examinations increased considerably, as shown in Table 1.10, though the pass rates in English Language and Mathematics decreased. Students preferred to sit for Oxford examinations rather than London examinations, especially in Mathematics and Physics, but many of them took a second chance to pass English Language by taking the examination with the two Boards. Table 1.10 Results obtained by Maltese students in the GCE O-level examinations held in Summer 1970 Oxford GCE O-Level Summer 1970 Subject Sat Passed % Passes English Language Mathematics Physics London GCE O-Level Summer 1970 Subject Sat Passed % Passes English Language Mathematics Physics

19 Following the reforms of the 1970s, the number of pupils who sat for the GCE O-Level examinations continued to increase as shown in Table 1.11 Between 1975 and 1990, registrations for GCE O-Level English Language increased by over 36% and for Mathematics by almost 53%. The increase in the number of registrations for Physics was even more dramatic with more than a 200% increase from 1975 to The subject was compulsory for admission to the Sixth Form between 1982 and From 1995 onwards, registrations for the GCE O-level examinations declined as pupils started to sit for the Secondary Education Certificate (SEC) examinations offered by the MATSEC Board of the University of Malta. The SEC examinations in English Language, Mathematics and Physics were first offered in 1992 in a similar format to the GCE O-Level examinations. From 1994 onwards, their format changed to the present one with differentiated papers. Table 1.12 shows how SEC registrations have increased since their introduction. Yet currently, about one third of the 16-year-old cohort either do not register for the examination or fail to obtain a certificate in any subject. Table 1.11 Registrations by Maltese students for UK GCE O-level examinations Year English Language Mathematics Physics Table 1.12 Registrations for Secondary Education Certificate Examinations Year English Language Mathematics Physics It is not easy to assess the achievements in vocational education as there is no agreed measure by which they can be gauged objectively. Clearly hundreds of pupils gained from learning a trade; many others did not benefit as much as expected. Sultana (1992) identified many factors that contributed to the 19

20 vicissitudes of the trade schools, among which were (a) a general deficiency of the technical culture that is required to value technical and vocational education and (b) the absence of an appropriate reward structure for those students who pioneered the new system Lessons to be learnt (i) The amount of research, planning and preparation required to bring about any major reform in education cannot be underestimated. Piecemeal reforms destabilize the system and carry a negative effect for a long time. (ii) An assessment system that had been operating for many years cannot be dismantled without it being replaced by another system which is seen to be at least as equally valid as the old system. (iii) Changes in the assessment system and their implications must be thoroughly understood by teachers, parents and pupils and have the support of all stakeholders. (iv) The curriculum must take into account the mix of pupils with diverse abilities, aptitudes and interests. The syllabi, the curriculum materials and the teaching methods must cater for diversity. (v) Schools require enough autonomy to develop policies, systems and structures that help them to adequately address network and community needs. (vi) All schools require different forms of support for individual learners and staff. (vii) The inclusion of technical and vocational education in secondary education requires considerable curriculum development, teacher training and the establishment of an appropriate reward or certification system with a comparable status to that of the academic certification system. 20

21 2:0 The Rationale of the 11+ Examinations 2.1 Introduction Selection for secondary education is deeply rooted in the Maltese educational culture. It can be traced to at least 150 years ago and the present review would be unwise to ignore its origin and development, and the changing rationale of the process of selection. This chapter therefore follows Paul Black s authoritative advice that the assessment and testing system within any given society can only be understood in relation to the history and culture of its educational and social systems (Black, 1998, p.22). The first section traces an outline of the early history of selection when secondary education was the privilege of a select few (refer to Appendix 2). It then presents the main developments that led to the provision of secondary education for all students after the end of primary education in The abrupt change from selection to a comprehensive entry to secondary education and the reforms of the 1970s are then outlined. The next section discusses the return to selectivity with the establishment of the Junior Lyceums in 1981 accompanied by the introduction of the Junior Lyceum entrance examination into Form 1. The third section presents a brief account of the development of the Common Entrance examination into a number of Church schools followed by a section about Independent schools, which provide secondary education to a significant proportion of year-olds. Another section analyses the principles and traditions that underlie the primary school curriculum. 2.2 The Purpose of Primary Education: the Preparatory Tradition Malta s educational system emerged first at secondary and tertiary levels and subsequently seeped down to the lower levels. This phenomenon has generated and reinforced what Blyth (1965) called the preparatory tradition in the education of primary school children. In this respect, it is highly significant historically that the Collegium Melitense, established in 1592, had a preparatory class to train pupils for secondary education. This occurrence set the pattern for future development characterised by the dominance of the preparatory tradition. Scarce attention was given to the developmental tradition of primary education, which will be discussed in a later section The preparatory tradition continued to be reinforced, at least at the level of ideas, with the introduction of mass schooling in the 1850s. In 1865, Sigismondo Savona delivered a lecture to the Maltese establishment entitled On the Necessity of Educating the People in which he began to articulate his rationale for selective 21

22 examinations. In 1881, he chaired a Commission, which he himself established, to suggest a scheme for the organization of the Lyceum. The Governor approved their report in April In his report as Director of Education in 1883, Savona expressed his belief in the opening statement of his account of the Lyceum. This institution, he wrote, was intended to complete the education given at the Government Primary Schools or in private schools, and, at the same time, to prepare candidates for admission to the Arts Faculty in the University (Savona, 1883, p.14). It is this firm belief in the preparatory function of primary schooling that is captured in the confident tone evident in his report of 1883: It is absolutely necessary to alter the present regulation concerning the admission of students to pursue their studies at the Lyceum, in order to maintain to that institution its character as a school for secondary instruction. No one shall be admitted therein, except after passing a searching exam in all the subjects taught in the higher classes of the city primary schools (pp.17-18) Savona legitimated this belief, which, as Director of Education, he translated into actual policy, by referring to four important reasons, which reflected both concerns and contingencies: (i) selection by examination was the current policy in countries like Britain, Prussia, Austria, Holland and Italy; (ii) non-selection would reduce a considerable portion of the Lyceum to the level of a primary school; (iii) there was lack of teachers to cope with increasing numbers of students, and (iv) the existing accommodation placed limits on the number of students who could be admitted This policy continued to be strengthened by the educational developments in the decades that followed. The following are the most conspicuous: (i) the detachment of the Lyceum from the University of Malta in 1913; (ii) the introduction of the scholarship examination in 1915; (iii) the establishment of the Central School (with an examination at point of entry) to offer secondary level of education to prospective primary school teachers in 1923; (iv) the amalgamation of the Elementary Schools Department with the Secondary Schools Department in 1932, and (v) the establishment of the Preparatory School by Dr Laferla in

23 2.2.5 However, the competitive dimension of the 11+ examination began to be obvious after the Second World War when the demand for education increased considerably. The Committee appointed by Dr Ganado, the then Minister of Education, to report on and recommend developments in the primary sector of education in 1948, re-affirmed the selection of pupils at 11 to ensure that successful candidates are able to follow a Grammar School course (Department of Education, , p. xxxv). They also recommended the extension of the opportunity to sit the examination to those beyond 11 years of age but not yet 14 in order to offset the handicap which pupils may have suffered through part-time attendance in the primary schools ( , p. xxxv). Although they recommended that the concession was an emergency measure and should be withdrawn when a return to normal conditions has been effected ( , p. xxxv), it continued to be established policy until 1970 when secondary education for all was introduced. During this period the subjects tested were Arithmetic and English Discussions about selection at 11+ re-emerged in the late 1960s. Cameron, the educational expert who was advising the Government on the implementation of secondary education for all at the time, pointed out that: The secondary education for all scheme is only one stage in a long process of development. It is a big one, but it merely continues the process of expanding secondary education which began at least two decades ago. Nor does it end the process. It is not the final solution. It will be amended and redefined again and again in the light of experience let us first put secondary education for all into practice, ensuring that it remains flexible enough to be easily adapted to meet the further need for changes which will inevitably occur (Cameron, 1970: para.3b.2) This was, in fact, the official position of the Government as evidenced by the ministerial statement during the parliamentary debate (Parliamentary Debates, 23/3/70, p. 5742) and the Minister s own intervention during a seminar organized by the MUT. An article in the April-September 1969 issue of The Teacher (p.3) reported that the Minister stated the form we have chosen will lead in its own good time to a comprehensive system. That good time came sooner than expected because in 1971 there was a change in Government and soon afterwards, in 1972, the 11+ examination was abolished and a comprehensive system of education was introduced. The year 1972 also saw the abolition of Annual Examinations in State schools. 23

24 2.2.8 Indeed, the 1970s brought about further and far reaching educational reforms such as the opening of Trade Schools (1971 ff.) and an emphasis on technical education; the rise in the age of obligatory education from 14 to 16 (1974) 2 ; the opening of Kindergarten centres for four year olds (1975) and later for three year olds (1988); the introduction of the teaching of Arabic which became an obligatory subject (1974), (by 1982, Arabic and Physics became obligatory subjects for admission to Sixth Form); the re-introduction of examinations in Primary and Secondary schools (1975/6); the introduction of strict streaming as from Year 1 Primary (1975/6); the introduction of co-education in Primary schools (1975); and the introduction of Primary schools A and B (1976). The Primary cycle now became six years long (Years 1 to 6). Many objected to the reforms and also to the fact that they were introduced during a relatively short period of time. One of the repercussions of these reforms was the rush for entrance into private schools (Church and Independent) A most significant reform was that of 1981 when Government decided to abolish the comprehensive system and re-introduce selection. This decision highlights the cultural embeddedness of selection in our educational system as well as the belief of policy-makers that a successful educational policy has to be based on strict selection at the age of The Establishment of the Junior Lyceums On 18 March 1981, Malta s Minister of Education issued the following statement: Il-Gvern Soċjalista Malti minn dejjem sostna li jemmen u jrid li ulied ilpoplu, speċjalment dawk li l-ġenituri tagħhom ma jistgħux iħallsulhom miżati, jilħqu l-ogħla livelli ta edukazzjoni mingħajr ħlas. [ ] Il- Gvern jaf li għad hemm bżonn li jsaħħaħ l-iskejjel sekondarji billi jagħżel l-aħjar għalliema u l-aħjar studenti, subien u bniet, u jiġborhom f Junior Lyceums (għas-subien u għall-bniet) wara eżamijiet kompetittivi, jiġifieri jidħlu biss dawk li huma l-aħjar minn eżaminaturi The Maltese Socialist Government always insisted that it believed and wanted the children of the people, especially those whose parents cannot pay fees, to reach the highest levels of education free of charge. [...] The Government knows that it still needs to strengthen secondary schools by selecting the best teachers and the best students, boys and girls, in Junior Lyceums (for boys and girls) - through competitive exams - as much as possible by foreign 2 The Compulsory Education Ordinance of 1946, described by Zammit Mangion (1992) as one of the most important pieces of educational legislation in the history of Maltese Education and an important milestone in its development (p.58) had made primary education compulsory for all Maltese children between the ages of 6 and

25 kemm jista jkun barranin. Isir kull sforz biex dan il- Junior Lyceum, ilbenniena tas-sixth Form, jibda jaħdem f Ottubru li ġej. examiners which means that only the best will enter. Every effort will be made so that this Junior Lyceum, the cradle of Sixth Form, will be open by coming October. In 1981 Government set up the Junior Lyceums to offer high quality schooling to those who had the potential to advance academically. This was free of charge and therefore available to those who could not afford to attend private schools. In other words, Government presented the Junior Lyceums as an alternative to private schools One aspect of the rationale behind the setting up of the Junior Lyceums in September 1981 is further clarified in the following draft of a press release, which is neither signed nor dated, but which was definitely written after 3 April By that date, the decision to set up the Junior Lyceums had already been taken and earnest preparations were under way to hold entrance examinations in order for students to enter into the different Forms. Bħalma diġà tħabbar, il-gvern se jiftaħ żewġ Junior Lyceums mis-sena skolastika 1981/82. Dawn l-iskejjel, waħda għall-bniet u l-oħra għassubien, sejrin jiġbru fihom l-istudenti ta l-akbar ħila u intelligenza minn Form 1 sa Form 5 tal-iskejjel sekondarji. Huma għalhekk skejjel għal specially gifted fejn it-tagħlim jiġi pprofondit kif ukoll imwessa skond l-intelliġenza tat-tfal u huma miftuħa għat-tfal kollha ta Malta. As already announced Government will open two Junior Lyceums as from scholastic year 1981/82. These schools, one for girls and one for boys, will cater for students of the highest ability and intelligence from Form 1 to 5 of secondary schools. Therefore they are schools for the specially gifted, the curriculum will be deep and broad in relation to the intelligence of the children and they are open to all Maltese children. Those who set up the Junior Lyceums had great expectations for their students. [ ] entrance into the different Forms will be based on the ability of the candidates, who certainly will be able to follow with great success courses for the G.C.E. and/or the Ordinary Level Maltese Matriculation Documents contemporary to this draft press release make it absolutely clear that the Junior Lyceums were set up to provide the optimum education for the best and most talented Maltese students. Writing on 6 April 1981, the Assistant Director 25

26 of Education remarked that, Government has decided to create the Junior Lyceums which [are] intended to cater for the education of the cream of the best school children coming from Secondary Schools. This reference to Secondary schools was made because the Government intended to open the Junior Lyceums to all Forms from the very beginning. Therefore, apart from the examinations that were intended for entry into Form 1, four other sets of examinations were planned for entry into Form 2 to Form 5. These examinations were highly competitive because of the Government s intention to choose the very best. Places were therefore limited, and this rendered the whole idea of the setting up of the Junior Lyceums a highly selective exercise. At one time, mention was made that only one hundred and eighty (maybe two hundred) boys, and the same number of girls should enter the Junior Lyceums in each Form. However, there must have been some rethinking because, in actual fact, the number of those who entered the Junior Lyceums was considerably higher than one thousand eight hundred. According to the Annual Education Report of 1981, a total of 8168 students from Malta and Gozo applied to sit for the entry examination at the different levels (Forms 1 to 5), that was held in June The number of students admitted into the different Forms in Malta and Gozo in September 1981 totalled 2430 (29.75% of applicants). The greatest number of entries was into Form 1 (767 students) and, as expected, the smallest entry was into Form 5 (98 students). Many students and parents must have felt disappointed with the results but the Report mentioned above made it very clear that the aim of these schools (the Junior Lyceums), entry into which is selective, is to provide accelerated education for students with high intellectual and academic abilities Certain measures had to be taken to live up to the high expectations of the Education authorities. (a) New syllabi were devised for these schools. (b) The school time-table was to consist of thirty-seven, three quarter hour periods a week (as opposed to thirty-five three quarter hour periods for the rest of the secondary schools). (c) Thirty-one expatriate teachers were recruited to teach in the Junior Lyceums alongside Maltese teaching staff. Teachers were recruited from various countries including the United Kingdom, Italy, Saudi Arabia, Germany, France, and Poland. 26

27 The sites chosen to house the new schools were: (a) the ex-services school at Tal-Ħandaq for the Boys Junior Lyceum, and (b) the Girls Secondary School at Blata l-bajda for the Girls Junior Lyceum. In Gozo, the new Junior Lyceums were attached to the Victoria Boys and Victoria Girls Secondary schools respectively. In order to accommodate the Girls Junior Lyceum at Blata l-bajda, the population of this Secondary school had to be transferred elsewhere. Other children in other schools were uprooted and, moreover, new Secondary schools for girls had to be opened Not everyone agreed with the opening of the Junior Lyceums. On 8 April 1981 a meeting was held between Education Officials and the Malta Union of Teachers (MUT). The official side explained the new policy of the Government regarding the setting up of the Junior Lyceums and asked for the reactions of the MUT representatives. Union officials stated that this was the first time they were meeting the official side to hear of the new policy and that they could not give their reactions there and then. However, it was reported that MUT officials stated that prima facie they were against the new Junior Lyceums because: (a) The MUT had its own ideas on education that it had expressed way back in (b) The MUT was against the selection of the best teachers mentality for these schools, as the adjective best implies that the rest were being left in Secondary schools because they were not good enough for the Junior Lyceum. This would have a negative effect on the schools, the staff, and the parents of Secondary schools students. (c) The MUT would prefer the selection of staff to be made by open application, and it would then be up to the Department to recruit the most suitable teachers. (d) The MUT was concerned that with the new Junior Lyceums and with the incentives given to students who entered Trade Schools, the Secondary schools would suffer. The MUT stated that there was the need to avoid a negative effect on the students remaining in Secondary schools. (e) The above facts also meant, stated the officials, that there was a need for changes in the curricula of Secondary schools. Moreover, these schools needed to be given greater support in order to maintain the good reputation that they had recently been building. (f) There was a need for stability in education as there had been too many changes lately. Stability breeds confidence, officials were reported as saying. 27

28 In a somewhat unclear statement, MUT officials were reported to have said that they would not obstruct the Government s decision to set up the Junior Lyceums, and that they would give it their support in so far as this was required within their stated ideas and proposals for educational reforms Having originated from the MUT, this last comment must have been greatly welcomed by Government and Education officials, as there were pressing decisions of an educational nature that had to be taken. One such decision was whether students should enter the Junior Lyceums after Year 6 or after Form 2 Secondary. This issue was debated during a meeting held on 25 March 1981 in the presence of the Minister of Education. Arguments were brought forward in favour of entry after Year 6 Primary. It was stated that: (a) students would not have to change school again after two years, (b) there was greater probability of attracting students from Private Primary schools, (c) there were less complications in planning for the implementation of the project considering the short time available: April August Two arguments against entry after Year 6 Primary were also considered. It was stated that: (a) Year 6 was too early an age for selection, though this might have been offset by allowing failures to try to enter the Junior Lyceum in the following year. This time round, however, they would have to apply for Form 2. (b) Entering the Junior Lyceum at Form 1 would tend to determine what was taught in the Primary school. It is clear that education officials were very much aware of the possible negative consequences of selection at the end of Primary education, yet the decision was taken that the intake for the Junior Lyceums would be at Form Another pressing decision concerned which subjects should be examined. During their annual examinations Year 6 pupils were being assessed in five subjects, namely Maltese, English, Mathematics, Social Studies, and Religion. In a Progress Report dated 9 April 1981 it was noted that: The selective examination for entry into the different forms for 1981 will be in English, Mathematics and Maltese for each form. This admission requirement was actually adopted as from 1981 so that entry into all Forms of the Junior Lyceum was through entrance examinations in English, Mathematics and Maltese. This requirement continued for a number of years. 28

29 2.4 Developments after Over the years, the percentage of successful candidates sitting the Form 1 Junior Lyceum Entrance Examination started to increase, reaching per cent in 1987 (Report on the Working of Government Departments: Education Annual Report, 1987), which was a 13 per cent increase over the previous year. Such a high percentage of passes stretched to the limit the resources of the existing Junior Lyceums, since, as had already been noted in the 1986 Education Annual Report, the Junior Lyceum Entrance examination was no longer a selective but became a qualifying examination. This meant that any candidate who obtained the necessary pass mark a Grade C in each of the subjects of the examination - was entitled to a place in a Junior Lyceum. In 1982 a new Junior Lyceum for girls opened at Mriehel and one for boys opened the following year at Hamrun. It was becoming very clear, though, that the original concept of a school for high flyers was put aside at an early stage in the development of the Junior Lyceums In 1987, Government announced that as from 1988 the Entrance Examination into Form 1 was to include Social Studies and Religion; due regard, however, was to be given to freedom of conscience. These changes were in fact carried out. In 1988 it was also decided to once again extend the possibility of entry into the Junior Lyceum to students already in Form 1 of the Area Secondary schools. Entry into all other Forms in the Junior Lyceum was to take place as from Another measure having effect on primary education was the Education Act of 1988 which lowered the age of compulsory schooling from 6 to 5. This meant in effect that the transition age from primary to secondary could be as low as 10 years 5 months During these years the comprehensive system was not generally debated though by no means forgotten. The authors of Tomorrow s Schools (1995, pp.5, 23, 29 & 43) and of the Draft of the National Minimum Curriculum (NMC) (1998, p.18) recommended its implementation while at the same time ensuring that the different learning styles and needs of students are catered for (Tomorrow s Schools, p.9). The Draft of the NMC maintained that examinations at the end of Year 6 were to be held for the purpose of benchmarking and not for selection (p.20). However, this recommendation was regarded as too radical (Borg and Mayo, 2006, p.39) and it was drastically altered in the final NMC document. This document stated that In the absence of different arrangements that would be of greater educational 29

30 benefit, in State schools the final examination at the end of Primary school and Junior Lyceum Entrance examination will be set by the Education Division (NMC, 1999, p.78) The debate surrounding student groupings in schools has not been extinguished. Indeed, hopes were raised once more with the publication of For All Children To Succeed (2005) and the establishment of the College System. However, up to now selection is still in effect since the Junior Lyceum and the Common Entrance examinations are still in place. 2.5 Review of the Rationale of the Junior Lyceums Twenty-six years have passed since the establishment of the Junior Lyceums. The percentage of successful candidates has continued to grow topping the 50 per cent mark in 1996 and peaking at per cent in 2005 (the percentage for 2006 was 60.23%). This high percentage of entries led many to the logical conclusion that the Junior Lyceums are no longer catering solely for high flyers as was one of the original motives for their establishment. Moreover, Church schools are no longer fee paying. Therefore, even the second motive, that is, the provision of good quality secondary education free of charge in contrast with the education provided by the fee-paying private schools, is no longer valid The re-introduction of the selective examination in 1981 was partly a confirmation of how deeply embedded in Maltese culture was the concept of preparatory curriculum, that is, the view that the Primary years are a preparation for secondary education. The experiences of the past twenty-six years suggest the need to re-think the rationale of the approach to primary education and the process of transition from primary to secondary education. 2.6 The Common Entrance Examination Church Schools and The Common Entrance Examination Church schools in Malta and Gozo from Kindergarten right up to sixth form have been educating Maltese children for quite a number of years, in fact some of them have already celebrated their first centenary of existence (Scerri, 2000, pp.13-20). When in 1946 the law binding every child under the age of 14 years to attend school was enacted, there were not enough State schools and Church schools 30

31 filled the void. The Government supported them with financial aid up to the 1970s (Scerri, 2000, pp.19-20) In the 1970s, and even before, Church schools were all fee-paying schools except for one and all enjoyed great autonomy. Some had both the primary and secondary cycles and consequently their pupils enjoyed continuity from one level to the other. In the primary level some Church schools were quite small and consisted of two, four or five classes, i.e. one class for two, four or five grades only. Other schools were a full primary and still others were secondary schools with one or two primary (prep) classes. Still another school was a full secondary only. Each school had its own curriculum, entry points and entry criteria. Pupils could join a school either at Year 2, Year 4, Year 5, Year 6 or Form 1. Pupils could also move from one school to another after Year 2, Year 4, Year 5 and Year 6 and finally settle in another school in Form 1. This situation had a negative effect on primary education The Common Entrance Examination for entrance into Form 1 Church Boys Schools came into existence in 1978, six years after the Government had abolished the State Secondary Schools Entrance Examination. The Common Entrance Examination was intended to replace the many entrance examinations into Form 1 held by individual Church boys secondary schools. It was organised by the Malta Private Secondary Schools Association (later renamed the Private Schools Association - PSA). Candidates from Church, State and Independent schools were eligible to sit for this examination. Five schools (De La Salle, St. Albert the Great, Stella Maris, Mount Carmel and St. Augustine) participated in this examination in However, the examination results were not the only criterion for entry as successful candidates were accepted on a first come, first accepted policy and the Head of each school retained the right to accept or reject any successful candidate The following year, 1979, St Paul s Missionary College joined the Common Entrance Examination and the criteria for entry were revised to include only the result obtained and parental choice. On the application form the candidates indicated their school preferences in rank order. The candidates had to sit for three examinations, namely English, Maltese, and Mathematics. The result was published as a global mark. Successful candidates were assigned to the school of their preference as indicated on the application form, places permitting. 31

32 This system, which remained in existence till 1989, had several drawbacks. Firstly, schools continued to hold their entrance examinations into Year 2, Year 4, Year 5 and/or Year 6. Secondly, the places made available were always limited as most participating schools had their primary level or Year 6 classes. Pupils in these schools moved to the secondary level of the same school after completing Year 6. Thirdly, successful candidates listed on popular schools with limited vacancies very often found it difficult to obtain a place in another school. Each school formed its own rank order made up of successful candidates who had chosen it as their first preference. A school would call successful students who had chosen it as their second preference only if it still had places available after exhausting its first list The 1991 Church Schools Agreement The Common Entrance Examination came into existence when the Church Schools issue was at the forefront of Maltese politics and two years after its introduction, Government introduced the JL examination. The Government tried to impose several conditions on Church schools, foremost amongst them were free education and non-discrimination in accepting students In order to guarantee non-discrimination, Government proposed a common national entrance examination for State and Church Secondary schools. Government insisted that even pupils in continuous schools had to sit for such an examination (Scerri, 2000 pp ). The Church Authorities, the Private Schools Association and the Federation of Parent Teacher Associations not only pointed out that Church schools were not elitist but also protested strongly against a national entrance examination as it disrupted the educational experience of boys and girls in Church Schools that had both junior and senior levels. This was considered as the principle of continuity. This principle applied to all girls secondary schools and to three boys schools During the 1970s and more so during the 1980s, intense discussions were held between State and Church Authorities on the issue of Church Schools. The technical issue of the harmonization of the syllabi of the JL entrance examination and the CE examinations into Form 1 was quickly resolved. In previous years, the PSA had published a short syllabus for the CE examination, but in January 1985 representatives from the Church Schools and the Education Department met in three groups to formulate identical syllabi for Maltese, Mathematics and English 32

33 (Scerri, 2000, pp ). Since then, the CE and JL examinations have been based on the same identical syllabi. This has been formally stated in the regulations of the CE examination since The Church Schools agreement between the State of Malta and the Holy See, signed in 1991, brought to a close the Church Schools dispute. The issue on entry criteria was resolved by Article 3 of the 1991 Agreement. Church Authorities are free to establish criteria for admission into Church schools as long as they are non-discriminatory. Some places for less privileged categories may also be reserved. Since then, pupils have been accepted into Kindergarten and Primary Church schools according to the following criteria: children from Church homes, siblings in the same school, special cases, and the rest by ballot Entrance into Form 1 continued to be realised through the Common Entrance Examination. Children in primary schools which had a secondary level proceeded to the next level without undergoing a high-stakes examination Article 4 of the Agreement also established that the Church continue the process of re-organization so that schools would have complete cycles of Primary and/or Secondary education. This meant that each school had to have enough classes to form a complete cycle, i.e. six classes for a primary and/or five classes for a secondary as a minimum. Consequently schools with an incomplete primary cycle were either phased out or extended their classes to form a complete cycle. Secondary schools with prep classes (Years 5 and 6) phased out these classes. As a result of this process, the previous situation of multiple entry points, in Year 2, Year 4, Year 5 and Year 6, in order to secure a place in one s secondary school of preference, practically disappeared. This change, however, brought about an increase of places available at Form Development of the Common Entrance Examination In 1988, Government decided to introduce examinations in Social Studies and Religion as part of the JL examination and the PSA immediately followed suit In 1990, the PSA re-organized the way parents expressed their choice of school. In the previous system parents were asked to indicate their school preferences on the application form before the examination took place. In the new system parents are invited to make their choice of school after the results are 33

34 published. All the candidates are rank-ordered following an order of merit, highest global mark first - the global mark being an aggregate of the marks obtained in the subjects examined. The results are published as a list showing the index number, marks obtained for each subject, global mark and rank order. This proved to be a positive change which streamlined enrolment into Form 1 enormously The schools which participated in the 1990 CE examination were St Paul s Missionary College, St. Elias College, Savio College, St. Albert the Great College and the Sacred Heart Minor Seminary, Gozo. The Minor Seminary, Malta, joined the CE examination in 1991 while St. Aloysius College and St Augustine College joined in St Michael School joined the CE in 1997 and the Bishop s Conservatory School in Gozo for girls joined in As the number of candidates was increasing, it was necessary to ensure a better spread of candidates within the rank order, so it was decided that as from 1993, each examination be marked out of a maximum of two hundred marks instead of one hundred marks. Exemption from the examination in Maltese was first expressed in the 1994 Regulations while the 1995 Regulations included exemptions from Social Studies and Religion. The regulations of May 1994 also informed candidates beforehand that they could answer the Religion and Social Studies examination questions in English In 1994, the PSA Examination Board decided to provide particular examination arrangements for candidates with dyslexia. In 1997, this provision was extended to include all those with certified educational needs. The arrangements most commonly granted in the CE examination have been: extra time, reader, prompter and scribe. In this way the examination became accessible to a wider range of candidates. The regulations of 1994 introduced the granting of a number of places on Grave Humanitarian grounds. St. Augustine, St. Aloysius, St. Elias and Savio College offered a number of places under this new scheme. Parents were invited to inform the Grave Humanitarian Cases Committee if they thought their child could be considered as a Grave Humanitarian Case. Candidates accepted as Grave Humanitarian Cases by the Committee were exempted from the examinations and were accepted into one of the schools. After discussions with the Archbishop, in 1996, it was decided that all schools participating in the Common Entrance 34

35 Examination accept students under the Grave Humanitarian Scheme. Over the past three years ( ) the Grave Humanitarian Board received 40, 57 and 56 requests for consideration as Grave Humanitarian Cases. The schools accepted 19, 24 and 27 of them respectively At first, the examinations were held over a three-day period: Thursday afternoon, Friday afternoon and Saturday morning. The introduction of the Religion and Social Studies examinations increased the pressure on the young candidates as they were required to take five examinations over a three-day period. Consequently, in 1995 the examinations were spread over four days (Wednesday to Saturday), and as from 1999 the examinations were spread over a five-day period - at first from Tuesday to Saturday and then, since 2002, from Monday to Friday. Currently, the examinations are still held in the afternoons Church Schools Today The Introduction of this Review presents an analysis of the number of children who are educated in continuous and non-continuous schools. This analysis is continued here vis-à-vis Church schools. Tables 2.1 and 2.2 show the percentage of students (boys and girls) following their education in continuous and noncontinuous Church schools in Year 6 and Form Table 1.2 (p.6) and Table 2.1 clearly indicate that at the primary level there are more pupils being educated in continuous schools. Over the four-year period studied ( ), an average of 63 per cent of all Year 6 Church primary pupils (boys and girls) attended continuous schools. This is due to the fact that the number of places available in girls continuous schools is considerably high: over the four-year period, the average percentage of girls in continuous schools was 78 per cent whilst 22 per cent were in non-continuous schools. In comparison, during the same four year period, the average percentage of places available for boys in continuous schools was 44 per cent while for boys in non-continuous schools it was 56 per cent. Table 2.1: Percentage of Year 6 Pupils in Continuous and Non-Continuous Church Schools 2002/2003 Continuous % Non-Continuous % M F Total

36 2003/ / /2006 M F Total M F Total M F Total Table 2.2: Percentage of Form I Students in Continuous and Non-Continuous Church Schools 2003/ / / /2007 Continuous % Non-Continuous % M F Total M F Total M F Total M F Total Table 1.4 (p.6) and Table 2.2 show that although the number of places for boys in non-continuous schools increases at the secondary level, the number of students attending continuous schools is still slightly higher with an average of almost 58 per cent over the four-year period studied. This is due to the fact that the proportion of places available for girls in continuous schools remains high. In examining the proportion of girls in continuous and non-continuous schools over the four-year period, one finds that the average percentage of the former is 87 per cent while that of the latter is only 13 per cent. On the other hand, the average number of places available for boys in continuous schools is 31 per cent while for boys in non-continuous schools it is 69 per cent. Overall, Church schools accept more pupils through non-selective examinations The adoption of the ballot system and the consolidation of the Common Entrance examination as a transparent and objective means of enrolling children into Church schools have confirmed the Church s intention of catering for 36

37 children from all social classes. However a new criticism has arisen. Those schools which accept students through the Common Entrance examination cater for the more able students and consequently also deprive other schools of a better mix of students. Over the past two decades, Church schools continued improving their education through an increase in teaching, support and administrative staff and better, sometimes larger and newer, premises and more facilities. At the same time, classes in primary Church schools and consequently those with continuity at the secondary level have increased the mix of their pupils by the introduction of the ballot system All Church schools, including the ones accepting students through the Common Entrance Examination, accept students with disabilities. Inclusion has become a common feature of all Church schools. Church schools are doing their best to respond to the needs of their students, to implement the National Minimum Curriculum and to keep on fulfilling their vocation. 2.7 The Independent Schools Independent Schools in Malta form the smallest, yet vibrant, sector within the Maltese Educational system. Independent Schools also sometimes called Private Schools, are those schools that are run by lay educators and neither by the State nor by the Church. These schools are fee paying though parents are given a tax rebate on the fees paid The type of governance of these schools varies, ranging from enterprising educators, families, Boards of Governors, to Parents Foundations. These entities have run these Independent Schools since their inception. Chiswick House School founded in 1905/06, probably the oldest Independent School still in operation, has just celebrated its 100 th Anniversary. St Catherine s High School will celebrate their centenary in 2009 and St Edward s College will turn 80 in San Anton School and St Michael s School opened their doors in 1988, San Andrea School in 1990, St. Martin s College in 1993 and St Michael s Foundation (catering for the Senior School component of St Michael s School students) in Newark Junior School started operating in 1996 and Senior School in Verdala International, the only 3 In the year 2001, Government introduced a tax rebate on the fees paid for Independent Schools: Lm200 on the Primary school fee and Lm300 on the Secondary school fee (per child). In 2007, Government has announced an increase in the tax rebate. 37

38 International school in Malta and Gozo, was established in These are the Independent Schools that offer their students a continuous educational experience The possibility of progressing from one level to another without having to sit for competitive examinations mid-way in the educational programme seems to be one of the reasons that have contributed towards the growing popularity of these schools. All schools in the Independent Sector are fee-paying. Entry into these schools takes place on a first-come-first-served basis, with many of these schools giving siblings preference. Once a child is registered, then progression between levels is automatic. Independent schools are inclusive schools and offer a range of support facilities for learning difficulties, which varies from school to school Another factor that seems to have contributed towards the growing numbers in Independent Schools is the fact that they are co-educational institutions. Chiswick House School and St Martin s College, San Anton School, San Andrea School, St Michael s School and Foundation, Newark and Verdala International have been co-educational schools since they were founded. St Edward s College is still a single-sex Boys School, though its Sixth Form was co-ed. St Catherine s High School, which was once a single-sex Girls School, now educates girls and boys in the Early and Junior Schools and as from October, 2007 it will become a fully fledged co-ed school, which will see boys joining the girls in their Senior classes. The Accelerated Christian Academy also educates boys and girls at most levels Within the parameters set down by the National Minimum Curriculum, these schools have developed educational programmes that are not conditioned by highrisk examinations. Whether the pupils pass or fail their Junior V or Junior VI (or Year 6/7) examinations, they automatically proceed to their Middle or Senior Programmes. Content and assessment procedures are planned according to level requirements. All these Independent schools use continuous assessment, Mid- Yearly and Annual Examinations procedures. Some schools have also developed a detailed Profiling System Apart from these schools, the Independent Sector also encompasses other primary schools that prepare their pupils for the CE and JL Examinations. Thi Lakin, established in 1970, Garendon School, (1987), Little Angels School (1986) and Mariam Al Batool School act as feeder schools to Junior Lyceums and Church Secondary Boys Schools, within a co-ed setting. 38

39 2.8 Conclusion The foregoing observations in this review of the rationale of the 11+ examinations put into question the validity of separating children at the end of Primary education for the following reasons: (a) In practice, the JL examination is excluding a large proportion of pupils from a quality secondary education and labelling them failures at an early age. (b) As pointed out by the MUT back in 1981, the flow of pupils to the Junior Lyceums is having very serious negative effects on Secondary schools which are left with unmotivated students who have been traumatised by the system. (c) The concern expressed in 1981, namely that entering the Junior Lyceum at Form 1 would tend to determine what is taught in the Primary school is still very valid. Subjects which are not examined for entry in the Junior Lyceum are not being given importance in Year 6. Teachers, pupils and parents tend to concentrate on the five subjects that constitute the Junior Lyceum examination In addition to these observations, it is important to consider international practices at a time when Malta is aspiring to achieve the Lisbon Objectives in Education of less than 10 per cent early school leavers and at least 85 per cent continuing post-secondary education. In this respect, the following should be noted: (a) The PISA survey (2003) showed that The more and earlier students are divided into separate groups according to their academic performance, the more students socio-economic background matters for their academic performance. Therefore, early separation clearly places children coming from a poor family background at a disadvantage and at a greater risk of failure. (b) Furthermore, the age at which segregation occurs seems to be [a] factor contributing to a country s overall performance such that countries in which differentiation occurs at a later stage perform better than countries in which students are segregated at a young age (Grima and Farrugia, 2006, p. 63). (c) Nation-wide selection by examination at the age of 11 has been abolished by all countries in the EU. 4 The only country that has a selective examination at 11+ is Singapore (Grima and Farrugia, 2006). 4 The Government of Northern Ireland has decided to abolish the 11+ examination as from

40 3.0 The Impact Review 3.1 Introduction In Malta and Gozo, the current practice of having different criteria for accessing different types of schools has short and long-term implications for students, families, educators and society at large. The experiences of schooling vary considerably in the different types of schools. Such experiences affect the students life in several ways and are directly linked to their certification at the end of secondary schooling and their future prospects and life chances. This section of the review attempts to describe and quantify the impact of the current selective system on the different stakeholders. The data were collected from the numerous interviews carried out by the 11+ Review Committee (refer to Appendix 1) and other data recorded in local research studies 5 and other publications. 3.2 Impact on Children Preparation for the entrance examinations impacts directly on the children s physical, psychological and emotional well-being. Cassar s (1991) observations in a Year 6A class, led her to conclude that the JL examination was in fact `the phantom of the classroom. She observed that: The examination haunted the pupils like some monstrous phantom. Whenever the pupils made a mistake, or failed to achieve as much as the teacher wanted them to, they were reminded threatened even, that they had better do as they were told for the sake of this examination (p. 36). She observed that all that is taught is taught for tests and examinations hardly anything is taught for life (p. 42). Mansueto (1997) who conducted a similar study six years later came to a similar conclusion: In such a context, the very nature and goals of education are perforce thwarted as teaching and learning become instrumentalised, tied too narrowly to achievement, where the joy of discovering and understanding the world becomes subordinated to the task of `getting through a final examination. Information is just collected and stored away to be recited at a later stage, very much in a parrot-like fashion (p. 189). Cassar (1991) observed that often the lessons spilled over into the break and the pupils were hurried to eat their lunch. There never seemed to be enough time to 5 Mosty undergraduate B.Ed (Hons.) dissertations and a few post-graduate research studies. 40

41 finish a task within the expected timeframe. Her interviews with the pupils confirmed that the children always felt hurried from one activity to another. The interviews conducted by the 11+ Review Committee with Year 6 pupils confirmed that their school days were dominated by the examinable subjects and that they had to rush activitites: Ħafna ħin għall-maths, ftit Religion fil-għodu, imbagħad Malti, Ingliż u Social Studies wara nofs in-nhar Social Studies mhux dejjem inlaħħqu s-sillabu kollu. Għalhekk it-teacher kienet tgħidilna nistudjaw id-dar u mbagħad hi ssaqsina l-mistoqsijiet. Lots of time spent on Maths, some time on Religion in the morning. Then Maltese, English, Social Studies in the afternoon. The Social Studies syllabus is not always covered completely. That s why the teacher used to ask us to study at home she would then ask us questions In Duca et al. s study (1999) the parents complained that the system left too little time for children to practise extracurricular activities inside and outside school. Mhux qed ikollhom ħin għall-pe, Music, Drama u ħruġ kulturali (p. 54). They are not getting time to do P.E., Music, Drama and cultural outings. In our interviews, the pupils confirmed this state of affairs: Suġġetti oħra bħal kraft u reading isiru wara l-eżamijiet. Other subjects such as Crafts and Reading get done after the Exams are over. Cassar s (1991) conversations with the children revealed that after school they had to complete various homework tasks and find time to study. The reality was similar in Catania s (1991) study in which many of the children wrote about their lack of free time, their long hours of study and the demands of private lessons after school: Niddejjaq li d-dar ma tantx inkun nista nilgħab għax il-ħin liberu tiegħi nqattgħu nistudja. Jien naħseb li nistudja nagħmel tajjeb, imma l-ħin kollu niddejjaq. Ġieli lanqas inqum minn fuq is-siġġu. L-iskola nħobbha imma niddejjaq il-ħin kollu nistudja (p. 101). I get very fed up at home because I cannot play since my free time is taken up by studying. I think I m doing the right thing when I study but studying all the time annoys me. Sometimes I do not leave my seat! I love school but hate having to study all the time. More recently, in Cauchi and Scerri (2003), pupils said that they were continuously being pressured by parents and teachers to study hard for these examinations, and many of them were not even allowed time to play or watch television: F din is-sena wara li nlestu l- homework, flok nilagħbu nibdew This year, as soon as we complete our homework instead of playing we start 41

42 nistudjaw, revision u naħdmu l-past papers (p. 40). studying, revising and working out past papers The parents and teachers of the Year 6 pupils we interviewed suggested that it is the system that is inducing the stress on the pupils with visible effects: Għalliem: Is-sistema qed iġġib stress enormi lit-tfal qed infarrkuhom. Ġenitur: Neħlu li aħna l-ġenituri li npoġġu stress fuq it-tfal iżda hija s- sistema li tpoġġi stress fuqna. Ġenitur: Jagħmlu kemm jagħmlu qatt mhu qed jagħmlu biżżejjed, qatt ma jlaħħqu max-xogħol. Ġenitur: Dan l-istress qed iħalli impatt negattiv fuq it-tfal, isiru aktar ribellużi Ġenitur: Jirrabja aktar milli qatt kien qabel. Teacher: The system is causing enormous stress. We are crushing children. Parent: We are blamed for stressing the children, however it is the system that is placing stress on us. Parent: Irrespective of how much they work it never seems enough.. they never catch up with all the work. Parent: This stress is having a negative impact on children thay are becoming more rebellious. Parent: He tends to get angry far more than ever before. In Bonnici and Soler (2002), the pupils admitted that they did not enjoy studying and had to give up other things they enjoyed in order to do it: L-istudju insomma għax trid tagħmel sagrifiċċju, tinsa kollox, illogħob, it-television, biex toqgħod tistudja, ġieli veru niddejjaq u ma jkollix aptit, imma x tagħmel? (p. 59). Study because one needs to sacrifice many things, forget everything, playing games, television because one has to study, sometimes I m bored stiff and annoyed and I really don t feel like [doing it] but what can you do? Insomma ma tantx nieħu pjaċir nistudja, għax meta nkun qed nistudja jew naqra nkun qiegħed waħdi u meta nara lil xi hadd joqgħod jara t-television u jien nistudja niddejjaq (p. 60). Anyway, I don t really enjoy studying, because when I study or read I m alone. When I notice that someone else is watching tv (and I m studying) it annoys me no end! Interviews with Year 6 pupils clearly indicated that the pupils were constrained to give up things they enjoyed: M hemmx ħin għall-hobbies. Ma kontx insib ċans nilgħab naqra football. Sena ta ġenn lanqas ħin biex ikolli siegħa fuq l-internet ma kelli. There is no time for hobbies. I never found the time to play football. A year of madness I did not even get the chance to spend one hour on the 42

43 Internet. Ma nilgħabx bil-playstation. Kien hemm festival u ma stajtx nidħol għalih. Waqqaft il-lezzjonijiet tal-pjanu. Waqqaft iż-żfin. Kelli nieqaf mil-lezzjonijiet tal-pittura. Sibtha bi tqila ħafna biex inkompli l-gymnastics, hectic biex inlaħħaq. Inħobb insuq ir-rota minħabba l- istudju s-sibt biss kont insib naqra ċans. I don t play with my playstation. There was a festival and I was unable to take part. I stopped my piano lessons. I stopped dancing [lessons]. I had to put an end to painting lessons. I found it very difficult to continue gymnastics lessons very hectic difficult to cope. I love riding my bike because of my study load I could only get out for a short while on Saturdays. Apart from hobbies, preparation for these examinations also affected other aspects of their life: Għandi ħija żgħir u ma stajtx nilgħab miegħu. Ħajti nbidlet ħafna, nistudja ħafna, nqtajt mill-ħbieb, eżempju lanqas ċans nilgħab u nitkellem ma ħija ma kelli. Biex kont immur il-mużew matul il-ġimgħa, ġieli kont nieħu l-homework għand in-nanna s-sibt filgħaxija. Lanqas konna mmorru l-mużew minħabba l-homework, l-istudju u l-privat. I have a young brother I couldn t play with him. My life changed dramatically I had to study very hard stopped seeing my friends, e.g. I did not even get the chance to play and talk to my brother. For me to be able to attend MUSEUM classes during the week, I had to take my homework to my granny s and do it on Saturday night. We couldn t even go to MUSEUM classes because of the amount of homework we had, the amount of studying we had to do and the private lessons we had to attend The practice of after school private lessons is very common among Year 6 pupils. The study by Debrincat and Falzon (1996) reported that 40% of the children they surveyed went to private lessons in primary school. The frequency of this practice in classes preparing for the entrance examinations varied from all the pupils, to most, and to half of the pupils in class. Those who did not go to private lessons were getting help from parents or older siblings at home. The subjects generally ranged from three (English, Maltese and Mathematics) to five (to include Social Studies and Religion). The group size of after school lessons ranged from four pupils to more than twenty: 43

44 Aħna konna nkunu xi ħamsa u għoxrin student il-privat. We were around twenty-five pupils at private lessons. The pupils expressed various reasons for going to private lessons: Dak li ma nifhimx l-iskola, jiġi spjegat il-privat, it-teacher ta l-iskola ma tistax tibqa fuq l-istess ħaġa. Anything I did not grasp at school was explained during private lessons the school teacher cannot spend too much time on one item. Il-privat ta l-ingliż għeni ħafna biex inġib marki aħjar. Il-privat iservi biex tagħmel extra work biex tipprattika dak li tagħmel l-iskola. Fil-privat titgħallem it-tricks talkarta. Fil-privat, naħdmu l-istess karti li naħdmu l-iskola imma nagħmluhom qabel. Private lessons in English helped to improve my marks. Private lessons are there to give you extra work so you can practise what you have done at school. At private lessons you learn the tricks found in the exam paper. At private lessons we work out the same papers as the ones we work out at school however we do them before [at private lessons first]. The time spent at private lessons has an impact on the time available to complete the work assigned by the class teacher during the week and significantly increased the pupils workload on a daily and weekly basis. It is common practice for pupils to spend time on the weekend on tasks assigned by the private lesson teacher. Jien ma kontx inlaħħaq bil-homework fil-weekends flok nistrieħ kont nagħmel il-homework tal-privat ta l-ingliż u tal-malti. I never used to catch up with homework on weekends, instead of resting, I would do the English and Maltese homework for private lessons This tense environment surrounding the time of preparation for the examinations coupled with heavy workloads of homework and study is leading to pupil stress and anxiety. In Cassar s (1991) study, the majority of the children wrote about their anxiety: Jien naħseb li jekk nibqa hekk, żgur ma ngħaddix mill-eżami għax qiegħda nibza ħafna. Jien naħseb li ma nġibx marki tajbin għax ikolli ħafna tension fija u xi kultant nitfixkel u anki nagħmel silly mistakes għalhekk. Jien nibża li fl-eżami nibża li ma kontx studjajt biżżejjed, neċita ruħi u neħel (p. 98). I think if I keep feeling this way I will certainly not pass my exams because I m petrified. I don t think I will get high marks because I am feeling a lot of tension and sometimes I get mixed up and even make silly mistakes because of the tension. I m afraid that during the exam I would fear not having studied enough, I would panic and then fail. 44

45 Cassar (1991) and Mansueto (1997) reported that parents observed that their children had numerous symptoms of stress and anxiety such as irritation, nail biting, lack of concentration, restlessness, frequent headaches and tantrums: Infieħ, tgiddem dufrejha, biki għal xejn ma tistax tkellimha li ma tgħajjatx, tqum tibki bil-lejl jew toħlom u tkun skonċentrata (Cassar, p. 79). Feelings of anxiety and having had enough, nail-biting, on the brink of tears and crying for no apparent reason you couldn t talk to her she would shout wake up at night or have bad dreams - she was very distracted. Both studies reported that the children were restless during the night. They talked in their sleep and had nightmares about the examinations: Jimxi bil-lejl waqt li jkun rieqed u jqum bil-lejl biex jistudja. (Mansueto, 1997 p. 195). Iva ħlomt għax il-bieraħ ħlomt li mhux ser ngħaddi filgħodu ħlomt li l-mummy qaltli nispera li issa għaddejt u mbagħad ma nafx kif ġie r-riżultat dak il-jum u ġibt C, C, D, D, E, imbaghad bdejt nibki għax għidt li mhix se taħfirli qatt... (Cassar, 1991, pp. 88-9). He walks in his sleep and wakes up at night to study. Yes I dreamt because yesterday I dreamt that I was not going to pass in the morning I dreamt that my mother said I hope that you have passed this time and I m not sure how but that day I received the results and I got C, C, D, D, E, and then I cried because I thought she would never forgive me. A number of parents talked to us about the anxiety their children were experiencing in preparing for these examinations: Fi żmien l-għid spiċċajt nieħu t-tifel għand it-tobba għax kif kien jara l-karta kien jibda jirtogħod. It-tifla marret għall-eżami b ħafna biża, kienet tqum tibki bil-lejl, tistaqsini jekk nibqax inħobbha jekk teħel mill-eżami. During Easter time I ended up taking my son to doctors because as soon as he saw the exam paper he would shake. My daughter went to her exam terrified she would wake up crying at night, she would ask me if I would still love her if she failed her exams. Such anxiety was due in part to the long hours of homework that the children were submitted to on a daily basis. In general, the Year 6 pupils and their parents were in agreement on the pervasiveness of this task: 45

46 Jien kont indum ħafna nagħmel il-homework, ġieli kont indum sa l ta bil-lejl. L-iskola tispiċċa fis-1:30pm, u dejjem nagħmel sas-7:00pm għaddej bil-homework. Fil-Year 4 ġieli damet sad-9:30pm tagħmel il-homework. Ġieli kellhom 90 somma x jaħdmu plus il-homework tal-malti u ta l-ingliż. It used to take me a long time to do my homework, it sometimes took me until 10:00 o clock at night [to complete]. School is over by 1:30pm and I would go on until 7:00pm doing homework. In Year 4 she often went on until 9:30pm doing homework. Often they would have 90 sums to do as well as Maltese and English homework. Many parents told us that their children suffered from tiredness and fatigue during their final year in Primary school: It-tifla ġieli raqdet waqt li qed nistudjaw. It-tifla dejjem għajjiena u torqod f xi 8.30pm My daughter often slept while studying. My daughter is always tired, she sleeps around 8:30pm. In one case in particular, the pupil was so exhausted by the end of the first set of examinations that he missed the second set, despite all the preparation: It-tifel ħallas għall-common Entrance imma ma marx, ma felaħx jagħmel aktar eżamijiet. [My son] We paid for the Common Entrance Examinations but he did not sit them as he could not deal with any more exams Several of the children interviewed by the 11+ Review Committee confirmed research findings that parents and teachers put children under undue pressure: Il-mummy kissritni dawn id-disa xhur. Tatni rigal u kien past papers dak rigal sabiħ! Issa nibagħtek ġo klassi oħra fejn ma jifhmux bħalek (Cassar, 1991, p. 60). Jekk ngħaddi mmur nara l-papà l-ingilterra jigifieri jekk ma ngħaddix ma mmurx narah l-papà? My mother overworked me during these last nine months. She gave me a present it was a set of past papers some present! Now I will send you to a class where the pupils do not understand things like you do. If I pass my exams I will visit my father in England so if I don t pass will I still visit him? Several stakeholders pointed out that the system is negatively affecting children s self-worth and hence their self-confidence and self-esteem. Guidance Teacher: Hemm effett ħażin ħafna fuq l-istudenti li ma jgħaddux mill-junior Lyceum jew mill-common Guidance Teacher: There is a very negative effect on children who do not pass their Junior Lyceum and 46

47 Entrance. Jispiċċaw fis-sekondarja. Hemm minn jeħel mill-ingliż biss. Jistaqsi: Jien x niswa? Jien fiex jien tajjeb? Common Entrance examinations. They end up in Area Secondary Schools. There are some students who fail only in English. They ask: What am I worth? What am I good at? Guidance Teacher: Hemm l-istigma. L-istigma fuq it-tfal. Fejn mort skola? Tibqa magħhom. Guidance Teacher: There is a stigma a stigma the children feel. Which school did you attend? It [the stigma] stays with them. Ġenitur: Inkwetata minħabba l-futur tat-tifel. Moħħu ma jtihx għall-iskola. Jibża li mhux se jsib xogħol, taffettwa ħażin lis-self-esteem. Dawn tfal għad għandhom biss tlettax-il sena. Politikant: Dan l-eżami jħalli effett ħażin fuq is-self-esteem tat-tfal, għax dawn iħossu li mhumiex kapaċi. Jispiċċaw imorru fi skola sekondarja li, fil-perċezzjoni tan-nies, hija skola ħażina. Għalhekk it-tfal iħossuhom falluti. Parent: I am worried about my son s future. He is not academicallyoriented, he is afraid that he will not find work, it affects his self-esteem. These children are only thirteen years old! Politician: This exam leaves a negative effect on the children s self-esteem, because they feel useless. They end up attending Area Secondary Schools perceived as bad schools. That is why children feel they are failures. Student : Jekk ma ngħaddix inħossni mhux brava biżżejjed biex nilħaq xi ħaġa fil-ħajja. Student: Jekk ma ngħaddix ninkwieta u naqta qalbi għax inkun ħdimt ħafna u jkun għalxejn. Student: If I don t pass, I will feel as though I am not capable enough to succeed in life. Student: If I don t pass I will worry and become demotivated because I would have worked very hard and would have achieved nothing. Foreign expert in Malta: Pupils who failed the 11+ examination adopted lower expectations and their selfconfidence declined The findings of the recently published research report by Sollars (2006) which included a sample of 6000 pupils from 31 primary schools and 15 secondary schools concur with the above findings. Sollars concluded that: She also reports: In the local context, anecdotal evidence does suggest that children and young people growing up in Malta have much pressure brought on by an exam-oriented education system. This means that academic success is rewarded to the exclusion of other activities (p. 7). 47

48 There is general agreement that among the key factors which deter children and young people from attending after-school activities are the pressures brought on by an exam-oriented system, homework and after-school lessons (p. 9). She explains such a reality has negative repercussions on other important aspects of children s development: If children and young people are giving priority to academic success, willingly or otherwise, it would suggest that there is limited if any time available for leisure, cultural, artistic and recreational activities such activities promote the overall healthy development of the human being (p. 7) The experience of schooling in the low-ability stream in the upper years of Primary school where pupils are not prepared for the entrance examinations is far from ideal. Galea (1991) studied the experience of a group of pupils over a two-yearperiod in Year 5 and Year 6. Her observations in the low-ability stream led her to conclude that the children came to see schooling as an alienating experience. They expressed their dissatisfaction with their school in general and claimed that schooling was irrelevant to their life. Many pupils disliked English because they did not understand it and disliked Mathematics because they had to memorise multiplication tables. Maltese was their favourite subject because they understood the language, and they enjoyed Social Studies which they studied in Maltese. For a number of pupils, the only worthwhile activity in school was PE because it was directly linked to their extracurricular hobbies The pupils said that they were subjected to various types of labelling and stereotyping by pupils in other classes and by those in authority. Galea observed that in an argument with a girl from another class, the girl from the low-ability stream was told: Għandek għalfejn tgħid int, f ta Mr. Mifsud, l-aħħar klassi (p. 93). You re one to talk Mr Mifsud has the worst class. Interviews with pupils indlcated that in the low-ability stream they were not treated in the same way as pupils in the high-ability stream: 48

49 Student: Jgħajruna għax qegħdin F. Student: Preferenzi 6A jistgħu jużaw il-gel. Dejjem jintgħażlu studenti mill-a u B. Huma sejrin il-pixxina u aħna qatt ma noħorġu lilhom jingħata aktar support, aħna bla support ta xejn. Student: They tease us because we are in the F class. Student: Privileges 6A [pupils] can use their hair gel the pupils from Class A and B get chosen all the time. They are going to the National Pool and we never get taken out they are given more support we are not given any support at all. In Galea s study (1991) the pupils were very aware of the teachers attitudes towards them and they knew that their experience was different from that of pupils in other classes: Student: Ijja, iżjed jgħallmuhom (fil-klassijiet l-oħra) Reċerkatur: Is-Sir tagħkom ma jgħallimkomx? Student: Iva, imma jarana injuranti u dan allura ma jagħtiniex it-tqal, jagħtina l-ħfief. Ma jagħtiniex ħafna għax aħna injuranti (p. 73). Student: Ħa nitkellem ftit jien issa. Ara Miss, is-sir tagħna taf x jagħtina hu? Għaxar sommiet biss. U English, taf x jagħtina? Kliem jagħżilhom mil- Ladybird. Niktbuhom tliet darbiet. Dak mhux homework ta l-iskola. Allura jien xtaqt li kont fi klassi oħra jgħidilna li jekk immorru l-iskola (tal-ħmir) t-tort tagħna. It-tort tiegħu, għax hu ma jgħallimniex (p. 73). Student: Yes, they teach them much more (in the other classes). Researcher: Doesn t your teacher teach you? Student: Yes but he considers us to be stupid so he does not give us hard things to do only easy ones. He doesn t give us too much because we are stupid. Student: Let me speak now. Look Miss, do you know what our teachers give us? Only ten sums. And in English do you know what he gives us to do? Words taken from Ladybird Books. We write them three times each. That s not proper classwork. So I wished to be in another class he says that if we are sent to the donkeys school [school for idiots] it s our fault. The fault is his, because he does not teach us. Galea (1991) also noted that teachers in the study talked disparagingly about the pupils in the low-ability streams: Dawn taħseb li hawn agħar minnhom dawn? Jien ma naħsibx (p. 80). Do you think you can get worse pupils than these? I don t think so In our interviews with the pupils from the low-ability streams, we asked them about their experience of homework and how they felt about the Junior Lyceum examinations. Homework? Qabel iva, imma issa le. Homework? He used to [set us homework] but not anymore. 49

50 L-iskola qatgħetli qalbi (biex nagħmlu l-eżami). Ma nafx x inhu (l-eżami). Qatt ma rajt il-karti ta l-eżami. School put me off (sitting the exams). I don t know what this exam is all about. I have never seen the exam papers In Pullicino s study (2001) on life in a low-ability stream the teacher spoke about the children s perception of themselves: It-tfal iħossuhom li mhumiex bravi. Qishom għandhom tikketta u jippruvaw igibu ruħhom hekk. Jgħidu: Jien mhux bravu. Jien ma naf xejn (p. 71). The children feel that they are not clever. It is as though they have a label These children seem to try to fit in with their label. They say: I am not clever. I don t know anything. A number of teachers who completed the teachers questionnaire for the Review Committee (refer to Appendix 3) commented that in 2006, the current system continues to label pupils from low-ability streams as failures from a young age. In Pullicino s study (2001), the parents spoke about the harm the experience of segregation was causing their children: Riċerkatur: Kif ħassitha meta marret C? Ġenitur: Għall-ewwel iddejqet għax ma baqgħetx ma sħabha. Kienet qaltilha naħdem ħa nitla B. Imma issa marret iktar għall-agħar (p. 52). Riċerkatur: Taħseb li qiegħed jiġri hekk? [li tfal fi Stream C ma jagħmlu xejn] Researcher: How did she (your daughter) feel when she was sent to the C class? Parent: At first she was upset because she was not in class with her friends. She told us she would work hard to get promoted to the B class. But now her results are worse than ever. Researcher: Do you think that this is what s happening? [that the children in the C class are not working?] Ġenitur: Iva, dik minn dejjem hekk kienet. Ħa ngħidlek, qishom ma tantx jirsistulhom. Ngħid li jiġri hekk, li dawk li ma jkunux jafu, aktar jitfgħuhom lura (pp. 52-3). Parent: Yes that has always been the same. It is as though they do not work hard enough with them. [They re not dedicated enough] I think what happens is that the ignorant (the ones who know very little) get totally left behind they regress.] In our interviews, we talked to a number of children from low-ability streams. Among other things, we asked them to tell us about what was most memorable about Year 6 that they were about to complete: Xejn speċjali. Nothing special. 50

51 Il-PE għax ikollna darbtejn f ġimgħa. Kull nhar ta Erbgħa jkollna crafts. Sajjarna l-gallettini. Niġġieled aktar. P.E. because we have this twice a week. Every Wednesday we do Crafts. We baked biscuits. I fight more [than I used to]. In general, the experiences in the low-ability stream were not positive. However, there was one school where this was an exception: Miniex kuntenta li se nitlaq. Aħna nħobbuha ħafna t-teacher u anke lissurmast. Immorru nkantaw fi skejjel oħra u fl-assembly. U kont nitgħallem ħafna speċjalment fl-ingliż... il-lezzjonijiet kienu interessanti ħafna. I am not happy to leave [this school]. We really love our teachers and the Headmaster. We [are taken] to sing at other schools and during Assembly. I used to learn a lot, especially in English... these lessons were very interesting Interviews conducted by the 11+ Review Committee in continuous schools which have no streaming showed that there is an inclusive philosophy which encourages pupils to help each other, develop good self-esteem and strive to achieve: Teacher: Il-klassijiet taghna huma mixed ability classes. It-tfal jitgħallmu minn xulxin. Il-klassijiet u t-tfal mhumiex labelled. It-tfal anke jekk batuti ma tonqoshomx is-selfesteem. It-teacher trid taħdem aktar meta tkun ma mixed-ability class. Teacher: It-transition hija waħda smooth ħafna. It-tfal iżuru l-iskola [sekondarja] minn qabel. X ħin imorru jissetiljaw malajr. Teacher: Our classes are mixed ability. Children learn from each other. Classes and children are not labelled. Also low ability pupils do not experience low selfesteem. The teacher has to work harder with a mixed-ability class. Teacher: Transition is very smooth. The pupils visit the [senior] school in advance. When they move, they settle down easily. 3.3 Impact on Families In general, during our interviews, parents confirmed the high levels of stress and anxiety that these entrance examinations cause Maltese and Gozitan families: Dawn l-eżamijiet iqabbdu stress qawwi lill-familja kollha. Il-fatt li Għawdex m hawnx opportunatijiet daqs Malta jpoġġi pressure qawwija fuqna. These exams stress out the whole family. The fact that Gozo does not have the same opportunities as Malta, places greater pressure [on Gozitan students]. 51

52 M hemmx li jieqfu stress fuqi u fuq it-tfal dwejjaq kbar. If only they were to be abolished!.. they cause me stress and they stress out the kids unhappiness all round Parents reiterated that they needed to spend a lot of time with the child preparing for the examinations. This meant that they either gave less attention to the other siblings or they tried to divide their attention: Ma naffordjawx privat allura noqgħod magħha jien. Minħabba l-eżami, għandi tifel fil-year 4 li kelli nħallih waħdu. Kien jgħidli lili ma tħobbnix. Ma kontx nista nlaħħaq magħhom it-tnejn. Ommi kellha stress qawwi għax ħija sena ikbar minni u riedet toqgħod magħna t-tnejn. It-tifel ma qabdux stress imma qabbad stress lili. I cannot afford to send her [my daughter] to private lessons, so I have to stay with her myself. Because of the exams, my other son in Year 4 had to study alone. He used to say that I don t love him. I couldn t cope with both. My mother was very stressed out because my brother is a year older and she had to help us both. My son did not get stressed out but he caused me stress Apart from the possibility of getting less attention, siblings were made to give up certain hobbies at home in order that the child preparing for the examinations could study in peace. This demand sometimes led to conflict: Per eżempju oħti riedet tisma d-diski u ommi ma kinetx tħalliha u kien jinqala ħafna ġlied. As an example I can mention that my sister wanted to listen to music but my mother did not let her and it caused a lot of fights. As reported in Sollars (2006), in Malta and Gozo, families traditionally enjoy outings together, visiting grandparents on the weekend, going out to lunch or dinner and going on outings to the countryside and the seaside. Preparation for the JL and CE examinations impacts on such practices in a negative way: Waqafna noħorgu l-ħadd. Waqafna noħorġu l-ħadd, u s-sibt ma bqajtx immur nara lin-nanna. Is-Sibt il-familja ma toħroġx minhabba fija, noħorġu l-ħadd biss. We stopped going out on a Sunday. We stopped going out on a Sunday and on Saturday I stopped visiting my Grandmother. On Saturday the family does not go out because of me, we only go out on a Sunday. For those families whose village feast falls in the period before the examination, the impact is also felt during that particular weekend of celebrations: 52

53 Aħna kellna l-festa u meta ħriġna, ħriġna b ġirja. We had our village feast and we could only go out for a short while. Rather than opting for some free time, some children actually use the weekly visit to their grandparents as an additional chance to revise examination content: Biex immur nara lin-nanna kont nieħu l-kotba miegħi. Anke s-sibt filgħaxija meta mmur nara lin-nanna u niltaqa mal-kuġini noqogħdu nsaqsu l xulxin. When I visited my grandmother I used to take my books with me. On Saturday night when I would visit my grandmother, I would meet up with my cousins and we would ask each other questions Although it is difficult to quantify, a significant proportion of parents who have children in Year 6 (and also those who have children in Years 4 and 5) experience the additional financial burden of sending their children to private lessons. This practice impacts on family life in several other ways: the children s time at home is reduced; they have to find time to do homework required by the class teacher and by the private lesson teacher; and in many cases the parents have to take the children to the lesson and bring them back. This means that the practice impacts on the parents time as well. Domestic tasks and precious family time such as meal preparation, family dinner time and quality time together are all being affected by the examinations. Parents also lament that they too have less time to spend together Parents also seem to take on personal responsibility for their children s success or failure in these examinations (Cassar, 2003). This was confirmed by a guidance teacher: Hemm l-istigma jekk ma tghaddix, u d-dulur tal-ġenituri li huma stess iħossuhom failures. There is stigma associated with failure and parents feel very unhappy and consider themselves as failures [if their children fail the exam]. Some parents admitted that they hated the post-examination results period as people in villages spent time gossiping about who passed and who did not: Meta joħorgu r-riżultat lanqas immur nixtri mingħand il-grocer. When the results are published I don t even go to the grocer s. The reality brought about by these high-stakes examinations is captured in a comment made by a parent in one of our interview sessions: 53

54 It-tagħlim sar infern għalina! Education [teaching] has become hell for us! The newly established college setup within the state system was discussed with different stakeholders. Parents initial reactions have been positive: Ġenitur: Is-sistema tal-kulleġġi sserraħ ras il-ġenituri. Membru tal-fakultà ta l-edukazzjoni: Il-Kulleġġi biddlu l-kuntest edukattiv. Wieħed jinnota li b dak li ġara f San Benedittu, lill-ġenituri naqqsitilhom ittensjoni għax jafu li anke jekk it-tfal tagħhom ma jgħaddux għall-junior Lyceum, it-tfal sejrin fi skola tajba. Għalhekk in-nies ta dik l-area kkalmaw. Qabel, f każ li tifel ma jgħaddix, kont iżżommu ħalli jirrepeti s-sitt klassi tal-primarja ħalli ma jibagħtuhx fi skola li f għajnejhom mhix tajba. Il-ġenituri ma jridux jibagħtu lil uliedhom fi skola, li huma jemmnu, se tinfluwenza lil uliedhom b mod ħażin. Membru ta l-assoċjazzjoni Kunsilli Skolastiċi: Fl-inħawi ta San Benedittu hemm nies li issa ma jħossux li t-tifel għandu jirrepeti jekk ma jgħaddix. Dan huwa sinifikanti għax qatt ma ntqal qabel. Parent: The College system has put parents minds at rest. Member of the Faculty of Education: Colleges have changed the educational context. One takes note of what has happened at St Benedict s parents minds have been put at rest because they know that even if their children do not pass the exam, their children will go to a good school. This is why the people in that area have calmed down. In the past, when a boy would fail his exam, he would be asked to repeat his Sixth Year in Primary so they [the parents] would not send him to a bad school. Parents do not wish to send their children to a school that they perceive will have a negative influence on their children. Member of the Association of School Councils: In the St Benedict locality people now feel that their child need not repeat if he fails. This is a significant fact since it was never the case before this was never felt previously. The perception that their children will be going to the same school significantly reduces parental anxiety: Veru li xorta se jibqa numru ta tfal li ma jgħaddux, imma jafu li se jmorru fl-istess skola bħal dawk li jgħaddu u bl-istess uniformi, u għalhekk it-tensjoni tonqos ħafna. It is true that there will still be a number of children who do not pass, but at least now they know that they will be in the same school as the ones who have passed, wearing the same uniform, therefore the tension is considerably reduced. A parent remarked that St. Benedict s is a newly built school with all the amenities in the South of Malta. The teachers at St Benedict s College agreed and stated that they are pleased with the building and physical resources available. However, they felt that since they were teaching students of different abilities they needed more autonomy and resources when it comes to content, teaching strategies and assessment procedures. 54

55 The teachers said that the children s attitudes varied and those who had not passed the JL examination seemed happier than those who had in terms of being in the same school. Nitħalltu ma tfal differenti anke ma dawk li għaddew mill-eżami tal-jl. Tfal li għaddew għall-jl tibqa tarahom anke jekk inti ma tkunx għaddejt. We mix with different children, even with those who passed the JL examination. You keep seeing [in touch with] those children who passed the JL examination even if you yourself did not pass. This attitude was confirmed when the 11+ review Committee interviewed the students at the school. However, several of those who passed the examination felt that it was unfair that others who had not passed went to the same school and shared the resources. Int tkun ħdimt u ħaddiehor li ma ħadimx daqsek u ma għaddiex jiġi f- istess skola You would have worked hard and someone else who did not work as hard and did ot pass comes to the same school. Teaches also emphasised the need for a more supportive pastoral care programme that would enable them to concentrate better on their teaching. 3.4 Impact on Teachers The preparation for the entrance examinations impacts on primary schoool teachers. A study by Brincat and Mamo (2001) reported high occupational stress amongst Year 6 State school teachers. In this study, such stress is mainly attributed to time constraints related to syllabus coverage. Bonnici and Soler s study (2002) supported this finding and also suggested that the syllabus left no time for creativity. Teachers complained that five subjects were too many for the pupils to cope with. They claimed that the Social Studies syllabus in particular was overloaded and the fact that there were no resits put even more pressure on both pupils and teachers. In Cauchi and Scerri s study (2003), the Heads of School confirmed the pressure felt by teachers to have their pupils do well at these examinations; on their need to drill pupils to learn things needed for the high-stakes examinations; and on having reduced the success of primary education to the outcome of this one-off experience. These views were reiterated during our interviews: 55

56 Guidance teacher: Għalliem ikun imkejjel kemm hu tajjeb skond kemm jgħaddulu tfal. Anke l-iskejjel inkejluhom hekk. Skola tajba jekk jgħaddu ħafna. Heads ukoll hemm isimhom at stake. Membru tal-fakultà ta l-edukazzjoni: Hawn għalliema li jagħmlu aktar milli suppost. Pedagoġikament, l-għalliema jgħallmu dak li jaħsbu ġej fl-eżami. Biċċa għerf li mhix se tkun eżaminata titħalla barra. Jiġifieri hemm konsegwenzi serji ta l-eżami. Membru tal-fakultà ta l-edukazzjoni: Is-sistema ta` bħalissa hija ingusta. Qed tfixkel it-tagħlim u qed tfixkel lillgħalliema mill-vokazzjoni taghom. Guidance Teacher: A teacher s worth is measured by the number of students who pass the exam. We also measure schools by the same criteria. It is a good school if many students pass. Heads have their professional reputation at stake. Member of the Faculty of Education: Some teachers go beyond their call of duty. Pedagogically, teachers teach according to what they think will appear in the exam papers. Knowledge that is not tested by the exam is left out. This means that there are serious consequences. Member of the Faculty of Education: The system in place is unjust. It is impeding teaching and teachers vocational approach to their work In the survey that the Review Committee conducted with Year 5 and Year 6 teachers in 2006, a total of 356 Year 5 and Year 6 teachers from 77 schools completed the questionnaire. There were 276 of them who were targeting the entrance examinations (refer to Appendix 3). In this study, teachers once again pointed out the stress caused by the syllabus which was not allowing enough time for discussion and creative approaches to teaching. During the face-to-face interviews with teachers in a number of schools these issues were confirmed: Tellieqa kontra l-ħin. It-tagħlim sar cramming. L-Istudju Soċjali hu vast ħafna, u ma jħallix ħin biex per eżempju mmorru naraw il-hypogeum u postijiet oħra. Kultant minħabba l-eżami jkollok tagħmel affarijiet li aħjar ma tagħmilhomx għax it-tfal ma jifhmux u jibdew jitgerfxu. A race against time. Teaching has been reduced to cramming. The Social Studies syllabus is very vast, and it does not allow enough time to visit for example, the Hypogeum and other places. Sometimes, due to the exams, it would be far better to omit some things because the kids do not understand and they get confused In the survey, teachers were asked to evaluate the extent to which the 11+ examinations were influencing what they emphasised in their teaching. The majority of the respondents: 72.2 per cent (257) overall and 79.7 per cent (220) of those targeting entrance examinations, claimed that the examinations influenced them a great deal. All the respondents targeting entrance examinations are greatly influenced 56

57 by these examinations. Most of the overall respondents: 66.3 per cent (236) and 73.2 per cent, (202) of those targeting examinations, considered this influence undesirable and some qualified their answer in ways that support earlier findings: the examination is given a lot of importance (105); there is a vast syllabus (66); these examinations are killing motivation and creativity (52); and there is a lot of pressure (50) The teachers were also asked to evaluate the extent to which the 11+ examinations were influencing how they teach. Again, the majority of respondents: 66.0 per cent (235) overall and 73.9 per cent (204) of those targeting entrance examinations, said that these examinations influenced them a great deal. The majority of the respondents: 69.9 per cent (249) overall and 77.5 per cent (214) of those targeting examinations, considered this influence as undesirable and some gave the following reasons: the syllabus left no space for them to adopt creative strategies (69); the main focus of teaching is the examination and therefore the emphasis is on the core subjects (48); there is very limited time to meet the required targets (28); it is a stressful process which leaves no time to enjoy learning (25); and it leaves no time to develop pupils potential through other subjects which are not examinable (21) On the issue of homework 6 and classroom resources, teachers responded as follows: around half: 47.8 per cent (170) overall, and 45.3 per cent (125) of those targeting entrance examinations, said that they gave pupils one hour of homework a day. The other half: 47.5 per cent (169) overall and 48.9 per cent (135) of those targeting entrance examinations, said that they gave pupils two hours of homework a day. Thirteen teachers (11 of whom were preparing students for entrance examinations) said that they gave pupils three hours of homework a day. During examination preparation, teachers use commercially produced material to assign work in the different subjects. Teachers were asked to quantify how often they used these resources. For each of the five examinable subjects, the largest category of teachers responded sometimes. Often was another popular choice especially for English, Maltese and Social Studies. When often and sometimes were considered together, 70.2 per cent of all respondents and 71.4 per cent of those targeting entrance examinations reported that they used commercially produced materials for English; 68.3 per cent of all respondents and 71.4 per cent of those targeting entrance examinations used them for Maltese; 59.8 per cent of all respondents and 58.0 per cent of those targeting entrance examinations used them 6 The time it takes pupils to complete their homework depends on their individual pace. 57

58 for Social Studies. Commercially produced materials were less popular for Religion and Mathematics. In fact, 40.4 per cent of the overall respondents and 42.8 per cent of those targeting entrance examinations claimed that they rarely or never used commercially produced materials for Mathematics and 37.4 per cent of the overall respondents and 39.5 per cent of those targeting entrance examinations claimed that they never used such materials for Religion In summary, the teachers questionnaire highlighted the strong influence of the 11+ examinations on teachers. The current system is having undesirable effects on what they teach and how they teach it. Teachers claim that in the current system, the focus is on the core examinable subjects. The system is leaving little or no time for the teaching of other subjects and fun activities that allow children to express themselves and explore their potential. Furthermore, teachers also feel that as long as the current system prevails, they feel powerless and cannot do things any differently because the high-stakes entrance examinations at the end of Year 6 still dominate. 3.5 Impact on the Education System Impact on Primary Education Primary education, especially in the upper years, has been reduced to coaching pupils for the high-stakes examinations at the end of Year 6. The JL and CE examinations are conveying the message that primary education is all about getting high marks in examinations. Catania (1991) made the following observation: Problems such as strict reliance on the requirements of this examination are affecting adversely a child s primary education. The concern of a good primary education should be the growing needs of the child but with the retention of this examination, this stands to lose (p. 117). The authors of Tomorrow s Schools (1995) also stated that teachers were teaching with examinations in mind, hurrying through the vast syllabi and applying methods of teaching more conducive to rote learning than understanding. The Association of School Councils, an organization for administrative councils present in schools composed of parents and school staff amongst others, also made a statement to this effect in its press release of 12 March 1997 where it said the present examination system is having a negative impact on students students are just focusing on 58

59 examinations to the detriment of creative work. In interviews with the different stakeholders, these views were reiterated: Għalliem: L-enfasi hija biex jitgħallmu bl-amment, hemm bżonn li nibnu l-kunfidenza u opportunatijiet ta socialization. Ġenitur: Lit-tfal ġenninnihom, għamilniehom robots. Ġenitur: Fil-Year 6 inbellgħulhom l-affarijiet bl-amment. Rappreżentant ta Trade Union: Għalliema jgħallmu b sistema ta` coaching. Teacher: The emphasis is on rote learning. There is a need for us to build confidence and offer socialization opportunities. Parent: We have maddened the children, we have turned them into robots. Parent: In Year 6 we ram rote learning down their throat! Trade Union representative: Teachers are using a coaching system Everyone agrees that stress is a consequence of these high-stakes examinations. Għalliem: L-istress fuq it-tfal huwa inkredibbli. Teacher: The stress the children are subjected to is incredible. In general, the impact on the teaching and learning is such that there is little joy of learning in the upper years of Primary school where the emphasis is on coaching for the examinations. These views were shared by pupils, teachers and parents alike: Għalliem: Minħabba l-eżami t-tfal mhumiex jieħdu pjaċir jitgħallmu. Ġenitur: Qed jistudjaw għall-eżami, imbagħad jinsew kollox. Teacher: Due to the exams, children are not enjoying learning. Parent: They are studying for the exams and then they forget everything they have learnt. Cassar (1991) had noted that the examination was a reference point in Year 6 and often at the forefront of the teacher s mind. She remarked, The teacher gets caught up in the system of teaching for the examination (p.39). Similarly, after a year s observations Mansueto (1997) concluded that the examination seemed to haunt pupils and teachers alike as if it were an invisible but very real phantom (p. 187). In Duca et al. s (1999) study, parents similarly observed that teaching focused on examination content, and subjects that were not examinable were given little or no importance. In Cauchi and Scerri s (2003) study, the Heads of School confirmed the pressure felt by teachers to have their pupils do well at these examinations, on their need to drill pupils to learn things required for the high-stakes examinations and on having reduced primary education to the outcome of this one-off examination experience. 59

60 In 2003, the Educational Assessment Focus Group (EAFG) conducted a study of assessment practices in schools in Malta and Gozo. The methodology included group interviews with all Heads of school and school questionnaires. A number of Heads of Primary schools had pointed out that examinations motivate pupils to study more: the students are motivated to make greater efforts. For example when subjects are not examinable such as Art, Music and Literature students do not take them very seriously (p. 10). although we give a lot of importance to course work we feel that students work harder for exams (p. 10). One respondent had explained that: although exams create a certain amount of stress, yet they remain the fairest form of assessment (p. 10). Another Head of School had said exams are the most reliable since at present there is a lack of standardization and common criteria in alternative assessment (p. 11). Another reason is the belief that examinations are valid and objective in finding out what the students know At the same time, a number of State Primary Heads of School had remarked that: there is too much memory work related to our exams Years 4, 5 and 6 are dominated by tests and exams. The Junior Lyceum entrance exam puts a lot of pressure on the teachers and students and introduces a lot of competition. There is not much time for extra curricular activities (p. 12). In the questionnaire, two Heads of school had explained that: Summative assessment practices create a great deal of anxiety and tension and lower the self-esteem of several pupils since the latter are compared with one another. Testing encourages rote and superficial learning and methods of evaluation are not always shared among teachers (p. 12). Examinations are extremely stressful on all parts concerned. There was too much work to assimilate, considering the young age of primary school children (p. 13). Similar concerns were voiced during the EAFG interviews. Heads of Primary School especially had felt very strongly about this: Summative assessment is being given exaggerated importance. This is working against the self-esteem of the children particularly those who do not do so well in examinations (p. 13). 60

61 Exams are branding children as failures. Too much drilling for exams is inevitable as results hold teachers accountable. Some children cannot read the exam paper and consequently their self-esteem is lowered further (p. 14) However, there are different views on the reality that there are two sets of high-stakes examinations for boys. For some, this situation doubles the stress. For others, it reduces it since there are twice as many chances of success. It was pointed out that the pressure is there because of the outcome of the examinations. Ġenitur: Żewġ settijiet ta eżamijiet u pressure doppja. Parent: Two sets of exams and double pressure. Student: Żewg eżamijiet ituk aktar ċans. Student: Jien naqbel li jibqgħu tnejn għax ikollok żewġ ċansijiet jekk ma tgħaddix minn wieħed forsi tgħaddi mill-ieħor. Għalliem: Il-pressjoni mhix l-eżami per se imma dak li jfisser l-eżami, jiġifieri l-għażla tat-tfal speċjalment il- Common Entrance għax wieħed irid jistinka kemm jiflaħ ħalli jiġi minn ta quddiem. Guidance Teacher: Il-Common Entrance huwa iebes għax trid tipplessja. Hemm l-interpretazzjoni tar-rizultat. Ġibt 75 per cent u nħossni li mort ħażin għax fil-common Entrance ma ġejtx ma l-ewwel ħames mija. Student: Two sets of exams give you a better chance. Student: I agree with having two sets of exams if you don t pass the first set, you get another chance at the next set. Teacher: It is not the exam itself that causes pressure, but rather the selection of children, especially with regards to the Common Entrance exam since the children need to work very hard to get a high ranking. Guidance Teacher: The Common Entrance Exam is hard because you have to get a good placing. There is the issue of the interpretation of results. I got 75 per cent but I feel I did badly because I did not place amongst the first 500 students in the Common Entrance Exam It needs to be pointed out that pressure starts to set in Year 4 when pupils start competing for high marks in examinations in order to secure a place in a high stream in Years 5 and 6 (Grech, 2003). Different stakeholders spoke about the stressful situation in Year 4: 61

62 Membru tal-fakultá ta l-edukazzjoni: Il-pressjoni tibda fir-raba klassi tal- Primarja meta jsiru l-eżamijiet, u fuq ir-riżultat isir l-istreaming. Allura hemm tranżizzjoni bejn ħajja bla eżamijiet u waħda fejn l-eżamijiet għandhom riżultat tant importanti. Għalliem: Hemm qabża kbira bejn l-ewwel tliet snin fejn mhemmx eżamijiet għas-snin li jiġu wara fejn kollox jiddependi mill-marki ta l-eżami. Politikant: L-aħħar sentejn tal-primarja huma kkanalizzati għall-eżami. Student: Mill-Year 4 bdejna nippreparaw ruħna imma fil-year 6 beda jiżdied il-homework. Ġenitur: Ġieli fil-year 4 it-tifla kellha mitt somma x tagħmel. Member of the Faculty of Education: The pressure starts in Year IV in Primary Education when exams are taken and these results determine how the children are going to be streamed. So there is a transition between life without exams and life where exam results are of the utmost importance. Teacher: There is an enormous difference between the first three years (of schooling) where no exams are taken and the years where everything depends on the exam marks. Politician: The last two years at Primary level are channeled towards exams. Student: We started getting prepared in Year 4 but in Year 6 homework increased considerably. Parent: Often in Year 4 my daughter was asked to work out 100 sums Streaming The issue of streaming is interlinked with the current state of affairs and impacts directly on the educational experiences in our primary schools. During group interviews very contrasting views were expressed by teachers: L-istreaming joħloq stress u pressure. Ara li ma nneħħux l-istreaming għax neqirdu kollox. Streaming brings about stress and pressure. Don t you dare remove streaming because we will ruin everything. In Malta and Gozo, teachers are divided on the issue of streaming. There are those who support it wholeheartedly as an organisational strategy and others who show an awareness of its negative impact on self-esteem and say it is an unfair system. Teachers who teach in the upper years of state primary school generally speak in favour of streaming pupils. More specifically, they speak in favour of teaching homogenous groups of pupils, especially in the high-ability streams. The three studies that have looked at teachers attitudes towards streaming have all concluded that locally teachers are still in favour of this organisational set-up. In Galea s study (1991) teachers general opinion was that streaming was necessary and advantageous. They said that both teachers and students stood to gain. Teachers 62

63 believed that they could teach more effectively in a context where the group was homogeneous in ability. They were of the opinion that mixed ability teaching was detrimental to both high- and low-ability students. They said that in their view, it was impossible to create a balance to interest pupils of different abilities. One teacher went so far as to suggest that in a mixed ability setting, the better students deteriorated In Gatt and Vassallo Agius study (1988) in which teachers were asked to share their views on the possibility of non-streaming in the upper primary years, the fear of lowering academic standards was expressed by all 20 teachers participating (100%). Equally popular were responses related to the potential negative effect on the gifted pupils (94%) and the A-stream staff (78%). The study focused mainly on the high achievers and their level of performance. In a later survey carried out by Pace Moore (1999) in 28 schools, 64 of the 100 teachers who completed the questionnaire said that they preferred a streamed class because they could adapt the material to be learned to the level of the homogenous group and this was easier for them and better for the pupils. However, in this study there were 28 teachers who referred to the labelling involved in the process of streaming and the lack of motivation in the low-ability streams. Together, these studies indicate that teachers speak in favour of streaming as an organizational procedure that makes their preparation and delivery of lessons easier. Moreover, when speaking in favour of he system, teachers tended to think of pupils in the high-ability stream and when they spoke against it they refered to pupils in the low-ability stream Parental views are similarly divided on the issue of streaming. Local studies dating back to the 1980s and 1990s and reported in Grima and Farrugia (2006) suggest that parents of pupils in high-ability streams continue to support the practice. They believe that children should continue to be separated on the basis of their ability. The parents of those pupils placed in the low-ability stream are conscious of the limited educational experiences of their children in this stream and are also aware that their children would benefit from mixing with more able and possibly more motivated children. These positions are exemplified in Gatt s and Vassallo Agius (1988) ethnographic study. The parents of the pupils in the A- Stream believed that mixing children from different streams would disrupt their children s high academic achievements: I would not like my child to be with children who are worse than him. I would prefer if he mixed with better children (p. 65). 63

64 However, the parents of the children in the low-ability stream held a different view: It is not fair on the children to be selected. The ones who are backwards can perhaps do better if they are mixed with brighter ones (p. 66) Such views were also reported in the extensive survey carried out by Duca et al. (1999) in nine localities in Malta. In this study, more than 80 per cent of the parents said that they were in favour of streaming. The three most common reasons given were the following: children are grouped according to their abilities; it creates groups which are homogeneous; and it was very time-consuming for the teacher to have to repeat explanations for the children who are falling behind. However, in the same study there were parents who were against streaming because it was unjust and little progress was achieved in low-ability streams. Morevoer, the experience affected children negatively throughout their life Generally speaking, parents have come to accept the current system of streaming and selectivity as natural. Support for the system is not limited to parents of high achievers: Membru tal-fakultá ta l-edukazzjoni: Hawn ġenituri li jaqblu mas-sistema minkejja li t-tfal tagħhom stess qed jispiċċaw fis-sekondarja. Il-ġenituri mhux dejjem ikunu jafu x inhu l-aħjar għal uliedhom. Issib ġenituri li huma favur is-sistema għax huma mingħalihom li hekk jaqbel għat-tfal tagħhom. Dawn il-ġenituri jafu jwasslu l-vuċi tagħhom, imma hawn ħafna ġenituri bla vuċi u aħna rridu nħarsu l- vuċi tagħhom u ta wliedhom. Member of the Faculty of Education: There are some parents who support the system, even though their own children end up in Area Secondary Schools. Parents do not always know what is best for their children. Some parents are in favour of the system because they think that it is right for their children. These parents are able to voice their opinions, however, there are many parents who cannot and we are responsible for speaking up for them and their children. Mallia (2003) suggests that the high-stakes situation at the end of primary schools puts parents in a crisis: Many of them do not know any better than to just push their children on to the only finishing line they know of And they do not know any other way, and even if they did, as the moment approaches all they can see is the finishing line and the possible repercussions of not reaching it, in an educational system that is so thoroughly streamed that it really has boiled down to the generation of a society of educational haves and have-nots The reality of this observation is evident in the daily life of children and their families. The system of streaming and selectivity brings divisions among children who attend the same school and divisions among their parents: 64

65 Student: Ħazin ħafna li A u B ma jikkomunikawx flimkien. Student: M għandiex taffetwa l- ħbiberiji, speċjalment tal-ġenituri. Rappreżentant ta Trade Union: Is-sistema silettiva toħloq elitiżmu. Student: It is very bad that the A and B students do not communicate with each other. Student: It shouldn t affect friendship. especially the parents friendship. Trade Union representative: Selectivity brings about elitism. The current system instils in children the belief that it is acceptable to differentiate between children on the basis of ability or achievement at both Primary and Secondary school: Jien naqbel li jkollna klassijiet A dawk il-bravi nett, B dawk li jridu jitgħallmu u C - dawk li ma jridux. Ma naqbilx li titħallat għax it-teacher tagħti importanza lill-batuti u twarrab lit-tajbin tajbin ma tajbin, ħżiena ma ħżiena. Ma tistax taħdem ma tfal batuti aħjar kulħadd ikun għalih. Bħala eżami għandu jibqa ħalli min hu tajjeb jista javvanza. Min hu batut imur skola addattata għalih. Għandu jkun hemm tliet tipi ta skejjel: Junior Lyceum għall-bravi ħafna; Area Secondary għall-inqas bravi; Skola oħra, mhux tal-bravi u mhux ta l-injoranti imma għal dawk li jixtiequ jitgħallmu u jimxu l quddiem imma ma jistgħux għax moħħom ma jlaħħaqx. I agree that we should have Class A for the very clever pupils, Class B for the ones who want to learn and Class C for the ones that don t (want to learn). I do not agree with mixed ability classes because the teacher will give the weaker pupils all the attention and ignore the bright ones clever children should be with clever children and weak pupils with other weak pupils. One cannot work with weak pupils in the class it is far better if pupils are separated. The exams should stay, so that clever children can proceed and improve. Weak pupils should be sent to a school which is suitable for them. There should be three types of schools: Junior Lyceums for very bright children; Area Secondaries for those who are less bright; and another school not for the ones who are bright nor for the ones who are stupid, but for those who would like to learn but find it difficult to do so, because of their mental capacity. 65

66 There were some exceptions to these views, with children favouring inclusion: Naqbel 50:50 għax min imur f Area Secondary u ma jġibx il-kwalifiċi qed tagħmillu tabella ta injorant. Jekk kulhadd imur ġo l-istess skola ħadd ma jkun jaf min hu injorant allura ma naqbilx li għandna ninfirdu imma nibqgħu nattendu l-istess skola. Naħseb li naqbel. Kulħadd hu speċjali. I partially agree (50:50) because those children who go to the Area Secondaries and do not get qualifications are labeled. If all children go to the same school, then no-one will know who the stupid children are therefore I am not in favour of selection we should attend the same school. I think I agree (with inclusion). Everyone is special During our meeting with the Faculty of Education, several members shared their views on the current system of streaming and selectivity. These views are presented verbatim as they depict the current thinking of education experts in Malta: Għalfejn l-istreaming? Għal konvenjenza. Minħabba li hemm l-eżami. Qed nagħmlu għażla kmieni u qed ngħidu li ċerti tfal mhumiex tajbin biex jitgħallmu. Dawn huma assunzjonijiet żbaljati. Why streaming? Because it is convenient. Since the exam exists. We are making early choices and we are saying that some children are not capable of learning. These are erroneous assumptions. Il-kurrikulu li bħalissa huwa one size fits all. L-eżami tal-junior Lyceum qed iqis lil kulħadd l-istess. Għandna l-istess miżura u allura taqta n-nies barra. Mhux kulħadd huwa adattat għal dak it-tagħlim akkademiku. Għandna 10% tat-tfal li ma jagħmlux l-eżami tal-junior Lyceum, u 40% ta dawk li jagħmluh li ma jgħaddux. Lil dawn it-tfal qed ngħidulhom: Intom m intomx tajbin biex tkomplu l- mainstream education, meta fil-fatt dawn ikun fadlilhom ħames snin skola fis-sekondarja. Is-sistema hi inġusta. Għandna 7% tat-tfal li huma illitterati. Għandna dropouts u tfal li juru nuqqas kbir ta interess fl-iskola. Għandna tfal li għandhom il-ħila jkunu litterati imma ma jsirux għax l-eżami qed jagħmel għażla bejn tfal li huma tajbin għalliskola u oħrajn li mhumiex. The present-day curriculum is one size fits all. The Junior Lyceum exam is treating everybody the same. We are using the same measure for all and therefore some get discarded. Not everybody is cut out for that type of academic work (learning). 10 % of the children do not sit the Junior Lyceum exam and 40% of those who sit, do not pass. We are saying to these children: You are not bright enough to continue within mainstream education when in reality they have yet another 5 years of Secondary schooling to get through. The system is unjust. 7% of our children are illiterate. We have a number of dropouts and children who show little or no interest in schooling. There are pupils who are capable of becoming literate but are not making it because the exam has chosen between the ones who are good enough and those who are not. 66

67 The members of the Faculty of Education also stressed that the current system of streaming goes against the spirit of our National Minimum Curriculum: Hemm prinċipji importanti ta l-nmc li rridu nagħtu kashom, fosthom il-prinċipju ta l-inklużjoni. Kif jista jkollok inklużjoni u fl-istess ħin tifred lit-tfal fil-primarja? Naqblu li l-preżenza tal-junior Lyceum examination tmur kontra l-ispirtu ta inklużjoni u diversità? Din il-mistoqsija trid risposta. There are important NMC principles we need to take heed of, among them the principle of inclusion. How can we say we are inclusive and then stream the children at Primary level? Are we in agreement that the existence of the Junior Lyceum exam goes against the spirit of inclusion and diversity? This question warrants an answer At this point it is worth looking at the actual numbers of pupils affected by streaming. Streaming affects a large proportion of pupils in the state sector. In the scholastic year 2006/7, in Year 5, 2684 pupils out of 3013 (89.1%) were in streamed classes. In effect, only 19 out of 151 classes (12.6%) were not streamed in this age group. In Year 6, the situation is very much the same with 2722 pupils out of 3038 (89.6%) being placed in streamed classes. Only 19 out of 150 (12.7%) classes were not streamed in this age group. Streaming is the norm in Year 5 and Year 6 in state schools. The only schools that do not have streamed classes in these year groups are the ones where there is only one class per year group. Therefore it is not a choice per se A study by Mercieca (1991) attempted to look at the outcomes of streamed and non-streamed classes in relation to performance in the Junior Lyceum entrance examination. The study included 27 of the 59 State Primary schools in Malta and Gozo (46%). The sample included 15 streamed schools out of a total of 43 (35%) and 12 non-streamed schools out of a total of 16 (75%) and an average of 31% of the pupil population in Years 4, 5 and 6. The study concluded that although the results he obtained did not reach significant levels, there were clear indications that: Pupils in streamed schools tend to perform better in the [Junior Lyceum] entrance examination than their peers in non-streamed schools. In fact, 12 out of the 15 comparisons made are in favour of streamed schools and only three out of 15 are in favour of non-streamed schools Although the differences have not all reached a statistically significant level, yet their persistent recurrence together with their four statistically significant results obtained all in favour of streamed schools and none in favour of non-streamed schools appear to be indicative that pupils in streamed schools tend to do better in the Junior Lyceum examination than their peers in 67

68 non-streamed schools [However] more low-ability pupils tended to be referred to the Opportunity Centres from streamed rather than from non-streamed schools (p. 115) Furthermore, an analysis of the published data on the performance in the Junior Lyceum entrance examination of pupils in 62 State primary schools in Malta and Gozo during the ten-year period from 1994 to was carried out as part of the current review. The analysis shows that it is difficult to reach firm conclusions about the effects of streaming in Maltese schools. Uncertainty arises for a number of reasons. First, streaming is not optional in State schools. Only schools with a Year 6 population of less than 30 have non-streamed classes. Indeed some small schools may have a mixed ability Year 6 class one year and two streamed Year 6 classes the next year. These observations question the validity of comparing the performance of students in small schools with those of large schools where mixed-ability classes are not an option. Second, the performance of schools with non-streamed Year 6 classes fluctuates erratically as it depends on the performance of a small number of pupils. For example, between 1994 and 2003 the Year 6 population of a small school in Western Malta and its performance fluctuated sporadically between 10 and 21 pupils and a pass rate of between 20.0% and 71.4%. Similarly, for a small school in Gozo whose Year 6 population and examination performance fluctuated between 9 and 19 pupils and a pass rate between 21.4% and 80.0%. Third, there are significant differences between schools in different regions. For example, in the ten-year period under study the average pass rate is less than 25% in six schools; five of these schools are located in the Southern Harbour Region. Schools with an average pass rate of less than 40% are distributed as follows: 9 out of the 13 schools in the Southern Harbour Region; 9 out of the 13 schools in the Northern Harbour Region; 5 out of the 11 schools in the South Eastern region; one out of the 8 schools in the Western region, and one out of the 6 schools in the Northern region. None of the 11 schools in Gozo has an average pass rate less than 40%. It appears that schools in certain regions have low success rates irrespective of whether they are streamed or non-streamed. These observations suggest that socio-economic and cultural factors also influence the success rate in the JL examination. Furthermore, the success rate of non-streamed schools is influenced by the performance of a small number of students so that no generalisation can be made. Our analysis shows that conclusions about the effects of streaming as practised in Malta and Gozo must be treated with extreme caution. 7 This was the last time that the pass rates of the different schools were published in the annual report of the JL examination. 68

69 3.5.3 Impact on Candidates The data in Table 3.1 below indicates that just under 50% of the Year 6 pupils in the State sector passed the JL entrance examination in the past five years. Table 3.1 also shows that, on average, about 10% of the pupils did not attempt the examination in the same period. The rest do not manage to achieve passes in all five subjects. It must be pointed out that some of these pupils may have obtained good grades in one or more subjects but then get a grade D or E in one or more subjects. Table 3.1: Junior Lyceum Examination: Non-Applicants and Unsuccessful Candidates in State Sector N % N % N. % N. % N. % Non-Applicants Unsuccessful and Non-Applicants The above data confirm that we have thousands of students who are labelled as failures at the age of 10 or 11 (Grech, 2003). Furthermore, the national percentage of student failure is giving the impression that the Maltese educational system is an industry of failures (Muscat, 2002) A number of unsuccessful candidates opt to do Year 7 in the Primary school and then resit the Junior Lyceum entrance examination at the end of this year. These pupils opt not to go to an Area Secondary school. Table 3.2 shows the number of pupils who completed Year 7 in the past three years. Table 3.2: Year 7 Pupils in Primary Schools in Malta and Gozo 2004/ / /2007 M F Total M F Total M F Total State Malta State Gozo Total For each of the three years, there were more boys than girls who completed Year 7. In Gozo, there are no Year 7 pupils in the current scholastic year. In Malta, there is a progressive reduction in the number every year. Table 3.3 below shows that all Year 7 candidates with the exception of two re-sat the Junior Lyceum entrance examination in the past two years. On average, just over half of them were successful the second time round. 69

70 Table 3.3: Year 7 Candidates for the Junior Lyceum Entrance Examination Male Candidates Male Passes Female Candidates Female Passes Total Candidates Total Passes (54.7%) (53.6%) (54.3%) (59.5%) (55.3%) (57.7%) For both the Junior Lyceum and the Common Entrance examinations, applications are received from candidates who request special examination arrangements in order to be able to sit for the examinations. Conversations with the Bodies involved in the processing of these applications have highlighted the steady increase of such applications. The numbers of requests which are presented in Table 3.4 below, show that the requests have doubled in the past five years. Table 3.4: Special Arrangements: Junior Lyceum and Common Entrance Examinations Junior Lyceum Common Entrance Applied for Special Arrangements Granted Special Arrangements Applied and Granted Special Arrangements On the one hand, such provisions indicate that both examination systems are compassionate and inclusive and do their best to accommodate different candidates needs. However, such a high-stakes system may be forcing some parents to resort to such requests in order to try and gain some extra time for their children who may not be fast thinkers or doers but who in non-examination conditions at this tender age may be competent and cope with tasks without special assistance. The impact of the experience of requesting special arrangements may be detrimental to these children s well-being and normal development Impact on the Achievement of the Objectives of the National Minimum Curriculum In the teacher survey, teachers were asked to evaluate the extent to which the 11+ examinations were facilitating the achievement of the objectives of the National Minimum Curriculum in their classrooms. The teachers responses showed their awareness of the incongruence between the 11+ examinations and the implementation of the NMC. More than half of the respondents suggested that the 70

71 examinations did not do much to stimulate analytical, critical and creative thinking skills (63.5%); to help develop children s potential (61.2%); to encourage the holistic development of children (57.9%) and to promote a pedagogy that respects diversity and celebrates difference (52.2%). The most common reasons were the following: the syllabus does not allow time to adopt creative strategies and analytic thinking skills (127 responses), the examinations are very important and very stressful (82), there is very little time to cover the syllabus (28), and the system makes slower learning pupils feel they are failures (20). Furthermore, they also believe there is a discrepancy between the current selective system and the principles of the NMC. They cannot co-exist in the same educational system and as long as the selective examination remains, the principles of the NMC cannot be implemented and developed in our educational system Impact on Secondary Education For the majority of Maltese children, going to an Area Secondary school means going to a second-rate school. Cassar (2003) describes the emotional state of children in the Area Secondary schools in the following way: Children come to our schools [Area Secondary] broken down with very low self-esteem which many a time is the result not only of Year Six drilling but also from the previous years of preparation, and hence labelling, for this examination In interviews with the stakeholders, there was general consensus on the labelling and feeling of inferiority felt by children who go to Area Secondary schools. The situation is complex. Membru tal-fakultá ta l-edukazzjoni: L-għażla qed tagħmel ħafna ħsara u qed tħalli effett ħażin anke mil-lat ekonomiku. Tfal fis-sekondarja jaqtgħu qalbhom u jmorru lura. Member of the Faculty of Education: Selectivity (choice Junior Lyceum or Area Secondary School) is causing a lot of damage, it is also having a negative effect on the economy. Children attending Area Secondaries get demotivated and regress. Membru tal-fakultá ta l-edukazzjoni: B din is-sistema t-tfal qed ikunu labelled u qed tħalli effett ħażin. Guidance Teacher: Meta ma tgħaddix, l-istigma tibqa hemm. Isiru l- injuranti u ma jagħtux il-kapaċità kollha tagħhom. Member of the Faculty of Education: This system labels children and it is having a very negative effect. Guidance Teacher: When a child fails, the stigma remains. They feel that they are stupid and do not do their best Moreover, there is a general perception that Area Secondary schools are not as well-kept as Junior Lyceums and that the environment inside the schools is rougher. 71

72 Membru tal-fakultá ta l-edukazzjoni: Lil dawk li jmorru s-sekondarja qed ngħidulhom: Il-Junior Lyceum mhux tajjeb għalikom, għalikom huwa addattat li tmorru fi skola anqas, allura se nibagħtukom fis- Sekondarja. Fis-Sekondarja ma jsibux affarijiet daqs kemm hemm fil- Junior Lyceum. Il-problema mhix il- Junior Lyceum imma l-fatt li s- Sekondarji mhumiex ta standard. Rappreżentant ta Trade Union: L-iskejjel kienu mitluqin għal żmien twil. Student: Skejjel maħmuġa, ħġieġ imkisser u ħafna bullying Student: Milli smajt hemm ħafna tfal kesħin. Student: Hemm ħafna bullying. Student: Niffaċċja ma nies li jdawruni għad-droga u l-alkoħol. Student: Ikun hemm aktar ċans biex tiġri mal-kurrent lejn il-ħażin. Member of the Faculty of Education: It is as though we are saying to the children that attend the Area Secondary Schools: The Junior Lyceum is not right for you, you are to go to an inferior school so we will send you to the Area Secondaries. The Area Secondaries are not as well equipped as the Junior Lyceums. The problem does not lie with the Junior Lyceums but with the Area Secondaries because they are not up to standard. Trade Union representative: Schools had been abandoned for a long time. Student: Dirty schools, broken panes and a lot of bullying. Student: I ve heard that there are a lot of silly children [show offs]. Student: There is a lot of bullying. Student: I will meet people who will lead me to drugs and alcohol. Student: There will be a greater chance to get into a downward spiral The system is having negative effects on teachers who are posted in Area Secondary Schools. Their expectations of students performance may also be affected negatively in this context. Rappreżentant ta Trade Union: Qed naqtgħu qalbna missekondarja. L-għalliema mhumiex kuntenti. L-għalliema jridu job b inqas stress u tensjoni. Trade Union representative: We are getting desperate in the Area Secondaries. Teachers are not happy. Teachers want a job that is less stressful and with less tension The current situation is such that pupils who go to an Area Secondary school still have the opportunity to move to a Junior Lyceum in any of the years by obtaining passes in a number of their annual examinations. The Notice in the Government Gazette (04/02/05) shows that in order to gain entry in a Junior Lyceum, students need to obtain at least 40% of the marks in the national annual examinations in the three basic subjects: English, Maltese and Mathematics; and in four of the following: one science subject (Integrated Science if they are sitting for examinations at the end of Form I or Form II and Physics from Form III onwards); one foreign language; Religion; Social Studies; Geography or History. Expatriates whose children join Secondary education in Malta need passes in five subjects: in 72

73 English and Mathematics; one science subject; one foreign language and Geography. Table 3.5 below shows the numbers of pupils who applied for entry into a Junior Lyceum from Area Secondary schools in the past four years and their success rates. Table 3.5: Numbers of Entries into Junior Lyceums: Form II to Form V Entry 2003/ / / /2007 into Form Apply Pass % Apply Pass % Apply Pass % Apply Pass % II III IV V Table 3.5 shows that, as expected, the numbers are largest for entry into Form II and they progressively decrease for every year group. These numbers suggest that among those students who end up in an Area Secondary, there is a considerable number each year who attempts to enter Form II in a Junior Lyceum, thus showing the desire to be with the others. It is worth noting that in the past two years the success rates have been very low. These figures indicate that in Area Secondary schools few students manage to obtain 40% in their annual examinations in a number of subjects. Such data continue to support the low achievement and possibly low expectations of students in this sector. Ironically, the current system is passing on the message to a significant proportion of children that they are not as academically able as their peers in Junior Lyceums. Yet they are presented with practically the same curriculum once they are sent to Area Secondary schools, a curriculum which the system was telling them was too difficult for them in the first place. Membru tal-fakultá ta l-edukazzjoni: Niftakru li din mhix għażla ta talenti. Qed ngħidulhom li mhumiex tajbin biex ikomplu l-iskola. Imbagħad nibagħtuhom fis-sekondarja u jagħmlu l-istess kurrikulu bħal dawk fil-junior Lyceums. Member of the Faculty of Education: We must remember that this is not a matter of selecting talents. We are telling the children that they are not good enough to continue their schooling and then we send them to the Areas Secondaries and expect them to cover the same curriculum as the students at the Junior Lyceums A study conducted by the School Social Work Service in 1997 and reported on in Children are Now Annual Report 2006 concerned 729 secondary school students who either failed to attend school completely for over a year or were 73

74 absent for long stretches of time during that year. The study concluded that there was a very low drop-out rate from Junior Lyceums for both boys and girls when compared to Area Secondary schools. The main reason for dropping out of school given by the participants as well as their parents was that they found school uninteresting. Other main reasons included bullying, school phobia, relationship with particular educators and the ease with which a medical certificate could be obtained. The study concluded that chronic absenteeism is one of many symptoms of alienation from schools. The findings of this study continue to support the views shared by different stakeholders that the concept of segregating pupils and sending them to different schools is perpetuating high rates of dropouts in the educational system The Report on School Attendance Improvement (2005) analysed different aspects, factors and causes of absenteeism and truancy, particularly visible in a number of schools and in certain geographic areas as well as the current pastoral provision that deals with absent children and their parents. Although the data are not broken down by year group, the overall picture immediately indicates a marked difference in absenteeism rates in the different types of schools. For example, on page 19 it is reported that in 2002/3 the total number of school days missed per pupil in Junior Lyceums was 13.4, while in secondary schools it was 30.0 and in ex-opporunity Centres it was Further analysis shows that unauthorised absences in Junior Lyceums are lower than in Secondary schools which in turn are lower than ex-opportunity Centres (Junior Lyceum 7.2; Secondary 16.3; and ex- Opportunity Centres 35.1 days per pupil). Authorised absences show a similar pattern with Junior Lyceums averaging 6.2 days per pupil; Secondary schools averaging 13.7 per pupil; and ex-opportunity Centres 24.1 per pupil. The results are similar for the following year. In 2003/4 the total number of school days missed per pupil in Junior Lyceums is 13.7; in Secondary schools it is 29.9; and in ex- Opportunity Centres it is Once again, unauthorised absences in Junior Lyceums are lower than in Secondary schools which in turn are lower than in Opportunity Centres (7.1, 17.3 and 34.1 respectively). Authorised absences show a similar pattern with Junior Lyceums averaging 6.5 days per pupil, Secondary schools averaging 12.6 per pupil and ex-opportunity Centres averaging 16 days per pupil. On average, males appear to be absent more often than females Hierarchy of Schools We need to acknowledge that locally we have a hierarchy of secondary schools. Within the state sector, there is a huge divide between Junior Lyceums and 74

75 Area Secondary schools. The high pass rate of the Junior Lyceum entrance examination, which currently stands at 61%, and the possibility of accessing a Junior Lyceum in any year of the secondary cycle continue to deplete Area secondary schools from a healthy mix of pupils. Church schools are generally considered on par with Junior Lyceums (especially girls schools) or even better than them (particularly some boys schools). In the case of boys Church secondary schools, where admission is dependent on the performance of the CE examination, there is a hierarchy between the different church schools belonging to different religious congregations. Generally speaking, most of the boys who pass the CE examination choose to go to a Church school even though they would also have passed the JL examination. Table 3.6 below shows the proportion of applicants from the three educational sectors in the past two years. As expected, most of the candidates come from the state sector which is the largest sector by far. Morevoer, there are both Church and Independent schools which only have primary schools and may be considered preparatory schools for these entrance examinations. Most of the candidates from the Church and Independent sectors would come from such schools. Table 3.6: Common Entrance Examination Applicants by Sector Year Applicants State Schools Independent Schools Church Schools N % N % N % Table 3.7 below shows the numbers and percentages of the successful candidates from the different sectors in the past two years. This table shows similar results to the table above. Most of the successful candidates who enter boys Church secondary schools come from the state sector. The current system suggests that the majority of the boys who do well at the CE and JL examinations opt to go to Church schools. Boys Junior Lyceums end up with very few high achievers compared to boys Church secondary schools and also girls Junior Lyceums. Table 3.7: Successful Common Entrance Examination Candidates by Sector Year Applicants State Schools Independent Schools Church Schools N % N % N %

76 High-stakes examinations at the end of primary school as well as the ballot system used for entry into Church schools at primary level have led a number of parents to send their children to Independent schools. There they are assured that their children have a place in an all-through school, where transition from the Primary to the Secondary cycle is not dependent on external factors Impact on Society The current system of segregation goes against the needs of the country where we need to motivate our children to stay in schools in order to increase the proportion of students who continue with post-compulsory education: Membru tal-fakultá ta l-edukazzjoni: Is-sistema ta bħalissa hija inġusta; qed tfixkel it-tagħlim u qed tfixkel lillgħalliema mill-vokazzjoni tagħhom. Hija sistema ħażina wkoll ekonomikament. Illum irridu 85% ikomplu jistudjaw, allura kif qed nifirdu lit-tfal u lil dawk li ma jgħaddux naqtgħulhom qalbhom? Membru tal-fakultá ta l-edukazzjoni: Illum tagħmel sens li ma nifirdux lit-tfal speċjalment meta niftakru li rridu nilħqu persentaġġi ta l-unjoni Ewropea f dawk li jkomplu jistudjaw. Member of the Faculty of Education: The present system is unjust; it is preventing learning and it is getting in the way of the teachers vocation. It is also economically detrimental. We are aiming at getting 85% of the students to continue with their education at Higher Education levels, so why are we selecting the students and further demotivating the ones that fail? Member of the Faculty of Education: Today it makes more sense for us not to select the children especially in the light of needing to reach the European Union targets for larger numbers to go on to Tertiary/Higher Education At this point in time it is worth widening the discussion by considering the results of the Programme for International Student Assessment (PISA). This is a three-yearly survey of the knowledge and skills of 15-year-old students in industrialised countries, and provides information for governments to draw policy lessons. PISA is concerned with students abilities for application of knowledge, their motivation to learn, their beliefs about themselves and their learning strategies. It is the product of collaboration between participating governments through the Organization for Economic Co-operation and Development (OECD) and draws on leading international experts to develop valid comparisons across countries and cultures. As pointed out in Grima and Farrugia (2006) among the results from PISA 2003 (OECD, 2004) are some very useful observations about the patterns emerging regarding the relationship between students performance and the organizational 76

77 structure of schooling in different countries. One of the results obtained by PISA 2003 was that countries with greater socio-economic inclusion tend to have higher overall performance. Other forms of segregation, pointed out by PISA 2003, resulting in lower overall performance can stem from educational policies that stream children into certain kinds of programmes early in their school careers. Students from Finland, for example, with a single structure school system were found to perform much better than students from countries that segregated students into a number of different types of schools at an early stage. One of the recommendations made by PISA 2003 was the reduction of segregation within schools of students of differing economic, social and cultural status, a recommendation which might require a review of classroom streaming practices PISA 2003 acknowledged the fact that catering for an increasingly diverse student body and attempting to narrow the gaps in student performance is a formidable challenge for any country. Different countries attempt to respond to diversity through educational systems located on a continuum ranging from systems with low stratification at system, school and classroom levels to systems that are highly differentiated. PISA 2003 has made a number of observations regarding this issue. Simple cross-country comparisons show that in countries with a larger number of distinct programme types (such as separate school tracks or streams), socio-economic background tends to have a significantly larger impact on student performance such that equity is much harder to realize Moreover, PISA 2003 also reported that separating students into different types of schools leads to a greater variation in student performance and to greater social selectivity. The report of the results of PISA 2003 provides possible explanations for these observations. If teaching homogeneous groups of students is more effective than teaching heterogeneous groups, this should increase the overall level of student performance rather than the dispersion of scores. However in homogeneous environments, while high performing students may profit from the wider opportunities to learn from one another, and stimulate each other s performance, the lower performers may not be able to access effective models and support. Another possible explanation suggested by PISA 2003 is that in highly differentiated systems it is easier to move students who do not meet certain performance standards to other schools, tracks or streams with lower performance expectations, rather than investing the effort to raise their performance. In addition, a learning environment with a greater variety of student performance and 77

78 backgrounds may stimulate teachers to use approaches that involve a higher degree of individual attention for students An important dimension of segregation into different school tracks and streams is the age at which decisions for separating students into different school types are generally made. Education systems with lower ages of selection tend to show much larger social disparities. PISA 2003 suggests that the reason why the age at which differentiation begins is closely associated with social selectivity may be explained by the fact that students are more dependent upon their parents and their parental resources when they are younger. In systems with a high degree of educational differentiation, parents from higher socio-economic backgrounds are in a better position to promote their children s chances than in a system in which such decisions are taken at a later age, and students themselves play a bigger role. The results from this comparative study of different educational systems show that systems with higher degrees of segregation seem to perform at lower levels and show a wider disparity in performance between schools than systems with greater inclusion. Similarly, the age at which segregation occurs seems to be another factor contributing to a country s overall performance such that countries in which differentiation occurs at a later stage perform better than countries in which students are segregated at a young age. 3.6 Impact on Certification and Life Chances An important aspect of an impact review is to consider the outcome of the educational experiences in the different types of school. This is a complex and ambitious issue and one which cannot be measured with any precision. An educational experience as such cannot be measured and much of what is gained is personal in terms of skills, values and outlook to life. An attempt will be made to give a glimpse of the outcome of such experiences using tools that are publicly available. These tools are the results of the SEC examinations which are taken by students at the end of secondary schooling and the tracer studies published by the Guidance Unit of the Education Division. Each of these will be used to provide evidence of the outcome of the experiences in the different educational sectors in Malta and Gozo Exploring the impact of the 11+ examination on secondary education and certification at SEC level 78

79 The 11+ examination has a wide-ranging impact on students. The immediate impact is that students are channelled to different types of school each with a different ethos and though they follow similar curricula, they do so at a different pace and aim at different levels of achievement. Students in Junior Lyceums and Area Secondary schools follow similar curricula with some exceptions in both types of school. However, Junior Lyceum students have more lessons and a wider selection of optional subjects to choose from after Form 2. The choice of school made by boys passing both sets of examinations has a significant impact on secondary schools since a large number of these high-ability boys decide to attend a Church school rather than a Junior Lyceum. The overall effect is a depletion of high ability students in the boys Junior Lyceums and a concentration of high-ability students in boys Church schools. At the same time, the boys Junior Lyceums receive boys from State, Independent and Church primary schools who passed the Junior Lyceum entrance examination but did not achieve a high rank in the Common Entrance examination. Boys from Church primary schools who fail both the Junior Lyceum and the Common Entrance examinations are generally admitted to the state Secondary Schools for boys though some of them may join Independent secondary schools. A similar impact does not occur to any significant extent in the case of girls Junior Lyceums. The 11+ examinations have practically no impact neither on the boys and girls Church continuous schools, where students are admitted by ballot in the early primary years nor on the Independent secondary schools There is also an immediate impact on the quality of secondary education that the students receive. This differs considerably depending on the schools ethos, teachers expectations, students self esteem and commitment, the quality of available resources, home factors and other factors such as private tuition that influence the students progress. Only a long-term study can show how each of these factors separately and in combination with one another affect students progress. This is beyond the scope of this review, however, it is possible to analyze the impact of the 11+ examination on the students achievements when these students complete their secondary education and sit for SEC examinations, which offer them the means to obtain external certification of their achievements at school. This impact can be investigated by analyzing the latest results of the SEC examinations by type of school and gender. The investigation is designed to answer the following four questions regarding the performance of students following different educational pathways: 1. Is the performance of Junior Lyceum boys different from that of boys coming from (a) Church schools where admission is by the Common Entrance 79

80 examination and (b) Church schools where admission is by ballot at the primary level? 2. Is the performance of boys from Church schools where admission is by the Common Entrance examination different from that of boys from Church schools where admission is by ballot at the primary level? 3. Is the performance of girls from Junior Lyceums different from that of girls coming from Church schools where admission is by ballot at the primary level? 4. Is there a difference between the performance of boys and girls from Independent schools where admission is neither by examination nor by ballot? The investigation 8 takes into consideration the results of the SEC examinations in English Language, Maltese, Mathematics, Religious Knowledge, Social Studies and Environmental Studies held in May 2006, which are presented in Tables 3.8 to 3.13 below. The results are given as the percentages of students who obtained Grades 1 to 7 and those who remained unclassified (U) by types of school. The first row presents the national results obtained by the 16 year-old cohort of candidates, while the last column shows the total number of 16 year-old candidates sitting for the examination and the number of candidates from each type of school. Table 3.8: SEC English Language Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 Grade ENGLISH LANGUAGE U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females Secondary schools from Gozo have been excluded because the analysis would have identified the performance of the individual schools that presented students for SEC examinations in The Area Secondary Schools have not been included but comments about the performance of their students will be made later. Absentees have not been included either. These are the reasons why percentages do not add up to 100%. 80

81 The SEC English Language results in Table 3.8 provide the following answers to the four questions of the investigation. 1. The Junior Lyceum boys performance is well below that of boys from selective and non-selective Church Schools and very well below Independent School males, especially when one considers Grades 1 to 3 and the percentage of students obtaining Grades 6, 7 and U (unclassified). However, most Junior Lyceum boys sit for Paper IIB and 45.2% of them manage to obtain Grade 4 or Grade 5, which is higher than the national average. 2. There is a difference between the results of boys from selective and nonselective Church schools but the performance of both groups is quite comparable. Both groups are among the top performers in this subject and well above the national average. Interestingly, the boys from non-selective Church schools perform just as well as girls in Church schools percentage wise; both types of school admit students by ballot in the early primary years. 3. The data show that Church school girls perform better than Junior Lyceum girls, though the difference is not as clearly marked as in the corresponding boys schools. This result shows that although girls are admitted into Church schools non-selectively and probably start off with a wide range of abilities in English, a high proportion of them obtain a very creditable result in the examination. 4. Candidates from Independent schools perform much better than the national average. The performance of boys and girls in Independent schools is comparable. There is, however, a higher percentage of boys who obtain the highest grade and those remaining unclassified. Almost three quarters of girls obtain Grades 1 to 3 and only four percent obtain Grades 6, 7 or U. Table 3.9: SEC Maltese Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 Grade MALTESE U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females

82 The SEC Maltese results in Table 3.9 lead to the following answers to the questions of the investigation: 1. The results obtained by Junior Lyceum boys are below those obtained by boys from selective and non-selective Church schools. It appears that few Junior Lyceum boys aim at obtaining Grades 1 to 3 and over 44% of them end up unclassified in the examination. Rather unexpectedly, this result is comparable to those of boys from Independent schools who are admitted without an examination. 2. The results of boys from non-selective Church schools are lower than those of boys from the selective Church schools. Indeed the latter group had the best performance in the examination which was well above the average of the 16 year-old cohort. 3. There is little difference between the results obtained by girls in Junior Lyceums and girls in Church schools. Almost equal proportions of both groups obtained Grades 1 3, and Grade 6, 7 or U. 4. There is also little difference between boys and girls in Independent schools; their performance is below that of the national average in the May session. Table 3.10: SEC Mathematics Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 Grade MATHEMATICS U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females The questions of the investigation can be answered in this way with reference to the SEC Mathematics results: 1. The performance of the Junior Lyceum boys is well below that of the boys from both types of Church schools. Less than 20% of Junior Lyceum boys obtained 82

83 Grades 1 to 3 which is much lower than the percentage of similar results of boys from Church schools and lower than the national average. 2. Both groups of boys from Church schools perform much better than the national average. Boys from selective schools have an exceptionally good result while boys from non-selective Church schools have a better result than that of all other groups. 3. Girls from Church schools, which are non-selective, perform better than Junior Lyceum girls. 4. Boys from Independent schools obtain results that are clearly higher than the national average. Girls are not too far behind, and their results are comparable to the national average though only a few of them obtain the top grade. Table 3.11: SEC Religious Knowledge Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 RELIGIOUS KNOWLEDGE Grade U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females The SEC Religion results point to the following answers to the questions of the investigation: 1. The results of the Junior Lyceum boys are again much lower than those of boys from Church schools and also below the national average except in Grade 5. About 27% of the Junior Lyceum boys obtain Grades 1 to 4, 20% just obtain a Grade 5 and 25% remain unclassified. 2. Boys from selective Church schools obtain better results than boys from the non-selective schools. However, the difference is not large and both groups perform better than the national average. A significant difference is that about one fifth of boys from non-selective Church schools end up with a Grade U though this result is slightly better than the national average. 83

84 3. The performances of girls from Junior Lyceums and Church schools are not very different from one another, though a higher percentage of Church school girls obtain Grades 1 and The percentage results of students from Independent schools show that the girls performance is clearly better than that of boys in this subject. The boys results are comparable to the national average while girls perform significantly better in fact performing better than non-selective Church Males, Junior Lyceum Females and Church Females. Table 3.12: SEC Social Studies Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 Grade SOCIAL STUDIES U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females Table 3.13: Environmental Studies Percentages of Candidates by Type of School and Grade Obtained SEC MAY 2006 Grade ENVIRONMENTAL STUDIES U N 16 year-old cohort Junior Lyceum Males Church Males (selective) Church Males (non-selective) Junior Lyceum Females Church Females Independent Males Independent Females

85 The results of SEC Social Studies and SEC Environmental Studies are taken together in this analysis as they are both related to the Social Studies assessed in the 11+ examination. Generally, Environmental Studies is more popular than Social Studies. Some schools present very few candidates for these subjects. Answers to the questions of the investigation are as follows: 1. Junior Lyceum boys perform lower than the national average in both subjects and clearly much lower than the boys from selective Church Schools, who show the best performance among all groups in both examinations. No meaningful comparison can be made with the performance of the small number of boys from non-selective Church schools who sit for these examinations. 2. The small number of candidates from non-selective boys Church schools does not permit a meaningful comparison with their counterparts from selective schools. 3. In SEC Social Studies, Junior Lyceum girls perform better than girls from nonselective Church schools as a higher proportion of them obtain Grades 1 to 3. In SEC Environmental Studies, the performance of both groups of females is comparable. 4. The SEC Environmental Studies results of students from Independent schools only show small differences between boys and girls, though boys perform marginally better than girls These observations indicate that selection by examination at the end of primary education and its consequences appear to have an impact on the performance at SEC level, when the students are at the end of compulsory schooling. In the case of boys, the consequences of the 11+ examinations are clearly very positive for boys from the selective Church schools. They had the best performance among all groups of students in practically all subjects. Junior Lyceum boys did not gain the advantage that passing the Junior Lyceum entrance examination and being educated in a Junior Lyceum should have given them. These observations are supported by the evidence from the SEC results, which show that in all subjects Junior Lyceum boys perform even lower than boys from non-selective Church schools and from the Independent schools who were admitted to these schools without an examination. The SEC results obtained by Junior Lyceum girls are generally comparable to those of Church school girls, though the latter obtain better results in English Language and Mathematics while Junior Lyceum girls obtain better results in Social Studies. 85

86 Independent school girls outshine all other groups in English Language examinations. The observations indicate that the absence of selectivity by examination in the case of Church school girls did not disadvantage them. On the contrary, these achieved very creditable pass rates in all subjects at SEC level So far, the results of students from the Area Secondary schools have been excluded from this investigation. These students, boys and girls, either failed or did not even attempt the Junior Lyceum entrance examination. Their failure to make it to a Junior Lyceum must have had a great negative impact on these students because many of them do not sit for SEC examinations and those who do so obtain poor results. Table 3.14 presents the results obtained by Secondary School boys and girls in the May 2006 SEC examination. The data include the number and percentages of boys and girls according to the grade obtained and the number registered for the examinations in English Language, Maltese, Mathematics, Religious Knowledge, Social Studies, and Environmental Studies. Absences have not been included The data in Table 3.14 show that hundreds of Secondary students attempt to obtain certification at SEC level. However, their overall results are poor and well below those of students in the other types of secondary schools. Moreover, boys perform at a lower level than girls. Suffice to note that only 3.5% of the boys and 5.3% of the girls who attempt the examination gain a Grade 5 or better in English Language. Likewise, only 5.1% boys and 18.9% girls in Maltese and 10.0% boys and 11.9% girls in Mathematics obtain Grade 5 or better in these core subjects. These observations indicate that selection at the end of primary education is not working to the advantage of these students, some of whom may have failed the Junior Lyceum entrance examination in one subject only. Although they are placed in a school where they can learn at their own pace, the large majority of the students do not attain a level that permits them to follow a post-secondary course at Advanced or Intermediate level. Many others do not even attempt to sit for the examination at SEC level Overall, the evidence from the SEC examination results may suggest that selection at age 11 has a negative impact on the students achievement at secondary school level. For boys, the two-stage selection process by means of the Junior Lyceum and Common Entrance examinations has a greater long-term negative effect than a positive impact on a large proportion of them. The evidence shows that while boys in selective Church schools are achieving high grades at SEC 86

87 level, Junior Lyceum boys are underachieving even when compared to boys from non-selective Church and Independent schools. Junior Lyceum girls are not affected so negatively since their achievement in SEC examinations is comparable to girls from the non-selective Church schools and Independent schools. Interestingly, boys and girls in Church and Independent schools who proceed from primary to secondary education without having to sit an examination perform creditably in SEC examinations. Non-selection at the end of primary education works well for these students even though it could mean that students with a wide range of abilities in various subjects are placed in the same class. Nonetheless, for them non-selection is not a disincentive to learning and it does not encourage students to study less. On the contrary, primary schools that are not concerned about a selective examination in the final year seem to provide the proper educational environment and support for all children. It appears that the less stressful the educational experience which children gain at such a delicate phase in their development, the better it is to help them meet the challenges of secondary education. Primary education, which is concerned with the holistic development of the young child rather than with the requirements of a selective examination in the final year, seems to be an important element for all children to succeed. Table 3.14: SEC Examinations Results of Secondary School Students in May 2006 Grade SEC MAY U Reg English Language Males N % Females N % Maltese Males N % Females N % Mathematics Males N % Females N %

88 Religious Knowledge Males N % Females N % Social Studies Males N % Females N % Environmental Studies Males N % Females N % Zooming out of the two systems of selectivity and continuity that are a reality in our schools allows us to observe the consequences on our school-leavers. Tables 3.15 and 3.16 below present the national data on SEC certification for the 16-year-old cohorts. One the one hand, there is reassurance that, on average, 40% of our school-leavers obtain SEC certification (grades 1 to 5) in six or more subjects. In fact, around 11% obtain SEC certification in 11 to 15 subjects. On the other hand, there is serious concern that around 33% of the same age cohort (that is, one in every three students) either sit for SEC examinations and obtain no passes or do not even attempt to sit for any SEC examinations. The proportion of noncertification is too high for our small country which relies heavily on its human resources especially when international statistics propose that at least 90% of school-leavers end up with certification that is of market-value Table 3.16 shows that even when SEC grades 1 to 7 are considered, the proportion of 16-year-olds with no certification is still too high, averaging around 25%. We acknowledge that the current range of SEC examinations is academic in nature with some exception as is the range of subjects available in our secondary schools. The issue of the secondary school curriculum (range of subjects, syllabi and pedagogy) as well as the spectrum of available school-leaving certification both need our urgent attention in an attempt to reduce the repercussions of the systems of selectivity. In addition to the challenge of increasing the national standard of the education system for all students, the findings of the current impact review support 88

89 two specific recommendations made by the MATSEC Review Committee (2005), namely introducing vocational subjects that lead to certification at the end of secondary education and developing a computerised examination system offering three-level certification options of basic functional competencies in five subject areas (Maltese, English, Mathematics, Science and Information Technology). Table 3.15: SEC certification (Grades 1-5) of 16-year-old-cohort SEC 11 to 15 6 to 10 1 to 5 No passes Non-Applicants N % N % N % N % N % Table 3.16: SEC certification (Grades 1-7) of 16-year-old-cohort SEC 11 to 15 6 to 10 1 to 5 No passes Non- N % N % N % N % A N li t % Exploring the impact of the 11+ examination on secondary education and students life chances The following table below provides an overview of the data of the tracer studies for the period 2003 to 2005 published by the Guidance Unit of the Education Division. The trends are similar in the different year groups. Unfortunately, the non-response rate was high for the three years for one sector: the secondary schools. However, with this limitation in mind it is still possible to observe certain patterns. There are significantly higher proportions of females who go to Junior College from the Junior Lyceums. About half of the females in the Junior Lyceum and of males and females from the Church schools opt for this academic stream. There are also more females than males from the single-sex schools in the Independent sector, who take the post-secondary academic stream. Moreover, the proportions are similarly high for students in the co-ed schools (data for the co-ed schools is not broken down for males and females in the tracer studies). 89

90 The Higher Secondary School is a smaller set-up. This post-secondary institution is equally popular with males and females from the Junior Lyceums. It is also popular with females from the Area Secondary schools and the Church schools and with students from the Independent sector. The most popular option for Junior Lyceum boys is MCAST. ITS is most popular with males from the Junior Lyceums and from Area Secondary schools. The students who frequent the private sixth forms come mostly from the Church and the Independent sectors. Over a third of Area Secondary students were registering for work or stayed at home at age Conclusion This section of the review has provided insights into the impact of the current selective practices at the end of primary education. It is evident that the impact is significant and affects the lives of many stakeholders. The impact is also felt by the system at classroom level and school level by means of different experiences, expectations, certification and life chances. The impact is also felt by the local society at large, with large numbers of students leaving the schooling experience with low self-esteem and no certification and the acceptance of segregation as a natural component of our lives. At this point in time, the unintended negative consequences of the current practices surpass the positive intentions and short-term positive impact felt in some schools. It is this reality that suggests an urgent need to look for an alternative which will make the transition less traumatic, the experience of schooling more relevant and holistic, an experience which will give everyone the opportunity to improve and achieve and eventually develop social cohesion in our local community. 90

91 91

92 4:0 Quality Review 4.1 INTRODUCTION Examinations are set for a purpose and the fundamental question about their quality must be whether they are fit for the purpose for which they are set. The two main 11+ examinations held in Malta at the end of primary education have rather different purposes. The Junior Lyceum (JL) entrance examination results are used to channel students either to a Junior Lyceum or to a general Secondary School where they follow slightly different curricula (syllabi, options, number of lessons per week). The Common Entrance (CE) examination serves the purpose of placing in rank order as objectively as possible those students seeking admission into ten Church secondary schools for boys before parents select the school of their own choice Fitness for purpose depends on two mutually supporting properties of examinations: validity and reliability. Validity concerns trustworthiness, that is, how well the subject of the examination has been assessed and the results interpreted. Actually, validity is a multi-dimensional concept, which includes considerations of construct, content, concurrent, predictive and consequential validity. Very briefly, construct validity refers to the validity of our understanding of the nature and content of the subject that is being examined, and this is specified in the syllabus. Content validity concerns the coverage of the construct by the examination. Concurrent validity relates to the degree of congruence between different assessments of the same subject taken by the same students concurrently, while predictive validity refers to how well the assessment predicts future performance. Consequential validity is essential especially in high stakes examinations such as the JL and CE examinations since they have a great impact on individual students, their families, the schools and the educational system as a whole Reliability concerns the consistency of the results. In the context of an examination, reliability depends mainly on consistency in marking and this includes consistency between different markers who are assessing the same task, and by the same marker over time. The importance of reliability of an examination cannot be underestimated because unreliability undermines the validity of the assessment. In examinations, reliability can be estimated by assessing internal consistency, that is, by comparing the performance of the students on different parts of the examination. This can be calculated statistically. Another method is to assess the consistency in marking between two markers of the same answer. This method is particularly useful for some tasks such as essay writing. 92

93 4.1.4 Besides validity and reliability, the quality of the examination also depends on practicality. This aspect concerns practical matters such as the appropriateness of the length of the examination for the target age group, the timing of the examination during the school year, the adequacy of the examination venues, supervision and security. Considerations of equity also contribute to the quality of the examination. Equity concerns access to all students who wish to take the examination irrespective of their disadvantage, whether it is due to disability or to economic and socio-cultural constraints The 11+ Review Committee decided to use a range of measures to evaluate the quality of the examinations as listed below. The Quality Review presents the main outcomes of these initiatives in the following sections, while the appendices carry the full evaluation reports as indicated in the text. The measures include: (a) (b) (c) (d) (e) An evaluation of the general characteristics of the JL and CE examinations using the World Bank indicators of good public examination systems. A quantitative analysis of the reliability and validity of the examinations. Evaluations of the JL and CE examination papers by local subject specialists. Evaluation of the JL and CE examinations in English and Mathematics by foreign subject specialists from Cambridge International Examinations (CIE). Analyses of the 2005 Syllabi by CIE subject specialists. 4.2 THE QUALITY of JL AND CE EXAMINATIONS: GENERAL CHARACTERISTICS There are no generally agreed criteria by which to evaluate a system of assessment. The checklist published by the World Bank was used. The document identifies five key characteristics and various indicators for each characteristic. An analysis of the Junior Lyceum (JL) Entrance Examination and of the Common Entrance (CE) examination was carried out by a member of each examination Board The Junior Lyceum Entrance Examination into Form The five key characteristics of good quality examination systems and their indicators are applied in the evaluation of the two examinations starting with the JL Entrance examination. In each case, an outline of the evidence supporting the evaluation is given. 93

94 1. Fitness for Purpose: The examination papers and the marking system should produce scores that are both reliable and valid. Indicators 1.1 Acceptance by teachers and students that the examinations are set according to the syllabus Junior Lyceum Entrance Examination Over the past ten years all paper setters made use of a Specification Grid to ensure that, amongst various other things, all test items are based on the knowledge, skills and processes laid down in the respective Primary State Schools syllabi. The Specification Grid for each subject is published in the Annual Report on the Examination to justify the inclusion of every test item in the examination paper by referring to the specific sections of the syllabus. In addition, the Specification Grid ensures that the marks allocated for the different strands that are being tested are in agreement with the established weightings in the assessment criteria. A subject specialist from the Curriculum Department heads each paper setters panel. Each panel is made up of three members, one of whom takes up the role of reviser to double check that no test item is out of the prescribed syllabus and that the paper is error free. The measures described above have resulted in the production of relatively high quality examination papers that, apart from having high validity, are candidate friendly and well-accepted by teachers, pupils and parents. Reliability is achieved by the use of triple marking of scripts which pass from the first marker to the subsequent marker without any marking signs to avoid influencing the next marker. Once the scores of the three different markers are transferred to a database, the computer identifies any scores that have variability greater than a preset threshold and, where this threshold is exceeded, the chief examiner decides on the final score that is to be awarded. 1.2 Statistical evidence of the examination s technical quality (e.g. level of difficulty) Every Annual Report includes an item analysis that indicates the facility index as well as the discrimination index for each test item. Various statistical measures are computed to provide policy makers, paper setters and school administrators with 94

95 evidence on the students performance both on a national basis as well as on a school basis. Data on the performance of the individual schools used to be published in the Annual Report but, as from 2004, this information has been omitted from the report to avoid potentially harmful comparisons among schools. Instead, the schools are informed of their own performance in relation to national results. 1.3 Adequate control measures resulting in few errors in examination papers As already indicated above, the utilisation of a Specification Grid, the thorough checking carried out by the reviser and the additional vetting of each examination paper by the test construction officers at the Educational Assessment Unit have resulted in error free examination papers. 2. Equity, Integrity and Public Confidence: The conduct of the examination system should be deemed fair and should achieve a high level of public acceptance. Indicators 2.1 The public has confidence in the results of the examination system 2.2 High level of trust in the honesty of the examination agency 2.3 High level of trust in the honesty of supervisory staff 2.4 Little evidence of candidates resorting to the use of unfair means (cheating) Junior Lyceum Entrance Examinations This is evident from the fact that the public has always accepted the results and, whenever parents requested a revision of paper, the outcome always strengthened the confidence in the reliability and robustness of the system. As a result, the number of requests for revision has declined steadily over the past eight years from 639 requests in 1999 to 359 in 2002 and 259 in Teachers have repetitively expressed their confidence in the integrity of the system and in the transparency of the whole process. Although supervision during the examination is carried out by the school teachers themselves, with the exception of those teachers who are involved in the teaching of Year 6 classes, there has never been a reported case of misconduct or irregular behaviour on the part of the invigilator or examination assistant. So far there has never been a need to take disciplinary action against a candidate. An invigilating teacher (not the class teacher) is assigned to no more than 20 candidates 9 The number of requests for revision of papers in each year from 1999 to 2006 were as follows: 639, 473, 399, 359, 378, 358, 297 and

96 in the pupils own classroom. 2.5 Examination authority has procedures for rechecking of marks and an appeals procedure The system includes a checking mechanism whereby the marks inputted in the database are checked one by one against the marks indicated on the markers marking sheets. Because three different markers mark each script separately, a check for variability is carried out by the database to ensure that the variability does not exceed preset limits. When these limits are exceeded a clerical check is carried out to ensure that the total score is correct and that the mark has been correctly transferred. If no errors are found, the chief examiner re-marks the script to decide on a fair mark the candidate actually deserves. An appeals procedure is in place and the service is given free of charge to all those who request a revision of paper. The process of reviewing involves the chief examiner meeting the candidate and the parents. Together they go through the whole script and review the candidate s performance in detail. A clerical check of marks awarded is carried out. 2.6 Special support is available for disadvantaged children 2.7 The question paper does not contain culturally inappropriate questions, or questions in a language with which some students are relatively unfamiliar The Educational Assessment Unit issues a call for applications for candidates requiring special examination arrangements. The Special Learning Difficulties Unit and the School Psychological Unit within the Education Division assess these candidates to determine the severity of their learning difficulty. Approved cases are eventually awarded special access arrangements which include a reader, extra time, prompter, scribe, amanuensis, enlarged print, communicator, etc. The number of requests for these access arrangements has increased sharply over the last five years, reaching a figure of about 10% of the whole number of candidates registering for the examination. 10 Examiners take extra care to avoid cultural biases. In addition, all examination papers are vetted for appropriate language to ensure accessibility to all candidates and not only to those with special learning difficulties. 10 The number of cases awarded special arrangements for each year from 1999 to 2006 were as follows: 164, 116, 159, 190, 179, 272, 339,

97 Children who do not profess the Catholic faith are exempted from sitting the Religion paper. Specific regulations apply for children of expatriates who may be exempted from Maltese, Religion and Social Studies. Returned migrants and some expatriates may be allowed to sit for the Social Studies and Religion examinations in English instead of Maltese. 2.8 The grading system is applied equally to all students Being a qualifying examination, no distinction is made among students in the awarding of grades. 3. Efficiency and Cost-Effectiveness: The examinations authority should deliver the required services, making the best possible use of physical, financial and human resources. Indicators 3.1 Examination fees do not place an excessive burden on parents and teachers 3.2 The Examination Authority can demonstrate cost effectiveness through accounting procedures. It can give a breakdown of costs including staffing, question paper preparation, printing, distribution, supervision, correcting, research publications, and exam costs by other agencies. 3.3 The Examination Authority is efficiently staffed 3.4 Examination papers are printed in the most cost-effective Junior Lyceum Entrance Examination These examinations are free of charge as they are state funded. The Educational Assessment Unit (EAU) is a section within the Curriculum Department which is essentially a branch of the Education Division. The Unit is responsible for the running of this examination from start to finish. Its employees are paid directly by the State. Setting, printing and distribution of examination scripts and delivery of results are carried out by the EAU. Teachers are recruited specifically to carry out the marking of scripts against payment. The Education Division covers all the expenses. The EAU employs three clerks, four teachers who are specialized in assessment and examinations, a Principal Education Officer specialized in Test Construction and Assessment, a Printer and an assistant. The Unit is under the responsibility of an Assistant Director within the Curriculum Department. The Unit is not only responsible for the JL Entrance Examination but also for national annual examinations in Primary and Secondary State Schools. The Unit copes with the current workload although at times it finds itself under great stress and pressure especially when human resources are reduced and when machinery requires maintenance/replacement during crucial periods. Printing is carried out on digital copiers located at the EAU to economize on printing 97

98 way without compromising security 3.5 Pre-examination administration is carried out in a timely manner 3.6 Results are issued in time and in an appropriate form for decision-making (e.g. selection for a higher level of education) 3.7 Reports on examination performance given to schools in time to influence instruction costs without jeopardizing printing quality. Printed papers are packed and stored in a secure room with restricted entry. The Unit is monitored by close circuit television. Timetables, registrations, allocation of candidates to examination centres, training of examination assistants to assist children with special access arrangements, suitability of examination premises and meetings with Centre Supervisors are all carried out satisfactorily and on time before the examination dates. Results are issued approximately six weeks after the examination period. Candidates/parents receive the results via text message (SMS) on their mobile phone and via mail at home the day after. Schools receive the results of their students the following day. The results are transferred from the EAU s database to the Operations Department s database in oder to carry out students placement in the respective schools. The EAU publishes a report on the examination in the first term of the scholastic year and delivers it to all schools. This report includes statistical information; the examination papers; the marking schemes; the Chief Examiners reports on the students performance, highlighting also the implications for teaching and learning; specification grids; item analyses indicating the facility and discrimination index for the main test items. 4. Transparency: The examination process should, as far as possible, be open to public scrutiny. Examinations should not be shrouded in mystery. Indicators 4.1 Non-confidential materials including regulations, syllabi, and sample/past examination papers are widely available Junior Lyceum Entrance Examination Notification for applications for the examination and the relevant regulations are advertised in the Government Gazette. An official Circular is also sent to all State and Non-State Schools with all the relevant details and regulations. Syllabi and sample/past papers are available in public libraries, in schools and online at Local councils meet requests for downloading these past examination papers for a nominal charge. 98

99 4.2 The Board involves teachers in the examination process (e.g. in syllabus construction and in marking) Apart from the past papers, the Curriculum Department s website also makes available the Chief Examiners Reports and the Marking Schemes used in the marking process. Education Officers in the Curriculum Department are directly involved in the design of the syllabus together with Subject Coordinators and other teachers. The paper-setters panels are composed of teachers and subject specialists. The marking process involves teachers who are subject specialists. Teachers who teach or give private tuition to candidates and those related to candidates are not allowed to participate in the setting and marking processes. The teachers that are chosen to conduct the marking attend a Coordination meeting with the Chief Examiner to discuss the details of the marking scheme and to make amendments as necessary. 4.3 Reports, including statistical data on examination performance are available 4.4 Marking system and criteria for grade awards are available 4.5 Criteria for employment of full-or part-time officials are published 4.6 The Examination Authority maintains records of administrative practices, results and marking schemes These reports are available at the Educational Assessment Unit, in Public Libraries and in schools. Schools receive performance data that compares their performance with the National performance. Marking schemes are published in the Report and online. Part-time officials are employed after a call for applications is issued by the Curriculum Department. These officials are required for paper setting and marking and must satisfy the criteria listed in the call for applications. Records of results are kept in hard copy format. Other administrative information is kept in hard copy form for a limited period of time as regulated by the Data Protection Act. 5. Beneficial Effect on Classroom Practice: The examination system should promote good teaching and learning practices. It should provide systematic feedback to teachers. Indicators 5.1 The Examination encourages the development of higher-order Junior Lyceum Entrance Examination Over the past ten years, there has been a progressive shift towards the testing of 99

100 thinking skills and does not place emphasis on recall of facts 5.2 The Examination promotes development of performance skills e.g. listening and speaking (languages), practical skills (sciences), production (art, drama) 5.3 Examination pressure does not exclude the development of non-cognitive skills (e.g. physical, aesthetic appreciation) 5.4 High quality subject reports for teachers and other interested parties (e.g. textbook boards) are regularly distributed understanding, and other higher order skills. However, there is still room for improvement in all five subjects. There are no practical examinations in place. Due to logistical problems, listening and speaking skills are not explicitly assessed in the language examinations. These are indirectly assessed in some cloze exercises and in written tasks involving dialogue. Examination pressure limits the range of non-cognitive skills developed. Subject Reports are published and distributed to local policy makers, Education Division directorates, Heads of School, teachers and other stakeholders. Extracts of these reports are also available online for use by the general public In conclusion, the JL Entrance Examination has been in place since 1981 and since then various measures have been taken to improve the examination system. These measures have collectively resulted in a transparent National Examination system that has earned the trust of the general public As has already been pointed out in the above analysis there are areas that require further development and attention. One particular area that is of great concern is the increasing number of requests for special examination arrangements. Currently, the EAU is encountering great difficulty in finding the required number of examination assistants to provide the necessary support during the examination itself. Although the increase in the number of special arrangements awarded could be due to the increased awareness of parents and teachers concerning the presence of learning difficulties, the alarming increase over the past three years may impact negatively on the trust in the examination system Since the way forward for a smooth transition from Primary to Secondary is being facilitated by the newly introduced College structure, it is time to consider whether the present examination system can be gradually transformed into a system which serves a dual function. It could serve to gather national benchmarking information for accountability purposes and to provide schools with the students attainment levels in order to help them take informed planning, teaching and 100

101 learning decisions. This issue will be explored in more detail in another section of the review report The Common Entrance Examination In this section, the same five key characteristics of good quality examination systems and their indicators are now applied in the evaluation of the CE examination into Form 1 of the boys Church Schools. 1. Fitness for Purpose: The examination papers and the marking system should produce scores that are both reliable and valid. Indicators 1.1 Acceptance by teachers and students that the examinations are set according to the syllabus 1.2 Statistical evidence of the examination s technical quality (e.g. level of difficulty) 1.3 Adequate control measures resulting in few errors in examination papers Common Entrance Examination The Regulations explicitly state that the examination is set according to the Year 6 Primary School Syllabus. As this is a competitive examination, it is scrutinized by all concerned and has been accepted year after year. A statistical analysis of the results for each paper is carried out to monitor the level of difficulty of each paper. The examination papers are checked by a panel after they are set by the paper setters. 2. Equity, Integrity and Public Confidence: The conduct of the examination system should be deemed fair and achieve a high level of public acceptance. Indicators 2.1 The public has confidence in the results of the examination system Common Entrance Examinations This is evident from the fact that the public has always accepted the results and, whenever parents requested a revision of paper, the outcome always strengthened the confidence in the reliability of the system. In fact, only a limited number of requests is made every year. (34 candidates requested a total of 125 revision of papers in 2004; 38 candidates requested a total of 102 revision of papers in 2005; 45 candidates requested a total of 144 revision of papers in 2006) Also, the choice of school is based on the global mark, made up of the marks of each subject. The Choice of School exercise works smoothly with hundreds of people participating. After the publication of results and the conclusion of the revision of papers, on a predetermined date, time and place, stipulated in the regulations the candidates and their parents are invited to attend a meeting 101

102 2.2 High level of trust in the honesty of the examination agency 2.3 High level of trust in the honesty of supervisory staff 2.4 Little evidence of candidates resorting to the use of unfair means (cheating) where they are asked to make their choice of school. On average, 175 candidates are invited for each session. Anybody else can attend. In the meeting hall, starting with the highest ranking candidate, each candidate is called by name. He and his parents proceed to an administrative centre and are invited to make their choice of school. These are entry examinations into popular schools. The examination in the present format has been in existence since 1990 and the Examinations Board enjoys a good reputation with the general public. Supervision is carried out by teachers from the participating schools. Teachers, whose students are sitting for the examination or who are relatives of a candidate do not carry out supervision duties. There has never been a reported case of misconduct or irregular behaviour on the part of an invigilator or examination Head of centre. So far, there has only been one case when disciplinary action had to be taken against a candidate. 2.5 Examination authority has procedures for rechecking of marks and an appeals procedure After the marker has finished marking a script, a checker carries out a number of checks. These include a check of the total of marks of each page, the correct transfer of the marks on each page onto the front page, the total of marks of the front page and the marks total on the front page against the total on the class sheet. Once the total has been inputted into the database, another checker checks that the total number of marks on the hard copy from the database is the same as that on the class sheet. An appeals procedure is in place as any candidate can request a revision of paper at a reasonable cost. The revision involves a rechecking of marks and re-marking followed by a report all of which are carried out by an independent reviser. 2.6 Special support is available for disadvantaged children The Regulations make provision for disadvantaged candidates. The Examination Board considers requests for special arrangements by candidates with certified difficulties. Cases approved are given appropriate examination arrangements such as: a reader, extra time, prompter, scribe, amanuensis, 102

103 enlarged print, communicator, etc. The number of requests for these arrangements is on the increase. Indeed, a growing concern is the increasing number of candidates requesting special examination arrangements. 2.7 The question paper does not contain culturally inappropriate questions, or questions in a language with which some students are relatively unfamiliar The Grave Humanitarian Cases Commission examines requests for grave humanitarian consideration. Cases approved by the Commission are exempted from the examination and are accepted directly by the schools. 11 Examiners take extra care to ensure that cultural bias is avoided. Children who do not profess the Catholic faith are exempted from sitting the Religion paper. Children who started living in Malta two scholastic years before the exam can be exempted from Maltese and those who started living in Malta only one year before the exam can be exempted from Social Studies. A candidate exempted from any paper has his global mark raised proportionally based on the exam results s/he sat for. Translations of the Religion and Social Studies papers are made available on request to those whose mother tongue is not Maltese. For Social Studies and Religion candidates can answer in either English or Maltese. 2.8 The grading system is applied equally to all students No distinction is made among students in the awarding of marks. 3. Efficiency and Cost-Effectiveness: The examinations authority should deliver the required services making the best possible use of physical, financial and human resources. Indicators 3.1 Examination fees do not place an excessive burden on parents and teachers 3.2 The Examination Authority can demonstrate cost effectiveness Common Entrance Examination The examinations fee of Lm12 (approximately 28) is very much within the reach of most people. The fee is waived for deserving social cases. The PSA Director of the Examinations annually presents a balance sheet with a 11 The Grave Humanitarian Cases Commission approved 19 cases in 2004, 24 cases in 2005 and 27 cases in

104 through accounting procedures. It can give a breakdown of costs including staffing, question paper preparation, printing, distribution, supervision, correcting, research publications, and exam costs by other agencies. 3.3 The Examination Authority is efficiently staffed 3.4 Examination papers are printed in the most cost-effective way without compromising security 3.5 Pre-examination administration is carried out in a timely manner 3.6 Results are issued in time and in an appropriate form for decisionmaking (e.g. selection for a higher level of education) 3.7 Reports on examination performance given to schools in time to influence instruction breakdown of the costs to the PSA Examinations Board. An itemised balance sheet is presented. The PSA Examinations Board is made up of Heads of school. Teachers, clerks, a printer and computer programmer give their services on a part-time basis. Given the part-time nature of these workers, sometimes work is done under pressure. However, the Board has always coped with whatever had to be done. The printing is carried out in-house in a secure room in order to minimize expenses and not to compromise on security. Timetables, registrations, allocation of candidates to examination centres, suitability of examination premises and meetings with Heads of Centres are all carried out satisfactorily and on time before the examination dates. Results are issued approximately 6 weeks after the examination period. Candidates/parents receive their result at home via mail. All the results are displayed for the public at the accepting schools. Results are organized by Index Number and show the mark for each examination, the global mark and rank order of the candidates. The results are published contemporaneously to the JL results so that parents would have all the necessary information to be able to make their choice of school two weeks later. The examination papers and answers, including a short statistical report, are sent to all preparatory schools. 4. Transparency: The examination process should, as far as possible, be open to public scrutiny. Examinations should not be shrouded in mystery. Indicators 4.1 Non-confidential materials including regulations, syllabi, and sample/past examination papers are widely available Common Entrance Examination Notification for applications for the examination is advertised in the local papers. A circular is also sent to all Preparatory Schools with all the relevant 104

105 details and regulations. A copy of the Regulations is given to each applicant and to whoever requests a copy. Syllabi are the same as those of Year 6 State Schools and are available from public libraries, schools and online at The Board involves teachers in the examination process e.g. in syllabus construction and in marking Sample/past papers are available at a small fee. The PSA Examinations Board is not directly involved in the process of syllabus construction as it makes use of the Year 6 State Schools Syllabi. The paper setters panels all include teachers and subject specialists. 4.3 Reports, including statistical data on examination performance are available 4.4 Marking system and criteria for grade awards are available 4.5 Criteria for employment of fullor part-time officials are published 4.6 The Examination Authority maintains records of administrative practices, results and marking schemes The marking process involves teachers who are subject specialists. Teachers who teach and give private tuition to a candidate or relatives of a candidate are not allowed to participate in the paper setting and marking processes. The teachers that are chosen to conduct the marking meet with the Chief Examiner to discuss any difficulties. A summary report is printed at the back of past papers when these are published. A brief statistical report is published every year. The marks of each item are indicated alongside each question on the examination paper itself and in the answers booklet. No grades are awarded as only raw marks are used. The officials of the Common Entrance examination are Heads and Assistant Heads of the organizing schools. A call for applications for invigilators and examination assistants is issued by the Examinations Board amongst staff of the organizing schools. Officials for paper setting and marking are selected directly by the Examinations Board without a call for application. Results and other administrative information are kept in hard and soft copy format for a limited period of time as regulated by the Data Protection Act. 105

106 5. Beneficial Effect on Classroom Practice: The examination system should promote good teaching and learning practices. It should provide systematic feedback to teachers. Indicators 5.1 The Examination encourages the development of higher-order thinking skills and does not place emphasis on recall of facts 5.2 The Examination promotes development of performance skills e.g. listening and speaking (languages), practical skills (sciences), production (art, drama) Common Entrance Examination Over the past years, there has been an attempt to test skills such as understanding and other higher skills. However, there is still room for improvement in all five subjects. There are no practical examinations in place. Due to logistical problems and the high-stakes nature of the examination listening and speaking skills are not explicitly assessed in the language examinations. 5.3 Examination pressure does not exclude the development of noncognitive skills (e.g. physical, aesthetic appreciation) 5.4 High quality subject reports for teachers and other interested parties (e.g. textbook boards) are regularly distributed This Common Entrance examination has replaced the various entrance examinations that were conducted by the different schools. However there is still considerable pressure on teachers, parents and children due to the high stakes consequences associated with the outcome of the examination. Development of non-cognitive skills is limited. No such reports are produced In conclusion, the CE examination was introduced as an objective criterion for accepting students into Church Schools. From the very beginning, the whole exercise was planned to be transparent to the public in each of its various stages. This transparency has generated trust and credibility. The Private Schools Association Examinations Board strives to strengthen continuously the mechanisms that regulate such a complex exercise. The CE examination has become a national examination taken by approximately 1400 candidates annually The structure of the examination provides for a ranking mechanism, which then makes it possible for candidates to go to the school of their own choice. This is of course dependent on the number of vacancies available in each school. In turn, this system builds up considerable pressure on the candidates as they strive to 106

107 obtain the maximum number of marks possible. Conscious of the importance of even one single mark on the rank order of a candidate, the Examinations Board gives the greatest attention to the setting and the marking of scripts, as well as the entire logistics of the examination itself One way of improving the examination further would be by specifying more precisely some aspects of the syllabi although these are quite well-articulated. However, an examination syllabus that defines more precisely what falls within the syllabus and what falls outside it will demarcate the boundaries more sharply for paper setters, teachers, parents and pupils Also, when a syllabus is changed, it would be very helpful for all to highlight all the changes made, instead of only publishing the new syllabus. Having an official statement which explains the rationale of the changes and states what has been eliminated and what has been freshly introduced will facilitate the implementation of the changes. 4.3 THE QUALITY OF THE 11+ EXAMINATIONS: A QUANTITATIVE ANALYSIS Introduction This section first presents a general background to the JL and CE examinations and then focuses on technical aspects of the validity and reliability of these examinations that can be described quantitatively. Other aspects of quality, which can be described qualitatively, are discussed separately in subsequent sections of this chapter The May 2006 Junior Lyceum and the Common Entrance examinations The JL and the CE examinations are prepared on the same syllabi, which were drawn up by Education Officers of the primary sector of the Education Division and published by the Ministry of Education, Youth and Employment. The English, Maltese and Religion examinations held in May 2006 were based on the syllabi published in 2005 while the Mathematics and Social Studies examinations were based on the syllabi published in Both examinations consist of five written papers: English, Maltese, Mathematics, Religion and Social Studies. There are some 107

108 differences between the JL and the CE examinations in the structure of the examination papers and the length of the examinations In May 2006, 3968 children (2183 boys and 1785 girls) sat for the JL entrance examination. These students came from State, Church and Independent primary schools and 52 came from Area secondary schools. In the same month, 1175 boys from Church, State and Independent primary schools took the CE entrance examination and competed for the 454 places made available in the Church boys secondary schools in Malta. In Gozo, 134 boys competed for 52 places at the Sacred Heart Seminary and 72 girls competed for 17 places in the Bishop s Conservatory School. In both examinations, students with special needs requested and obtained special examination arrangements The JL examinations took place in the mornings of 8-12 May, 2006 and were held in the students own schools under the supervision of school teachers, excluding the Year 6 teachers. The CE examinations took place in the afternoons from 4.30 p.m. to 6.00 p.m. or 6.15 p.m. from the 15 to the 19 May 2006 and were held in the schools offering places for admission under the supervision of teachers from the same schools. Marking took place soon afterwards. In the case of JL examinations, each paper was marked by three markers separately and the average mark of these markers was taken as the final score The JL results were published on 23 June 2006 and students were informed by text message (SMS) and in writing about the grades they obtained. Appeals were received by the 3 July and resolved by the 4 August. All students who obtained at least Grade C in the five subjects of the examination were offered places in Junior Lyceums. The individual CE results were sent by post to candidates on the 3 July and made available for public viewing on 4-5 July 2006 in the Church secondary schools offering places. Appeals were received by the 7 July and resolved by 19 July. The top 525 students in the list and their parents were then invited one by one according to their rank order to opt for one of the available places in the Church secondary schools. Students next on the list replaced students who declined the invitation until all available places were filled. 12 In 2006, the CE Examination Board made special arrangements for 88 candidates; nine candidates were exempted from Religion and one from Maltese; six candidates were provided with an English translation of the Social Studies paper and five with an English translation of the Religion paper. The Board also received 56 requests for consideration as grave humanitarian cases and accepted 27 of them. 108

109 4.3.3 Dataset and Analysis The analysis of the CE examination is based on the raw scores obtained by a randomly selected sample of 234 boys who sat for the examination in May The data included scores for whole questions. With these data, it is possible to obtain a statistical description of the global marks and the marks obtained in each subject. The reliability of the examination in each subject can be estimated using Cronbach s alpha and the standard error of measurement Six Church boys schools also provided the raw scores and the ranks obtained by 244 of their students in the May 2001 CE examination and the grades that the same students obtained in the May 2006 SEC examinations. With these data, it is possible to obtain a measure of the predictive validity of the CE examinations. The data also permit an analysis of other aspects of validity. Since the ranking of candidates can be carried out in different ways, it is possible to explore the consequences of using different methods of ranking students The analysis of the JL examination is based on the scores of 200 students (boys and girls) who sat for the examination in May 2006; these represented about 5% of the total number of candidates. The mean scores and standard deviations in the five subjects of the sample and of the whole cohort of candidates are compared in Table 4.1 as a check that the sample is truly representative of the population. The differences shown in the table are not significant statistically. As in the case of the CE examination, the analysis of the JL examination includes reliability estimates and standard errors of measurement. Table 4.1 Sample and population average scores and standard deviations (in parenthesis) obtained in the five subjects of the May 2006 JL examination English Maltese Mathematics Religion Social Studies Sample 51.4 (19.9) 67.1 (16.5) 63.8 (23.3) 74.9 (14.2) 65.5 (17.1) Population 49.8 (20.0) 66.2 (16.1) 61.5 (23.1) 74.0 (13.9) 64.7 (16.5) Besides the separate analyses of marks obtained in the JL and CE examination, a direct comparison between the two examinations is possible by considering the results of 205 out of the 234 randomly selected boys in the CE examination sample who sat for both examinations in May With these matched scores, it is possible to compare the distribution of marks in the five examinations and comment on the relative difficulty of the examinations. 109

110 Furthermore, calculations of the correlations between the JL and the CE examination scores in each subject would provide a measure of the concurrent validity of the examinations. This means that if the correlation between them is high and either the JL or the CE examination in each subject is taken as a criterion of validity, then the other examination would also be considered valid The next sections present the results of the analyses in the following order. First, the distribution of global scores is given for the CE examination only since the selection of students depends on their ranking according to the global scores they obtain. The global scores are not relevant in the JL examination since students pass by obtaining at least Grade C in each subject. Then the distribution of the scores obtained in each subject of the JL and CE examination are presented along with relevant statistical data that permit comparisons between the examinations. Next comes a section on reliability estimates and another on issues of validity Distribution of Scores The distribution of raw scores indicates the efficiency of the examinations in discriminating between students of different abilities. This is an important function of a selective examination, and concerns especially the CE examination. The CE examination papers are marked out of a maximum of 200 so that the maximum global score over the five subjects is 1000 marks. Figure 1 shows the distribution of global scores in the May 2006 CE examination for the randomly selected sample of 234 boys. Figure Distribution of of Global Global Marks Marks Common Entrance Examination May 2006 Common Entrance Examination May Frequency

111 The distribution is clearly not normal. Indeed no student obtained a score of less than 290 and only about 10 students scored less than 500, that is, 50% of the global score. Most of the boys were well prepared for the examination and obtained an average score of 705.3, that is, 70.5 percent. With such a distribution, it becomes more difficult to rank students because many students may be grouped in one rank and the difference between one rank and another may be minimal. Such a distribution demands high reliability in marking and great concentration from the students on the day of the examination since a small error in marking or inattention by the student may result in a drastic drop in rank order. Individual students may also perform better or worse depending on the particular questions and topics chosen for the examination with the consequence that they gain or lose in the ranking. The distribution of global scores in the JL examination is not presented since it is meaningless in the context of that examination Figures 4.2 to 4.6 show the distribution of marks by subject in both the JL and the CE examinations in Maltese, English, Mathematics, Religion and Social Studies. This figure shows marked differences between the distribution of marks in English, Maltese and Mathematics and the distribution in Religion and Social Studies. Figure 4.2 Maltese: Distribution of Marks for JL and CE Frequency MaltJL MaltCE Marks 111

112 Figure 4.3 English: Distribution of Marks for JL and CE Frequency EngJL EngCE Marks Figure 4.4 Mathematics: Distribution of Marks for JL and CE Frequency MathsJL MathsCE Marks Figure 4.5 Social Studies: Distribution of Marks for JL and CE Frequency SocJL SocCE Marks 112

113 Figure 4.6 Religion: Distribution of Marks for JL and CE Frequency RelJL RelCE Marks Most of the scores in the JL Maltese examination fall within the range 50 per cent to 100 per cent while those of the CE examination fall within the range 40 per cent to 90 per cent. Both ranges show a fair spread of marks and permit discrimination between students of different abilities in the subject. The peaks of the distributions are also different and show that most students obtained a score of about 70 per cent in the CE examination and about 80 per cent in the JL examination. Clearly, students found the Maltese paper of the CE examination more difficult than that of the JL examination. This is confirmed when the average marks and standard deviations 13 of 205 candidates taking both examinations are compared as in Table 4.2. The t-test 14 further confirms that the differences are significant. Table 4.2 Average scores and standard deviations in the Maltese examinations Maltese Junior Lyceum Common Entrance Average percentage score Standard deviation Paired t-test t = , df=204, p< The standard deviation (SD) is a measure of the spread of the marks; the higher its value the wider is the spread of marks. In an examination marked out of 100, the SD would be about The paired t-test provides a measure of the significance of the difference in average scores which members of the same group obtain on different tests. Statistical significance is shown by a p value. If the p value is less than 0.05 then the difference is significant; the smaller it is, the more significant is the difference. The term df refers to the number of degrees of freedom, which in the case of paired t-tests is one less than the total number of members in the group. 113

114 In the English examinations, the ranges of scores of both examinations are very similar. Both show a wide spread of marks as most students obtained scores between 30 per cent and 90 per cent. This spread permits good discrimination between students of different abilities. Fewer students obtained high marks in the JL than in the CE examination and the average scores in Table 4.3 reflect this distribution. The t-test shows that the differences are significant. Table 4.3 Average scores and standard deviations in the English examinations English Junior Lyceum Common Entrance Average percentage score Standard deviation Paired t-test t = , df=204, p< The distributions of scores in the Mathematics examination are apparently bimodal for the sample of students in this analysis as they show two peaks in both the JL and the CE examination. In the CE examination, the distribution is flatter as more students obtained lower marks than in the JL examination. This distribution allows a better discrimination between students than the distribution of marks in the JL examination. However, one must recall the different purposes of the two examinations. The students found the JL paper significantly easier, as reflected in the difference in the average marks and confirmed by the t-test Table 4.4 Average scores and standard deviations in the Mathematics examinations Mathematics Junior Lyceum Common Entrance Average percentage score Standard deviation Paired t-test t = , df=204, p< In Religion, the distributions of scores in both JL and CE examinations are practically the same. The great majority of students scored marks between 80 per cent and 100 per cent, as reflected in the relatively low standard deviations (Table 4.5). This narrow range of marks does not allow for good discrimination, as most students seem to have a relatively high level of ability in the subject and found both examinations easy. The t-test shows no statistically significant difference in the scores obtained in the two examinations. 114

115 Table 4.5 Average scores and standard deviations in the Religion examinations Religion Junior Lyceum Common Entrance Average percentage score Standard deviation Paired t-test t = 1.340, df=204, p> The distributions of scores in Social Studies show that the students found both the JL and the CE examinations relatively easy. Most students in the sample obtained scores between 60 per cent and 100 per cent in the JL examination and even more students obtained scores between 70 per cent and 100 per cent in the CE examination, with a high proportion of them scoring close to 100 per cent in the CE examination. The average scores and the relatively low standard deviations shown in Table 4.6 quantify these observations. The t-test confirms that there is a highly significant difference between the scores in the JL and the CE examinations. Table 4.6 Average scores and standard deviations in the Social Studies examinations Social Studies Junior Lyceum Common Entrance Average percentage score Standard deviation Paired t-test t = , df=204, p< Reliability Estimates Internal consistency Reliability refers to consistency in assessment results, that is, if students are assessed in a subject using equivalent forms of assessment, then the results from these assessments should be consistent and reflect the students true ability in the subject. Black and Wiliam (2006) identify three sources of error that threaten reliability in assessment: (i) any student may perform better or worse depending on the actual questions used for assessment, (ii) the same student may perform better or worse from day to day, and (iii) different markers may mark the same piece of work differently The 11+ examinations adopt three processes to avoid sources of error and to ascertain high reliability in marking. Marking of both JL and CE examinations is regulated by means of a detailed marking scheme, which is agreed to by the markers before the marking process. In both examinations, moderation by the chief examiner of each subject takes place to ensure consistency between markers during 115

116 the marking process. Moreover, in each subject of the JL examination, three markers score each script separately and their scores are averaged to obtain the final scores of each candidate The reliability of the results of an examination can be tested by estimating the internal consistency of the scores on different parts of the examination. Cronbach s alpha coefficient provides a good measure of the correlation between scores obtained in various parts of the examination and is therefore commonly used to estimate reliability. The alpha coefficient has a range from 0 to 1 and in a high stakes examination high reliability is indicated by a coefficient of about 0.85 or higher. Table 4.7 shows the results of calculations of Cronbach s alpha coefficient from the raw scores awarded to the sample of students on each question in the five subjects of the May 2006 JL and CE examinations. Table 4.7 Values of Cronbach s Alpha coefficient of the five subjects of the May 2006 JL and CE examinations Subject JL exam CE exam English Maltese Mathematics Religion Social Studies The values in Table 4.7 show that eight out of ten alpha coefficients exceed the 0.85 criterion indicating a satisfactorily high reliability. The other two values, namely, the alpha coefficient of the JL Religion examination and the alpha coefficient of CE English examination, are also close to the criterion value. Standard Error of Measurement The effects of the sources of error mentioned in paragraphs and on the results of particular candidates cannot be estimated; some candidates may gain while others may lose marks as a result of these errors of measurement. When large numbers of candidates are involved it is possible to describe the distribution of errors and to calculate its standard deviation from the reliability of the assessment and the standard deviation of the scores. 15 The standard error gives an indication of how far a student s actual score may be from his/her true score. The true score may be defined as the score which actually reflects the student s ability in the subject and it is the score that the student would 15 The standard error of measurement, s.e.m. = SD. (1-r), where SD is the standard deviation of the total scores and r is the reliability of the assessment (Cronbach s alpha). 116

117 obtain if there were no errors of measurement. Table 4.8 presents the standard errors of measurement of the JL and the CE examination. Table 4.8 Standard Errors of Measurement (s.e.m.) of the May 2006 JL and CE examinations Subject s.e.m. JL exam s.e.m. CE exam English Maltese Mathematics Religion Social Studies The values in Table 4.8 indicate the spread of the errors in each subject. These values may be interpreted as follows taking the s.e.m. for Mathematics as an example. (a) There is a 68% chance that the scores obtained by the students taking the Mathematics examination are within 5 marks (i.e. one standard deviation) of their true scores. In other words, if a student scores 50 marks in the examination, then there are two out of three chances that the student s true score in this subject is somewhere between 45 and 55. (b) There is a 96% chance that the scores obtained by the students are within 10 marks (i.e. two standard deviations) of their true score. That is, for a student who scored 50 marks in the examination, there is a 96% chance that the student s true score is somewhere in between 40 and 60. (c) There is also a 4% chance that the student s true score is from 10 to 15 marks above or below the score obtained in the examination It is important to note that the standard error of measurement is not due to defects in the quality of the examinations. It is an intrinsic characteristic of any examination and an indicator of the limitations of what examinations can do. In the 11+ examinations, its effect is felt most at the borderlines between Grades C and D in the JL examination and in the case of students whose ranking is close to the limit of the number of places available in Church schools Validity The quantitative data available allows a discussion of concurrent and predictive validity and a reference to consequential validity. These will be discussed 117

118 in the following paragraphs. Other aspects of validity, including construct and content validity are discussed in sections 4.4 and 4.5 below, which present the qualitative analysis of the examination papers. Consequential validity is discussed in more detail in the chapter on the impact of the examinations. Concurrent validity The JL and the CE examinations are held within a week of each other and many boys take both examinations in the same subjects based on the same syllabi. The matched scores of the random sample of 205 boys who sat for both the JL and the CE examinations permit the calculation of the correlations between the two examinations (Table 4.9). They show a high correlation in all subjects, that is, students who obtained a high score in a subject in the JL examination also obtained a comparable score in the CE examination. Similarly, students who obtained a low score in a subject in one examination also obtained a low score in the other examination. This evidence points towards high concurrent validity, that is, if the results of one examination are valid so are the results of the other examination. Table 4.9 Correlations between matched scores in the May 2006 JL and CE examinations Subject Correlation English 0.85 Maltese 0.82 Mathematics 0.87 Religion 0.74 Social Studies A more detailed analysis shows that many of the differences between scores in the two examinations occur at the lower end of the scale, that is, between the scores of students who obtain a low score in the two examinations. This is most evident in the case of students who scored 75% or less in Religion in both examinations and is reflected in the lower correlation between the scores in this subject when compared to the other subjects. Predictive Validity Six Church boys secondary schools provided data about the scores that 244 of their students obtained in the five subjects in the Common Entrance examination held in May 2001 and the grades that the same students obtained in 118

119 the SEC examinations held in May These data make it possible to estimate the predictive validity of the results of the CE examination, that is, how well the CE examination results predict the students performance in the Secondary Education Certificate (SEC) examination. Of course, during the five years of secondary education between the two examinations various educational, psychological, sociological and other factors come into play that may detract the students or make them more committed to their studies. Notwithstanding these intervening factors it is expected that students performance in the admission examination will be reflected in their SEC examinations, assuming that both examinations have a high degree of validity and assess the students abilities reliably in the various subjects for which they are set With the available data, the correspondence between the results of the two examinations can be shown in two ways. In the first method, the students are divided into five groups according to their raw score in each subject in the CE examination and the distribution of SEC grades obtained by each group is shown alongside. Students in the top 20% of a subject in the CE examination are predicted to obtain better grades than students in lower groups. The sharper the difference between the top and the lower groups, the better would be the predictive power of the CE examination. In the second method, predictive validity is found by means of the correlation between the results for each subject in the CE examination and in the SEC examination. Since CE results are given as scores and SEC examination results are given as grades, both results are used to put the students in rank order and Spearman s Rank Correlation is used to calculate the correspondence between the ranks in the two examinations Tables 4.10 a. to g. below have been prepared by using the first method of presenting predictive validity indicators of the CE Examination. The first table (Table 4.10a) shows the SEC English Language grades obtained by 242 students divided into five groups by their score in the CE English examination. In the top 20%, 18 students obtained Grade 1 and another 18 students obtained Grade 2, while 11 students obtained Grade 3 and one student was awarded Grade 4. The next 20% group of students also obtained high grades, but not as high as those in the first group. Similarly, students in the other three groups predictably obtained even lower scores as shown. Only four students out of 242 failed to obtain Grade 5 and, as expected, they were all in the group with the lowest scores in the CE examination. These observations show that the results of the CE examination in English had 119

120 substantial predictive validity as the differences between the groups are sharp and in the right direction. Table 4.10a. SEC Grades (English Language) CE English U Totals Top 20% nd 20% rd 20% th 20% Lowest 20% A sub-group of 85 students who sat for the CE examination in May 2001 also took the SEC English Literature examination in These have been divided into three groups according to their scores in the CE English examination and their SEC English Literature results are presented in Table 4.10b. The differences between the three groups are again quite sharp and in the expected direction, indicating a moderate predictive validity of the CE results. Table 4.10b. SEC Grades (English Literature) CE English U Totals Top 33% Middle 33% Lowest 33% In Maltese, the SEC results of students in the top, the fourth and the lowest groups follow the trend expected from the CE examination results (Table 4.10c). However, two groups of students show anomalous results. Students placed in the second group, according to their CE results, performed less well than predicted in the SEC examination, while students in the third group performed much better than predicted. A possible explanation for these anomalies is that the SEC Maltese examination has a whole paper on literature, which is not assessed in the CE and JL examinations. Table 4.10c. SEC Grades (Maltese) CE Maltese U Totals Top 20% nd 20% rd 20% th 20% Lowest 20%

121 In Mathematics, all groups of students performed exceptionally well in the SEC examination (Table 4.10d). A large proportion of students in all groups managed to obtain high grades. Indeed, more than half of the students in the lowest 20% group, according to the CE results, managed to obtain Grades 1, 2 and 3. Students in the first two groups obtained particularly excellent results. Yet, surprisingly, six students obtained Grades 6 and 7 and six other students were unclassified overall, including two students who were in the second group according to their CE scores. Table 4.10d. SEC Grades (Mathematics) CE Maths U Totals Top 20% nd 20% rd 20% th 20% Lowest 20% The SEC results in Religion also show exceptionally good performance by students in the third group, while the results of the other groups generally follow the trends predicted by their results in the CE examination. There are also several unexpected negative results, especially the relatively large proportions of unclassified results (U) found in all groups of students. Table 4.10e. SEC Grades (Religion) CE Religious Knowledge U Totals Top 20% nd 20% rd 20% th 20% Lowest 20% The results of the 156 students out of 244 who sat for the SEC Social Studies examination are presented in Table 4.10f. The differences between the five groups are not sharp and there is only a weak trend, which shows that students in the lower groups performed marginally less well than students in the higher groups. More students sat for the SEC Environmental Studies examination whose content is similar to that covered by the 11+ Social Studies syllabus. The results of the

122 students who sat for the examination are presented in Table 4.10g. These show the expected trend predicted from the CE Social Studies results with students in the top groups obtaining higher grades than students in the lower groups. There is however only a small difference between the SEC results of students in the third and fourth groups. Table 4.10f. SEC Grades (Social Studies) CE Social Studies Totals Top 20% nd 20% rd 20% th 20% Lowest 20% Table 4.10g. SEC Grades (Environmental Studies) CE Social Studies Totals Top 20% nd 20% rd 20% th 20% Lowest 20% The results presented in Tables 4.10 a. to g. show that, notwithstanding various intervening events and changing circumstances, the students performance in the CE examination can be used as a good indicator of how students will probably perform in the SEC examinations five years later. In other words, the CE examination has good predictive validity Predictive validity can also be quantified as mentioned above. Table 4.11 presents the results of the calculations of the correlation coefficients (R) obtained by means of Spearman s Rank Correlation for the ranks assigned to students according to their results in the 2001 CE and the grade obtained in the May 2006 SEC examinations. The correlations range from a moderately positive for English Language and a weak positive for Social Studies, all of which are highly significant. Reasons for the differences between the correlations have already been discussed in the previous paragraphs. One might add that it is possible to calculate the part of the variance in the SEC results which is explained by performance in the 122

123 CE examination. This is given as R 2 and is normally expressed as a percentage. Thus, the performance in CE English explains 27 per cent of the variance in the SEC English Language and the SEC English Literature results. Table 4.11 Correlations between ranks of students in the May 2001 CE and the May 2006 SEC examinations Subjects Correlated R CE English and SEC English Language CE English and SEC English Literature CE Maltese and SEC Maltese CE Mathematics and SEC Mathematics CE Religion and SEC Religion CE Social Studies and SEC Social Studies CE Social Studies and SEC Environmental Studies The performance in CE Mathematics only explains about 5 per cent of the variance in the SEC Mathematics results, mostly due to the large proportion of high grades that the students obtained at SEC level. The explained variances in the other subjects are 24 per cent in Maltese, 12 per cent in Religion, 13 per cent in Environmental Studies and 4.5 per cent in Social Studies. Consequential Validity Ranking of Students The results of the CE examination are published as marks obtained by individual students in each of the five subjects and students are placed in rank order according to the results obtained. The ranking process involves the addition of the raw mark obtained by each student in the five subjects and the student with the highest total of marks is assigned the first rank. Subsequent ranks are assigned to students with the next highest total of marks. Students exempted from one or more examinations are awarded pro-rated marks in the subject/s in which they were exempted This ranking process is the easiest to understand and is therefore most commonly used. This method assumes that a score in one subject is equivalent to a score in another subject. However, this may not always be the case. For example, an 80 per cent mark in Social Studies, where the majority of students scored more than 70 per cent, is very different from an 80 per cent mark in Mathematics, where the majority of students scored between 40 per cent and 70 per cent. One of the ranking processes that takes care of such differences involves the standardization of scores. In this method, the scores in the different subjects are treated 123

124 mathematically so that the average score and the spread of scores is the same in all subjects. The standardized scores in the different subjects are then considered to be equivalent and can be added without objections about comparability Clearly, different methods of ranking students produce different results. The method of standardization of scores was applied to the May 2006 CE examination marks and students were ranked according to their total standardized marks. When these new ranks were compared to the ranks obtained from the addition of raw scores, there were only small differences between the two methods. It is therefore reasonable to stick to the current method of ranking students by the totals of raw scores in the five examinations Conclusions Both the JL and CE examinations produce a wide spread of scores in English, Mathematics and Maltese which facilitate discrimination between students. In Religion and Social Studies, the distributions of scores are very narrow since most students obtain high or very high scores. This observation indicates that students have a high cognitive ability in these subjects and find the examinations easy; however, the narrow distribution of scores confounds the process of discrimination among students. In the CE examination, which ranks students according to their total score in the five subjects, the majority of students obtain 50 per cent or more of the total marks available. This observation shows that the candidates taking this examination are of high ability and this makes it more difficult to discriminate between them since the distribution of total scores is narrow. Setting examinations that are more difficult would not be an acceptable option because it would increase discrimination at the expense of raising the already high level of stress and anxiety among students and parents. Removing the Religion and Social Studies examinations to improve discrimination would not be a wise option, because it would probably narrow the actual curriculum covered in primary Years 5 and 6 even further The JL and CE examinations have a high reliability as indicated by the measures of internal consistency using Cronbach s alpha, which reach a value of 0.85 or more except in two cases where the value is between 0.80 and The standard error of measurement is fairly narrow in the ten examinations as it ranges between 3.8 and 6.6 per cent, which indicates that the scores are dependable. Of course, it is always possible to improve the reliability of the scores. This can be achieved by considering the item analysis of examination questions in order to 124

125 identify the factors contributing to the difficulty of the questions and their discriminating power. Paper setters can then consider these factors when they are constructing a new set of questions for the next examination. Improvements in the clarity and detail of the marking scheme accompanied by close moderation of marking also lead to scores of higher reliability The foregoing quantitative analysis considers only some aspects of the validity of the examinations. Concurrent validity between the JL and CE examinations is found to be high, ranging from a correlation of 0.74 to very high correlations of 0.82 to The comparison between the results that students obtained in the CE examination in 2001 and the performance of the same students in the 2006 SEC examination indicates that the CE examination has good predictive validity even though there is an interval of five years between the two examinations. The analysis discusses the methods of ranking students in the CE examination and finds that the method of adding raw scores and ranking by the global raw score produces results which are not very different from the method of standardizing the scores in each subject and ranking the global standardized scores These observations lead to the conclusion that the JL and the CE examinations are of a good technical quality. Strictly speaking, this conclusion refers to the examinations held in 2006, however, the degree of quality achieved has been built up over the years through experience and careful attention to the various processes involved. If the purpose of the examination were to be changed from that of selection to that of benchmarking, one would expect the quality to be maintained and possibly improved by constant monitoring of the setting of examination papers, marking, moderation, the checking of marks, and the publication and communication of results. 4.4 EVALUATION OF THE JL AND CE EXAMINATION PAPERS BY LOCAL SUBJECT SPECIALISTS The following sections present summaries of the evaluations of the English, Maltese, Mathematics, Religion and Social Studies examination papers by local subject specialists and summaries of the English and Mathematics examination papers and syllabi by foreign subject specialists. The detailed reports by the subject specialists are presented in full in Appendices 4 to

126 4.4.1 English The following paragraphs summarize the main issues highlighted in the analytical report of the JL and CE English examination papers for 2004, 2005 and The findings of the analysis have implications on the construct, the content and the consequential validity of the examinations The report first notes that both the JL and the CE examinations are based on the same teaching syllabus, which is different from what one would expect in an examination syllabus. The lack of a proper examination syllabus effectively means that there is no document that describes how candidates will be assessed. One consequence of a lack of proper syllabus is that the examiners can give different weightings to the language use, reading comprehension and writing components of the examinations, as evidenced in the different examination specifications. Furthermore in the absence of an examination syllabus, examiners may opt to reuse the same question/exercise formats which have been used in previous sessions. This decision would effectively freeze the shape of the examination and also the teaching of the subject. The absence of an indication of the vocabulary items which candidates are expected to recognize and use puts students and teachers in a difficult position on how to prepare for the examination The analysis of the examination papers begins with a general comment on the communicative language abilities tested by both the JL and CE examinations. By restricting the assessment of language to reading and writing and omitting listening and speaking, the examinations have given the construct of language ability a shape that is neither consonant with the teaching and learning that precedes the examinations nor with that which follows them. The report is critical of the excessive overall concentration on accuracy in all components of the examination and the insufficient attention given to the receptive skill of reading, especially in the CE examination, when this skill is particularly useful at this stage of education The evaluation then uses a common scheme, as appropriate, to analyse the different components of the examinations, namely, language use, reading comprehension, and writing skills. The scheme focuses on the test methods used; the response required of the candidates (production or recognition); the elements of language ability tested (e.g. grammatical structures, vocabulary), the meaningfulness of the test input or prompt; the topic choice, the difficulty level, cultural bias, and rubric, visuals and layout. The following paragraphs present some 126

127 of the issues highlighted in the analysis. The detailed analysis of the various exercises in the examination papers is presented in the full evaluation report, which is included in Appendix The evaluation of the language use component of the JL examination notes that all test items are presented in a realistic context which helps learners to achieve meaning. The topics in the test input are all appropriate and suitable for the candidates age; the rubrics are generally clear and the use of visuals even in this section of the examination is commendable. The emphasis on production items rather than on recognition coupled with the focus on accuracy increase the difficulty of the examination. In some exercises the level of the vocabulary and grammar structure, such as the present perfect, are beyond the candidates expected level. The coverage of the kinds of vocabulary and grammar tested is not systematic and the report remarks that, for example, punctuation and question formation are not tested in the papers under review The evaluation presents a number of positive and negative comments on the comprehension component of the JL papers under review, which relate to the construct and content validity of the examination. The comprehension texts are appropriate as they involve children, the questions are clear, layout of the text into paragraphs is helpful and the accompanying visuals are useful and attractive. The evaluation notes, however, that the same text type is repeatedly used in the three papers and the source is from children s literature. This choice communicates the message that the comprehension text will be a narrative with its inherent characteristics of chronology and past tenses while the syllabus indicates that the learners should come across different text types from various sources. The difficulty of the texts varies from year to year and it appears that all the texts were written for native speakers not second language learners. Some texts are pitched at a high level and the frequency of unfamiliar words in the texts renders them even more difficult for Maltese students. There is not enough clarity in this component of the examination, since besides demonstrating reading comprehension skills the students are required to answer in complete sentences and are penalised for incomplete sentences and syntactical errors. Some questions ask students to give the meaning of vocabulary items whose meaning cannot be deduced from the context. Such questions turn the task into a pure vocabulary test and not a test of reading skills The evaluation notes that offering a wide choice of prompts for the writing component as found in the JL examination papers may have undesirable 127

128 consequences. For example, the reliability of marking may be reduced because of the different demands that different writing tasks make on students. By further restricting the prompts to the assessment of narrative and descriptive writing skills, students may however focus their preparation on one writing skill only and exclude other writing skills mentioned in the syllabus. Varying the text types will make for greater construct validity as the writing skill will be more fully represented over the years. The evaluation notes that students are informed about the minimum number of words required but not the maximum. Moreover it suggests that it would make for greater validity and reliability if candidates were required to produce two short pieces of writing of different types rather than one long writing task. Reliability can also be improved by giving students some guidance on certain prompts since these can be interpreted in many different ways putting the markers in an awkward position in deciding whether the writing is out of point The report notes that in the CE English examination, the language use component tests an appropriate range of language elements through a series of topics suitable for the candidates age. The majority of exercises assess production skills rather than recognition skills. For example, students are asked to formulate questions and sentences, and to insert punctuation marks. This emphasis on the construction of responses renders the examination more challenging. The main limitation of this component noted by the evaluation is that most of the test input is at sentence level and consists of decontextualized sentences which bear no meaningful relationship to one another. This practice limits the testing of higher order language abilities, such as the ability to carry meaning from one sentence to another, and how the meaning of one sentence can influence the meaning of another. The evaluation suggests the use of short texts which create a mini context as this will allow the testing of a wider range of language elements. Indeed, in some exercises, the test input is a meaningful paragraph, which is suitable for the age of the candidates and works well The report points out that the reading comprehension component of the CE examination uses a variety of text types and a range of question formats with a commendable trend towards more use of multiple-choice formats. There seems to be also a trend in presenting a text whose difficulty is more appropriate for the expected reading ability of the students taking the examination. However, the report suggests that the topic of the text was not always suitable for young children. The ability to understand referring expressions is tested appropriately; however, the ability to infer is under-tested as most of the questions require candidates to locate 128

129 specific information from the text. A few questions may not be altogether appropriate to test reading ability. For example, questions about the meaning of words that cannot be worked out from the text and a question about similes are not expected in a language examination. Some aspects of the reading comprehension component need clarification, including consistency in the length of the text presented, which varies significantly from year to year, and the policy on the marking of spelling, punctuation and syntactical mistakes The CE papers offer five writing prompts in the writing component of the examination, as in the case of the JL examination. This practice elicits the same negative remarks already made on the JL examination about the problems of reliability in marking and the undesirable wash-back effect on teaching and learning. The evaluation comments about the validity of this component which presents a majority of prompts that require candidates to demonstrate descriptive and narrative writing skills. It is also important to avoid cultural bias which might inhibit candidates from expressing themselves in writing to their full benefit because they misunderstand the situation or they may not be familiar with the appropriate vocabulary. In general, however, candidates could identify with the majority of writing tasks set in the three examination sessions under review. The rubrics are clear and contain additional information which guide candidates. This makes the task more candidate-friendly with the added benefit of achieving a greater degree of reliability in marking Concluding comments: The existence of two parallel examinations testing largely the same skills is questionable. The possibility of collapsing the two examinations into one should be seriously considered and better use made of the available resources to improve the examination. The omission of the listening and speaking skills from these examinations has a significantly deleterious washback effect on an entire generation s language development and should be rectified. The drawing up of an examination syllabus should take top priority. In the process, several areas for improvement that are pointed out in the report will be dealt with Maltese The evaluation of the JL and CE examinations in Maltese focused on a comparison of the two sets of examination papers in terms of construct validity, 129

130 syllabus fulfilment, levels of difficulty, and language skills required. It also considered their appropriateness for their stated functions. The comparison took into consideration the two sets of Maltese examination papers for 2004, 2005 and 2006, as well as the JL examination papers that were set in 1981, 1989, 1999 and the CE examination paper set in The latter papers were useful in identifying developments in the syllabus. The following paragraphs present an outline of the main points in the evaluation report. The full evaluation is included in Appendix The analysis of the language section of the examination papers counted and grouped the questions/items that appeared in all the papers in seven categories, namely, lexical items, morphology, nouns, creative writing, proverbs/idioms, syntax, and various grammatical items, and recorded their frequency. The analysis showed that both in the JL and the CE examinations there was a positive trend towards presenting language exercises in a context, including the questions on proverbs and idioms. Both JL and CE examinations seemed to focus almost completely on the semitic element of Maltese, especially on constructions from mamma and għerq, while CE papers also tended to give more importance to lexical knowledge and general vocabulary. A range of grammatical items were present in the papers, though some appear more frequently than others, such as pronoun suffix, partiċelli, and numbers in words. The ability to answer correctly the language section in both JL and CE required a significant knowledge of meta-language, from the basic nom, verb aġġettiv, artiklu etc to stat kostrutt, imperattiv, to whole phrases and commands. In this respect, CE papers tended to make heavier use of such higher-end meta-language. Since 2004 both examinations have given more space and marks to creative writing, in the form of prompted paragraph/sentences or sentence continuation. The analysis by type of response showed a tendency to increase the number of items and the overall per centage of the marks for fill-in word/phrase answers, for both JL and CE papers and a corresponding diminution of importance given to the transformation / elicitation of sentences and to sentence completion. Multiple-choice has made a minor appearance in the last two years and word choice (from a bank of available words) has a minor but steady presence. Generally, over the period , the language content of the syllabus was well represented in both sets of papers indicating a good level of content validity in this aspect of language learning. 130

131 The comprehension section on all papers under review was analysed in terms of the type of response requested, of which there were the following: factual information, inferential information, vocabulary and grammar. In both examinations, comprehension-like exercises were included besides the questions on the comprehension text. In the CE examination there is a clear shift of marks from factual questions to vocabulary-based questions, with inference and grammar-based questions showing signs of an increase of difficulty as well. In the JL examination, the 2000, 2004 and 2005 papers generally follow the same trend but there was a predominance of factual questions in the 2006 paper. The 1997 Maltese syllabus makes explicit reference to comprehension as a reading skill that includes the skills of extracting specific information from a variety of texts, while the 2006 syllabus widens slightly the skill to including extracting of relevant information, and the understanding of words, phrases and expressions in context. Therefore, strictly speaking, inferential questions could only be included in the JL and CE papers as from However, in practice the conclusion must be that both panels of paper setters have very similar criteria of what constitutes testing a comprehension passage, although not on what constitutes an 11+ level comprehension passage. Indeed, in the CE examination, there is a general tendency over time for longer texts, shorter average sentences, and a higher ratio of idiomatic expressions (less words per idiomatic expression used). In the JL examination there seems to be a trend for longer texts, although again the shorter text set in 2006 may indicate a reversal of this trend. There is also a clear trend over time for JL comprehension texts to have longer average sentences, reaching the CE level for this measure. There is no clear general increase in the use of idiomatic language over time in both examinations in the language and comprehension sections of the examination papers. These observations on the types of questions, the length and difficulty of the texts, the idiomatic loading and compliance with the syllabus have clear implications for the construct and content validity of the examinations The analysis of the composition section of the examination papers showed that the most popular genre in both JL and CE is the narrative, followed by the descriptive and the expository. All the CE examination papers, but not the JL papers, gave the possibility for a picture composition. There was a clear shift over the years from titles to prompts. The 1981 and 1989 JL papers had straightforward titles; these gave way to titles with some explanation, to general prompts that required the adaptation or creation of a title for the writing piece. This could be due to the distinction in both the 1997 and 2006 syllabi between personal writing, creative writing and writing for a purpose, in all cases to a real audience and for a real 131

132 purpose. No picture composition had a title or further written prompts. With respect to the syllabus, the range of composition genres set in both JL and CE papers is significantly narrower than expected. The 1997 and 2006 syllabi either explicitly or implicitly refer to the need for pre-writing and re-drafting. One would expect that the result of this would be time being allowed during the exam proper for prewriting and re-drafting. The analysis also looked at the layout and illustrations in the examination papers and remarked on their possible impact on the students during the examination In summary, the comparative evaluation of the JL and CE examinations in Maltese in question, as well as of these with the relevant syllabi, indicate that: The present 11+ syllabus may, with caution and specific caveats, be considered appropriate for this age/ability level; The JL examination in Maltese is consonant with this syllabus and has the potential and most of the prerequisites to be transformed into a national diagnostic tool; The CE examination, with significantly more work per mark, higher idiomatic loading, use of higher-end meta-language, longer comprehension texts, and higher-order comprehension exercises, with less factual answers and more inferential ones is designed around a different assessment paradigm than that of the JL, with all the implications this may have to the implementation or otherwise of an assessment-for-learning approach at the critical primaryto-secondary point of transition Mathematics The evaluation of the Mathematics papers focuses on the validity of the examinations held in 2005 and in 2006 and constitutes an examination of the content and cognitive demands of the questions used in both JL and CE examination papers. The data used include the relevant syllabus, the examination papers, the marking schemes and the markers reports (refer to Appendix 6). The following paragraphs focus on comments about the examination papers The evaluation shows that the paper setters for the JL examinations are very careful to use language that is clear, brief and to the point. Diagrams are clear and unambiguous. The mark allotted to each question is declared in the opening rubric to the examination paper. Some of the questions can be worked out independently from the other items in the same question while others are structured 132

133 and suitably graded in difficulty. Although at first sight the layout of the paper appears to be very well organised, the evaluation report remarks that some questions require more working space to accommodate lengthier responses. More space for rough work would not only encourage candidates to demonstrate their mathematical understanding but also reflect the importance of using rough work when doing Mathematics. In the case of the CE examination papers, the report notes that the language is carefully chosen to be clear and to the point, thus avoiding unnecessary linguistic demands on the candidates. Diagrams are clear and unambiguous. Although the papers are write-on and the space for each question is necessarily limited, the candidates in the CE examination had ample space for the working. This happens because the questions are printed on one side only, leaving the opposite blank pages for any additional workings. Questions are generally suitably graded and divided into smaller items carrying one or two marks In discussing the distribution of content across the four papers, the report remarks that Problem Solving ought to be seen as an on-going process spreading across the four content domains identified in the syllabus namely, Number, Measurement, Shape and Space, and Data Handling rather than being treated as a stand-alone strand to be assessed. Considering the domains separately, the analysis shows that all the syllabus subsections of the four content domains are tested in both the JL and CE examinations. The Number category is allotted most marks in all examination papers reviewed, though the allocation of marks in the CE examination papers is slightly on the higher side. The report also notes that although the Number category is given roughly the same weighting in all four examination papers reviewed, the mark distributions across the subcategories are very dissimilar for example, in terms of the weighting given to Fractions, Decimals and Percentages The next focus of the evaluation is the cognitive demand of the items in the papers reviewed in terms of three categories: knowing, applying and reasoning. 16 Knowledge of mathematical facts, conventions and notations, mathematical procedures and concepts fall under the category of knowing. The solution of routine problems would require the application of knowledge whereas non-routine problems would entail more demands on reasoning. 16 For a more detailed interpretation of these three categories in the content domain please refer to the full report in Appendix 6 (pp.5-7). 133

134 With regard to the cognitive demand of the papers, the analysis shows that a good proportion of the marks are allotted to items involving reasoning. This applies to both JL and CE papers. The marks allotted to items involving reasoning come mostly from items testing the Number content domain. In the JL papers, most reasoning items come from situations involving whole numbers and, from a few items involving decimals and percentages. On the other hand, in the CE papers the reasoning items testing Number are more varied to include a wider representation across the Number subsections. In fact, there are more reasoning items testing fractions, decimals and proportion than in the JL papers. Both the JL and CE papers include a few reasoning items testing Measurement, Shape and Space, and Data Handling though the items involving the latter are very scarce. All four papers reviewed feature some good quality reasoning items embedded in real-life situations with the exception of a couple of questions that require a True/False answer. A reasoning item in one of the CE papers also requires a written explanation rather than a numerical answer The main conclusions and recommendations are: In write-on examinations, it is important to ensure that ample working space is provided to allow for different methods of solution and for candidates use of rough work. True/False items should be avoided unless a written explanation is demanded from the candidates to justify their choice. More items requiring short verbal written explanations should be included. In the classroom, the use of such items helps students learn to articulate their mathematical reasoning and to develop their communication skills. Both JL and CE papers assess widely the different categories or strands in the syllabus. They also reflect a good spread of item difficulty and feature clear language and unambiguous diagrams. Reasoning items need to be better spread out between the syllabus content domains. In particular more such items need to target the understanding of fractions in the Number content domain. Measurement, Shape and Space and Data Handling also deserve a better representation in terms of Reasoning items than is the case in some of the papers under review. The publication of the facility index of the individual test items is a useful resource for both Primary teachers and paper setters. Examinations need to be aligned to the curriculum. The rationale of the syllabus for the examination session exhorts teachers to avoid carrying out tests which focus on a narrow range of skills such as the correct 134

135 application of standard algorithms (Department for Curriculum Management, 2006, p.11). This will not happen in content areas where the high-stakes examination itself focuses on a narrow range of skills Religion The evaluation of the Religion examination (refer to Appendix 7) starts by examining the syllabus and each of the JL and CE exam papers of the last three years. It then concludes by examining pertinent issues relating to the papers and makes some recommendations. This section presents the analysis of the examination papers. The evaluation of the syllabus is presented in section below. The evaluation points out that Religious Education (RE) is taught in almost every country of the Western World but there is no single model and each model is deeply contextual. In Malta, Catholic RE in State Schools is guaranteed by the Constitution, the Education Act and a Treaty between the Holy See and the Republic of Malta. Since 1988, Religion has been assessed as one of the subjects of the JL and CE examinations The evaluation notes that the examination papers are set in Maltese, although a few candidates opt to have an English translation. The papers have no rubric but the language used is simple, with no gender bias or incorrect use of language. The CE examination papers are more demanding with regard to content and the amount of writing required. The Eucharist and the celebration of the Holy Mass were assessed in all papers while Christ s mediating role is the least assessed. The papers assess mainly the lowest levels of the cognitive domain and, on average, the JL papers have a better balance between understanding and recall items than the CE papers. Words and phrases from the textbook are often used in the examination papers thus encouraging learning by heart. The evaluation points out that affective content is minimally assessed but notes that it is difficult to examine The evaluation comments about each paper and analyses each item. The analysis indicates the section of the paper and the type of question, the question number, the textbook chapter where to find the content, the domain (cognitive or affective) and domain level, marks, total of marks per exercise, and comments wherever necessary. 135

136 The JL 2004 paper mainly covered content from the second and third scholastic terms of Year 6. The distribution of marks was fair though it would have been helpful if the marks allotted to each item were shown on the question paper. The use of graphics is commendable. The wording of Section D is slightly ambiguous but was taken into account by the marking scheme. The JL 2005 paper had a fairer representation of the Year 6 content and focused on remembering what had been taught. Distribution of marks was fair but fewer images than the previous year were used. The JL 2006 paper was the shortest of the papers examined. The instructions given were clear and the distribution of marks fair. Only five images were used and although related to the content, they did not help students in their processing. The questions were mostly of the recall type but the students were left free to choose from amongst three themes and were not given words or phrases that they had to use. The evaluation comments about the paper setters oversight of including content from chapters 7 and 12 (of the 2001 edition of the textbook) which are not part of the new 2005 syllabus The rubric of the 2004 CE paper included the time allotted and a statement that candidates can answer in either Maltese or English. Each section had a title indicating the topic being assessed. The paper was rather long but candidates could complete it in the time allotted. The distribution of marks was fair and 88% of the marks were for recall items. The language used was clear and to the point, few images were used and one particular image, whose meaning was self-evident, was outdated. An example of how this image can be improved is given. The evaluation concluded that the 2005 CE paper was more balanced, giving candidates the chance to recall information, to show understanding, awareness of or sensitivity to certain ideas and commitment to attitudes and/or values. Text was more spaciously presented but the section headings were not included. The language used was generally clear and to the point. Although few images were used they were well chosen and appropriately placed. The evaluation commended the exercise of the short essays where the candidates were asked to assume the role of an object or animal. The evaluation noted that the 2006 CE paper assessed the cognitive domain only. The questions in this paper were all of the recall type. The language used was 136

137 simple and the instructions given were clear. No images were used except for Section H. The evaluation observed that like the JL 2006 paper, the CE paper included content from Chapters 7 and 12 which were not part of the new 2005 syllabus The evaluation examined the paper setters aim of testing both the cognitive and the affective domain and finds that the latter is only addressed through a small number of items accounting for an average of 12.33% of the marks. The evaluation pointed out that when compared to the other four subjects, the Religion examination has the highest number of passes, however only a small amount of items, accounting for an average of 11% of the marks, turned out to be non-discriminatory The evaluation concluded by recommending a change from Religion (Religious Knowledge) to Religious Education, both in the teaching of Religion and in the examination papers. The subject needs to help students move beyond memorisation. It should help students understand the themes, and be analytical and critical. The evaluator insisted on clarifying objectives since these influence the writing of textbooks and the design of examination papers. It also recommended one examination instead of the present three (Year 6 annual, JL and CE) Social Studies The evaluation focuses on the content analysis of the Primary Social Studies syllabus and the JL and CE examinations for 2005 and 2006 (refer to Appendix 8). This section outlines the comments on the examination papers. The comments on the syllabus are given in section below The evaluation notes that some of the set examination questions do not address any of the presumed learning outcomes of Social Studies, but call for a recollection of what is understood in society as proper a format of response that is typical in a primary school environment and one with which teachers would be very familiar and comfortable. Given the examination paper format, it is problematic for paper setters to come up with suitable instruments that measure skills and attitudinal aspects found in the syllabus. Thus, what the examination actually measures is a reported understanding of the required skills and attitudes, rather than the skills and attitudes per se. These can be learnt irrespective of and separately from the actual acquisition of the skills and attitudes themselves. Thus, 137

138 the examination papers are more suitable for testing formalism namely, whether the student has studied or has been taught these concepts in a sufficiently proper manner for the same student to remember and reproduce what they are about, rather than whether the student has actually achieved the curriculum objectives. Past examination papers represent the manner in which the curriculum is actually translated into an examination format. The analysis reveals a consistent style of paper setting, question formatting and language use, with similar questions being asked in some instances. The format of the examination forces the curriculum to emphasize factual understanding, even if in a subliminal way. Therefore, if one needed to change the curriculum, the simplest way would be to reform the examination. Moreover, if the current examination practice and format persists, one can never discriminate as to whether an answer had been committed to memory or is the outcome of reflection by the student during the examination proper. The evaluation also remarks that some of the exercises in the Social Studies papers could be lifted as they are and reproduced without difficulty in an English or Maltese comprehension or general language paper. Nevertheless, some examination questions are well-crafted to the extent that the three aspects, namely, the social/human/civic, the physical geographic and the historical aspects are assessed in the same question. This comment holds particularly for questions in the CE examinations. The JL examinations however tests these three areas separately Although there are close similarities in both sets of papers concerning timing, number of questions, types of exercises, images and maps, there are some basic consistent differences between the JL and the CE examination papers. The CE papers are more wordy, especially in comprehension type questions, putting more time, energy and attention demands on the pupils. The CE papers are occasionally less graphic, having fewer questions supported by images and/or with images occupying less space. The CE papers have clear examples of questions that elicit knowledge and understanding of Social Studies holistically whereas the JL papers are consistently organised in terms of social or physical-geographical or historical material, with hardly any interaction among the three. Overall the CE papers take longer to complete and are less candidate-friendly than the JL papers The main conclusions are: It is doubtful whether the JL and CE examinations as they stand are actually inculcating a sense of admiration, appreciation and respect for our human, geographic, natural and historical environment. 138

139 The examination format is ill-suited to assess any achievement of competence, attitudes or values as it focuses on the ability to transmit knowledge and understanding. Social Studies has the potential to be exciting, engaging and stimulating perhaps more than any other primary school subject. It is important that assessment is carried out in such a way that teachers are encouraged to develop the subject matter in a more exciting, innovative and engaging manner, bringing history, nature and geography alive and making them tangible and relevant to the students. The JL and CE papers are suited to test what students have been taught. They do not encourage any interaction across the three areas of Social Studies. The examination papers are reliable as far as the accurate measurement of correct answers goes, but their validity is questioned since they cannot measure all the stated objectives of the syllabus. 4.5 CAMBRIDGE INTERNATIONAL EXAMINATIONS (CIE) EVALUATION OF THE MAY 2006 JL AND CE EXAMINATIONS Introduction Further to the evaluation of the quality of the JL and CE examinations by local subject specialists, the Review Committee thought that the evaluation would be enhanced by the input of foreign subject specialists who are familiar with the assessment and testing of pupils at the primary level. Therefore, the Committee invited Cambridge International Examinations (CIE) to evaluate the JL and CE May 2006 examination papers in English and Mathematics. They were also asked to evaluate the 2005 syllabi. CIE have been selected for these tasks since they have experience in producing Key Stage 3 tasks in English and Mathematics for testing at the national level in England. They have also produced a Primary English Curriculum Framework, broadly based on the English National Curriculum, for international use as part of the Cambridge International Primary Programme. A Primary Achievement Test for the end of primary education and Progression Tests for interim stages accompany this Framework The Review Committee provided CIE with copies of the May 2006 examination papers, the 2005 syllabus on which they were set, the specification grids, marking schemes and the examiners reports for the 2005 examinations. The evaluation took place over two days. The following sections present a summary of 139

140 the main points made in the reports. The full texts of the reports of the evaluation are available in Appendices 9 and The May 2006 JL English Examination The CIE evaluation of the JL English examination focuses on the three sections of the examination, that is, language use, comprehension and composition. The remarks on each section relate to the content and construct validity of the examination. Content validity relates to conformity with the published syllabus as far as coverage and weighting of marks are concerned. Construct validity relates to the appropriateness of the examination for the examinees considering their age and language competence, and the nature of the examination Comments on the language use section of the examination highlight the importance given to it as reflected in its weighting, which is 40 per cent of the total marks. The heavy emphasis on correct grammatical usage of parts of speech and verb tenses within discrete sentences is greater than might be expected from the 2005 syllabus. The evaluation report noted that this section assessed the students ability to deduce meaning from context, though the context was that of sentence structure rather than a broader context. The questions did not probe richness of vocabulary or understanding of word meanings. Some of the questions in this section also tested logic and reasoning, as in a verbal reasoning test. In this sense, they are likely to be a good indicator of general ability, rather than showing specific skills in reading and writing. A comment that concerns construct validity is that the emphasis on the structure of grammatically correct sentences rather than on the application of grammar in the course of reading and writing may reflect the needs of second language learners, but this is not explicit in the syllabus, or in the structure of the examination The report noted that in the section on comprehension, questions are varied in presentation and only a few of them required the simple lifting of words from the text. The questions show progression from literal to inferential comprehension, as detailed in the syllabus. Many test the understanding of word meanings, using different words in the question to those used in the text. Later questions require inferences about motivations and feelings and an element of 'reading between the lines' about what is not explicitly stated. There was little testing of children's ability to draw out themes and ideas from the story as a whole, however, or to link information from different parts of the text. 140

141 Remarks on the composition section noted that the examination followed the syllabus in some aspects but there was inconsistency in others. The examination gave a choice of five topics for writing offering the genres of narrative, descriptive and letter writing, in line with the Year 6 syllabus. The requirement was for 'not less that 150 words', matching the syllabus requirement that pupils in Year 6 'be able to write a composition of not less than 150 words'. However, this appeared to be a fairly modest requirement, perhaps reflecting examination time constraints. Shorter writing is pegged to a lower band in the mark scheme, but the examiner's report (2005) made no overall comment about length in relation to quality. Comparison between the syllabus, the mark scheme and the specification grid revealed some inconsistencies. The syllabus refers to writing legibly and using correct language forms, spelling and punctuation. The mark scheme focuses on grammar, spelling and punctuation, vocabulary, and coherent and appropriate organisation, but does not include handwriting. It provides descriptors for each of the elements for each band of marks but does not show the weighting of each element or how marks are to be allocated. The specification grid does not relate directly to these descriptors but shows an allocation of 10 marks each for planning work and writing logically, for writing 'relevantly and functionally' and for writing 'accurately and creatively. These attributes are respectively described as 'low', 'medium' and 'high' in level of difficulty. This categorisation lacks precision and does not fit easily with the descriptors in the mark bands. These differences may reflect negatively on construct validity and on reliability, since insufficient guidance is provided to markers. By contrast, guidance in the mark scheme is very specific about the deduction of marks for irrelevance, or an incorrect format for the letter The report also commented about the face validity of the examination paper. Positive comments were made about the comprehension section. The fact that the reading passage was broken into three half-page sections was considered a helpful device. Furthermore, the first three questions all related to the first paragraph. The questions broadly followed a chronological sequence, or flagged the section to be drawn on. One question (a sequence of true/false statements) required some scanning across the sections, although the statements to be tested appeared in order in the text. The questions were graded and showed progression from literal to inferential comprehension. The composition section of the paper attracted some negative remarks. The report noted that the layout of the composition page of the examination paper was unappealing and somewhat confusing. The guidance given varied among titles. While two offered no additional instructions, the letter option had a dense paragraph setting the context and suggesting what to include. The 141

142 story in pictures was expected to attract pupils because of the visual prompts. Moreover, although the syllabus does not refer specifically to the planning of writing, the editing and revision of work is mentioned. This expectation was not made explicit in the examination rubric, and no framework was provided to help pupils plan their writing, although planning was included in the specification grid for writing The May 2006 CE English Examination The CIE report on the CE May 2006 English examination noted that generally the questions were not categorised or ordered according to level of difficulty, and questions were more consistently demanding than in the JL examination. The section on language use had a weighting of 50 per cent of the total marks. This was 10 per cent more than in the JL examination, but the scope of questions was broader. There was more emphasis on vocabulary, word structure and punctuation, as well as on grammar. Most of the questions required choosing the right word to complete a sentence. Some were more open-ended, such as writing questions to match given answers The report remarked that the 20 per cent of the total marks awarded for reading comprehension seemed a low percentage, given the importance of the skills being tested. The passage itself was fairly demanding and written from an adult viewpoint. The questions provided a good level of challenge for able pupils, often requiring the drawing together of information from different parts of the passage. Pupils had to think about what they have read in order to make inferences, and to back up their answers from the text. Word meanings and usage were explored in context. The level of challenge was high and reflected the more demanding aspects of the reading syllabus requiring students to justify inferences, to make evaluative responses and to understand the passage as a whole as well as in parts. These remarks concern the construct validity of the examination The CIE report noted that the weighting and the length of the composition were the same as for the JL examination and implicitly attracted the same remarks. In the CE examination, five choices of topic were offered, although only one of these gave a title to work from. The types of writing reflected the variety of types of writing included in the syllabus. Pupils were expected to plan their writing, using a blank page. The mark scheme was straightforward, giving 15 marks for content, 15 for vocabulary and expression, 10 for spelling and 20 for grammatical accuracy. There was no specific reference to organisation which is 142

143 coherent, or appropriate for different types of writing. These remarks have significance with respect to consistency in marking and consequently to the reliability of the results. The report commented that the focus on vocabulary and expression, also seen in the mark scheme for the Junior Lyceum exam, accords with the attention given to vocabulary in the 2006 syllabus The May 2006 JL Mathematics Examination The CIE evaluation focuses on the curriculum strands, the difficulty of the questions and the construction of the paper. The balance of the examination was set according to a specification grid that indicated the weighting for each strand. Analysis of the questions revealed that some questions were allocated to incorrect strands of the syllabus. Other questions were allocated to strands that did not reflect their main assessment purpose. The marks that were available for Data Handling did not reflect the importance and coverage that the topic deserved The analysis has revealed that the individual questions were well matched to the syllabus, covering the greater part of the Year 6 Syllabus. A wide range of questions styles were used within the examination. The use of quick questions that tested the quick recall of number facts at the beginning of the paper facilitated accessibility to the lower ability pupils. The questions appeared to be pitched at an appropriate level for the majority of pupils upon reaching their final year at primary school. The specification grid showed the levels of individual marks awarded in the questions. The analysis of the questions showed that the levels given to these marks were appropriate Overall, the majority of the marks in the paper were allocated to questions at the medium level, with slightly more marks for high-level questions than low level ones. This was appropriate within the context of the examination. However, the breakdown within the strands was not always matched to this overall allocation. Within the Number strand, there were approximately equal numbers of questions at each of the levels, as would have been expected. The Measures questions had an emphasis at the medium level, with fewer low level questions than high level ones. Whilst not achieving the balance shown in Number, this allocation was appropriate when the questions were compared to the syllabus. Both Shape and Data questions were disproportionately biased towards the lower end of the assessment, with no higher level Data questions. The lack of higher-level Data questions was linked to the balance of the questions across the strands. The 143

144 syllabus for Data Handling showed that opportunities existed for setting more difficult questions The questions were generally clearly laid out although the layout did not make clear the importance of showing working. The mark scheme allocated marks for choosing correct methods and gave partial credit to pupils who are able to show some understanding of the question. Whilst space was given to the pupils in which to work out their answer, no mention was made in the paper to the use of space for this purpose, nor were the number of marks allocated to each question made explicit. Both of these would have encouraged the use of space for showing working in spite of the fact that the opening rubric had indicated that the initial 10 questions carried 4 marks each, the final 10 questions 6 marks each The May 2006 CE Mathematics Examination The CIE evaluation focuses once again on the curriculum strands, the difficulty of the questions and the construction of the paper. The analysis has shown that questions were not categorised or ordered according to level of difficulty. The level of the questions on some parts of the syllabus was slightly more demanding than those in the JL examination. The summary statistics confirmed that 26 per cent of the questions were set at a low level, 39 per cent at medium level and 35 per cent at a high level whereas for the Junior Lyceum examination for the same year were 27 per cent, 41 per cent and 32 per cent respectively Analysis of the syllabus shows that the CE examination covered most of the syllabus though the analysis has revealed once again that there was too little emphasis on Data Handling. Calculations in the CE paper were more complex and often involved more than one-step to solve them. A few of the questions extended pupils beyond the syllabus. A case in point was a question on speed which involved calculations which could be beyond what is expected of a Year 6 pupil The marks for each question are indicated. This helped the pupils in knowing the marks available for each calculation. The mark scheme was less generous than that of the Junior Lyceum examination. Although it did give partial credit, this was done on a basis of removing marks for each error, rather than awarding marks for attempting a question and selecting the right method. This made it harder for pupils to gain good marks on this particular examination. 144

145 4.6. AN ANALYSIS OF THE ENGLISH AND MATHEMATICS 2005 SYLLABI BY CIE Analysis of the 2005 Primary English Syllabus by CIE This evaluation considers the structure and content of Malta s new Primary English Syllabus, in relation to the National Curriculum for English in England. It then compares more precisely the expectations in English for the end of primary education in Malta with expectations at the equivalent stage in England. (Children finish primary at age 10 in Malta and 11 in England, so Year 6 in Malta is equivalent to Year 5 in England.) The evaluation provides an insight into the construct validity of the syllabus, that is, whether the syllabus addresses the teaching and learning of those aspects of language that are considered important at primary level. It also explains some of the differences as due to the fact that the Maltese syllabus is meant for learners of English as a second language unlike the English National Curriculum, which is meant for native speakers of the language. The following paragraphs have been extracted from the full report of the evaluation, which is available in Appendix The following overview of syllabus structure and content is necessary to explain the context of the evaluation. (1) The new English syllabus for Malta 2005 replaces the 1997 syllabus on which the examinations have been based. It is structured in terms of learning outcomes for each year, with notes that exemplify content to aid teachers planning. It is clearly organised in years from Year 1 to Year 6, with headings that run through all years. (2) The National Curriculum for England is not broken into years but provides a programme of study in English for each Key Stage: Key Stage 1 for Years 1 and 2, ages 5-7; Key Stage 2 for Years 3-6, ages Outcomes are separately described in terms of attainment targets for speaking and listening, reading and writing. In each of these areas, there are descriptors for attainment across Key Stages 1 and 2, from Level 1 to Level 5. These levels are not age specific, but over the years the assessment regime has crystallised age-related expectations. The national benchmark is that children should reach Level 4 in literacy by the age of 11, as shown in nationally set and marked reading and writing tests, taken by all children. Around four in five 11 year olds 145

146 succeed in reaching this level in literacy, while around a third achieve the higher level 5. (3) The Primary Literacy Framework (England), introduced in 1998, expanded on the National Curriculum for English. It detailed the content of teaching for each year and guided teaching methodology by dividing literacy teaching into word level, sentence level and text level work. The renewed Primary Literacy Framework (2006), which is being introduced to schools this year, is organised in 12 strands that run through all the years. It is less prescriptive about teaching, gives more emphasis to speaking and listening, to building word recognition in the early stages of reading, and to the inter-relatedness of speaking, reading and writing. Learning objectives show progression within each strand and are related to year groups. The Primary Literacy Framework continues to sit within the National Curriculum for English and its level descriptors Comparison of the primary English curriculum in Malta and in England shows much that is similar. The following appear to be differences of emphasis: There is greater emphasis in Malta on listening to stories and poems, and on building up vocabulary. In England there is more focus on speaking and listening in different contexts and when working in groups, in Malta on clarity and correctness in conversation. Both include role play but drama has a higher profile in England (although this is a new development). More is expected in England about the use of information and communication technology across the curriculum. Outcomes related to attitude are explicit in Malta, and implicit in England. There is a strong recent shift in England towards phonics for word recognition in the early stages of reading rather than the use of a range of strategies. Focus on word recognition reduces beyond age 7, assuming early mastery and a later emphasis on language comprehension. The reading strategies in Malta remain broad throughout (as in the original Primary Literacy Framework in England). In England, more emphasis is given to response to texts, and appreciation of writers use of language, making stronger connections between reading and writing. In Malta there is more focus on achieving grammatical knowledge and correctness. 146

147 In England there is more explicit emphasis than in Malta on non-fiction, perhaps anticipating more cross-curricular linkage Comparative expectations of achievement in English at the end of primary schooling will now be discussed. The expectation in England is that children will achieve Level 4 in English by the age of 11 when they complete primary school. Children are expected to achieve Level 3 by the age of 9. At the age of 10 (when their peers leave primary school in Malta) they would be expected to be at the boundary of Level 3 and Level 4 (i.e. 3A/4C: each level has three subdivisions for the purposes of annual assessment). Above average pupils at age 10 would be working at the high end of Level 4, in anticipation of reaching Level 5 by the age of 11. Results in reading and writing are brought together to give a level in English. However, when separated out, results at age 11 show only 67% reaching Level 4 in writing, compared with 83% in reading. There has been much work in recent years to try and close this gap which is particularly marked for boys Despite the importance it is given in the curriculum in both countries, there is little formal assessment of speaking and listening against expected outcomes. The syllabus for Malta does not indicate any differences in approach stemming from the learning of English as a second language by most pupils. It may be that this is reflected in a greater emphasis on building confidence in speaking and ensuring correctness in grammar and pronunciation. The English curriculum makes more assumptions about pupils mastery of spoken English, though requiring that pupils develop awareness of standard English and how it may differ from their own use of language In reading, there seems to be a greater emphasis in England on the development of inference in comprehension (reading between the lines), and of exploring themes requiring an overview of a text or texts. In Malta there seems a rather stronger focus on the mechanics of reading, including the building of vocabulary. Again, this may be an appropriate emphasis for English second language learners In England and in Malta, there are clear expectations about progress in writing and these are broadly similar at the age of 10. The strong linkage between speaking and listening, reading and writing in England leads to an emphasis on contexts for writing. There are also more explicit expectations in England about handwriting, emphasising fluency rather than individual style. 147

148 At the end of primary school in England, National Curriculum tests in reading and writing are taken by all children at the age of 11, and are also available for the end of each primary year, for children aged 7, 8, 9 and 10. These tests include reading comprehension, a separate spelling test, and a writing test (including a shorter and a longer writing task, in narrative or non-narrative form, reflecting the wide variety of types of writing that children are expected to have experienced). Language usage is not separated out from other questions Tests are attractively presented, in colour, in keeping with pupils' normal work in literacy, and usually have a content theme that runs through them. The questions and the mark scheme draw closely on the National Curriculum level descriptors. Thus in England most children at the age of 11, and higher attaining pupils at the age of 10 would be expected in their reading comprehension to show understanding of significant ideas, themes, events and characters, beginning to use inference and deduction. to refer to the text when explaining their views. to locate and use ideas and information. In testing writing, marks are allocated for sentence structure and punctuation, text structure and organisation, composition and effect, and also handwriting. In the writing test, most children at the age of 11, and higher attaining pupils at the age of 10 would be expected to demonstrate writing in a range of forms that is lively and thoughtful. ideas that are often sustained and developed in interesting ways and organised appropriately for the purpose of the reader. vocabulary choices that are often adventurous and words that are used for effect some use of grammatically complex sentences, extending meaning. accurate spelling, including polysyllabic words that conform to regular patterns correct use of full stops, capital letters and question marks, and some use of punctuation within the sentence. handwriting that is fluent, joined and legible 148

149 The italicised characteristics above indicate Level 4 achievement in reading and writing (the requirement for 11 year olds). Tests for 10 year olds indicate that just over half reach this level in reading by the age of 10, which is comparable with the percentage passing the Junior Lyceum entrance exam at this age. However, only a third of pupils in England reach Level 4 in writing by age Analysis of the 2005 Mathematics Syllabus by CIE The evaluation by CIE considers the structure and content of Malta s Primary Mathematics Syllabus in comparison with the curriculum in England. It then compares the expectations in Mathematics for the end of primary education in Malta with expectations at the equivalent stage (Year 5) in England The Malta syllabus is structured in terms of learning outcomes for each year, with notes that exemplify content to aid teachers planning. The English curriculum is not broken into years but provides a programme of study for each Key Stage. Outcomes are separately described in terms of attainment targets for the different strands. In each of these strands there are level descriptors that are not age specific though age-related. The national (English) benchmark is Level 4 by the age of In England, a renewed Primary Numeracy Framework is being introduced and is organised in 7 strands, though the content is largely unchanged. Learning objectives have been updated to reflect the important aspects of mathematics to be included in teaching. These objectives show the progression within each strand and are related to year groups Comparison of the primary mathematics curriculum in Malta and England shows that much is similar: Mental calculation work is largely identical. The curriculum in England would cover negative numbers through sequences and ordering. Negative numbers are not included in the Malta curriculum. The use of the calculator would be taught in Year 4 in England with a specific calculator paper in the tests taken at Year 6. The curriculum for Malta excludes calculator use. Pupils in England would cover perimeters at an earlier age than they would in Malta, with calendars covered later. 149

150 Knowledge of the angle sum of a triangle and the sum of angles at a point would be included in Year 6 in England, as is the case in Malta. The data handling curriculum in Malta does not contain any references to probability. Pupils in England would be referring to the likelihood of events happening at this stage. The curriculum in England covers Mode in Year 5, leaving Mean until Year 6. The curriculum in Malta starts with Mean in Year 6 and makes no mention of Mode The evaluation also focuses on the comparison of expectations of achievement in Mathematics at the end of primary school. The expectation in England is that children will achieve Level 4 in Mathematics when they complete primary school. Approximately 76% of pupils reach Level 4 at the age of 11+, with no large differences in results between the genders. The teaching of many of the calculations in Malta would map across to Level 5 of England s Curriculum. No formal assessment is made in Malta of mental calculation skills in addition and subtraction. The knowledge of number facts is assessed in both countries. However only in England these are assessed within calculations. The curriculum in Malta puts less emphasis on the beginning of Algebra, which is assessed through number patterns and relationships, as is the case in England. The outcomes for Measures, Shape and Space contain less detail than their equivalents within the English Curriculum. No mention is made of accuracy when using measuring instruments except in work relating to the measurement of angles to the nearest 5 degrees. In England, the National Curriculum includes very explicit requirements in the Data Handling strand. These detail the methods of collecting, representing and interpreting data that are appropriate at each level. In contrast, the syllabus in Malta merely requires children to extract and interpret data. The problem-solving outcome in the syllabus for Malta reflects a strong emphasis in the skill of solving word problems with no specific mention of strategies and recording. The emphasis in England is on pupils developing strategies and recording these In England, all children take National Curriculum tests in Mathematics at the age of 11. These tests include a mental test and two written tests. Children are allowed to use calculators in one of the written tests. The questions and the mark schemes draw closely on the National Curriculum level descriptors. 150

151 4.7 Conclusion The quantitative and qualitative evidence presented in this chapter indicates quite clearly that the JL and CE examinations are of good quality. The examinations fit the purpose for which they are set and produce fair and equitable results. This conclusion does not mean that the examinations are perfect. It means, however, that the negative effects of the current system of transition from primary to secondary can hardly be attributed to the examination papers per se. Of course, the examination papers and the syllabi can be improved and the analyses in this chapter raise a number of issues that the examination Boards need to consider in their continuing effort to produce assessments that fit the ever-changing educational context in which they operate. 151

152 5.0 The Way Forward: Towards a Developmental Approach to Primary Education 5.1 Introduction The evidence gathered and presented in Chapter 3 suggests that the 11+ examinations are a major concern. Pupils and parents feel stressed and suffer anxiety, the content of the examinable subjects dominates the curriculum in the upper classes of the primary school, teaching methods focus on the narrow skills needed to pass the examination and pupils seek further coaching by attending private lessons. Failure in the 11+ examination leads to a drop in self-esteem and a loss of motivation which are reflected in a loss of commitment and eventually in poor results during secondary education. The analysis of Chapter 2 has shown that the rationale of the selective examination is no longer valid. Further evidence from the impact and quality reviews has indicated that in spite of the many improvements in syllabi and examination papers that have been effected over the years, many students lose interest in formal education at an early stage. All these factors signal the need to re-think the whole approach to primary education. Currently, the approach falls within the preparatory tradition, which can be described as examination-centred. In this approach, pupils, parents and many teachers focus their attention on achievement in the examination, which from Year 4 (age 8-9 years) onwards determines the stream in which the pupil will be placed. At the end of Year 6, the 11+ examination determines which secondary school the pupil will attend and consequently the pupil s chances of success in secondary and further education. The evidence shows that the preparatory approach in primary education is not beneficial to a large proportion of pupils. With this approach to primary education, geared as it is towards an examination and which effectively labels about one third of the cohort as failures at the age of 11, it is difficult to see how Malta can ever aspire to reach the Lisbon Objective of 85 per cent of each age cohort of students completing post-secondary education The need for a new approach to primary education also becomes evident when one considers the educational response to the cultural, social and economic challenges faced by our country at the beginning of the 21 st Century and the guiding principles advocated by the National Minimum Curriculum (2000). In particular, the NMC stresses that in a democratic society, The educational process should cultivate within students a sense of social justice and solidarity and that The educational community should actively oppose all forms of discrimination by promoting the corresponding attitudes and the readiness to act (p.25). The NMC also emphasises 152

153 the concept of the holistic development of the person, which places the student at the centre of the system and the idea that the curriculum should be at the service of the students and not the other way round. These considerations lead to the idea that An effective educational system provides society with persons who are aware of their own potential and are keen to develop further. Such a system seeks to eliminate the waste [of human potential] throughout the entire educational process. (NMC, 2000, p.26). 5.2 The Guiding Principles All the fifteen principles identified by the NMC are of utmost importance in consideration of any change in approach to primary education. However, the following principles are of direct relevance to the present Review: Quality education for all, which holds that the ultimate aim of a national curriculum is to develop an educational ethos that stimulates the development of the students potential without undermining the principles of solidarity and co-operation. Respect for diversity by recognizing that each school is endowed with a vast repertoire of skills, experiences and needs and that this diversity enables and requires a pedagogy based on respect for and the celebration of diversity. In the local context, diversity exists mainly because students come from different social contexts, learn differently, and go through different phases of development at different rates. Systems should be in place to provide support to students who are denied a support system outside school and other students who need more time and support for their personal development. An overarching aim of education is to promote social cohesion though the Review Committee understands that social differentiation cannot be eliminated. An inclusive education, which is based on a commitment, on the part of the learning community, to fully acknowledge individual difference and to professing as well as implementing inclusive policies. This principle recognizes the full range of the students educational interests, potential and needs. Besides these principles, the Review Committee would like to add other principles. Fair and equitable assessment, the purpose of which is to indicate accurately and fairly individual pupils progress and achievement in their educational process. The modes of assessment to be adopted should, 153

154 without discrimination or bias, allow all students to demonstrate what they know, understand and can do in the various areas of the curriculum. Seeking the common good by advocating changes in the process of transition between primary and secondary education that are beneficial to all pupils whatever their intellectual, social, cultural and economic background, and detrimental to none. The changes should strike a balance by assuring that standards are maintained, that all pupils realise their intellectual, affective and physical potential and that primary education provides everyone with a good foundation of knowledge and skills and a positive attitude to learning. Whatever their potential, pupils are the major resource of the Maltese economy and every effort must be made to maximize the development of each and everyone s potential. It is equally important to avoid infecting them with a sense of failure, disaffection and alienation. Evolution not revolution is the recommended way forward as time is needed for all stakeholders to understand, become familiar with and adjust to the several attitudinal and practical changes required by the shift from the current educational paradigm to a new paradigm that is more consistent with the guiding principles. However, it is important that the gradual changes should always benefit pupils passing through the system during the process of paradigm shift. 5.3 The Developmental Approach in Education The guiding principles and the concerns that have been raised by educators about the current system point towards the developmental approach as the way forward for primary education. The developmental approach is definitely childcentred in contrast with the preparatory approach, which is exam-centred. It considers the individual as an active being, who is entitled to control over his or her destiny, and consequently sees education as a process by which the degree of such control available to each individual can be maximized. Its central concern is with individual empowerment. (Kelly, 1999, p.84). From this perspective, education is seen as the process of development of the child s ability to act autonomously. This means that education should be concerned with the development of those capacities which enable the child to make the personal choices, decisions, and judgments that autonomous living implies, and will give as much genuine control over one s destiny as possible. In effect, however, autonomy is not simply freedom from constraints. It is also essentially moral since the decisions and judgments imply the making of autonomous moral choices which affect the individual s interactions with society. Education is then more than the development of the ability to make autonomous 154

155 decisions; it is also moral development as well as social development. Indeed the developmental approach derives from the idea that education cannot be viewed solely in terms of cognitive or intellectual growth but also in terms of the whole spectrum of the affective development of the individual (Kelly, 1999, p.87). This view is reinforced by Blyth (1984) who suggests that the primary school curriculum should enable development and experience to take place beneficially. It must also enable each individual to become a person with an emerging set of values and ideals. It leads to the enablement of choice and to the acceptance of limitations (p.50) This theoretical perspective has wide-ranging practical implications for the primary curriculum concerning subject content and process skills, teaching methods, assessment and evaluation. The following sections will present recommendations on the various aspects of the primary curriculum starting with assessment, which is the focus of the Review. The recommendations keep in mind that change should be evolutionary not revolutionary and therefore the proposals suggest gradual changes over a number of years. Some of the first changes may seem inconsistent with a developmental approach but in the long run the whole series of changes should be fully coherent with the principles of the developmental paradigm The new approach also has implications for the administration and management of primary and secondary schools. In this regard, it must be emphasised that the practice of rigid streaming goes against the basic principles underlying the proposed recommendations and should not be used. The negative effects of streaming and segregation of pupils at any early age have been highlighted by the report of the 2003 Programme of International Student Assessment (PISA). Its detailed analysis of how the education system affects the achievement of students in 41 countries showed that educational systems with higher degrees of segregation seem to perform at lower levels and show a wider disparity in performance between schools than systems with greater inclusion. Similarly, the age at which segregation occurs seems to be another factor contributing to a country s overall performance such that countries in which segregation occurs at a later stage perform better than countries in which students are segregated at a young age. This empirical evidence provides strong support for the elimination of streaming especially in primary and early secondary education. Indeed European countries such as England and Germany are actively discussing ways of eliminating segregation and ensuring that school admissions offer a fair choice for parents and pupils (Tough and Brooks, 2007). 155

156 5.3.4 Recommendation 1: Given the theoretical support and empirical evidence, we recommend that the practice of streaming and selection in the primary and lower secondary school years should be discontinued It is recommended that streaming should be discontinued as from with the Year 4 cohort. It is from this cohort onwards that streaming will not apply in Years 5 and 6. The cohort that is currently in Year 5 will be the last cohort to be streamed in Years 5 and 6. The discontinuation of streaming should not present organizational difficulties in small schools and therefore should be adopted immediately in schools with a three-class entry or less. However, in primary schools with a larger entry, an interim measure may be considered in which pupils may be grouped in two bands 17 such that each class in each band has a mix of abilities. Alternatively, schools may decide to have mixed ability classes throughout but for some subjects (e.g. English, Maltese and Mathematics), they may group pupils in sets. 18 Whichever system of grouping is adopted, it is important to have an individual support system in place especially for pupils who find great difficulties to reach the required levels of achievement in basic literacy and numeracy. This support should be school-based but there may be students who are linked to afterschool support of the type currently offered by the Ħilti, Id f Id and Nwar programmes. Appendix 11 presents a brief description of each of these programmes. The support system may also be extended to the high achievers who need to be challenged further. The discontinuation of selection for secondary education will be discussed later. 5.4 The Assessment System The design of a system of assessment that avoids the difficulties of the current system without losing its benefits and the need to be consistent with the developmental approach to primary education presents a number of dilemmas. The new system needs to: reduce anxiety and stress but should not remove the motivation and challenge of assessment; follow the principle of inclusion but maintain standards; 17 Banding refers to the grouping of students into two bands according to their achievement in a number of subjects. For example, a school with an entry of 180 pupils may divide the entry into two bands by achievement. In each band, classes are grouped by mixed ability. 18 Setting refers to the grouping of students according to their achievement in a particular subject. 156

157 have a beneficial effect on teaching and learning but ensure full coverage of subject content; provide feedback to individual pupils and parents but avoid undue stress and unhealthy competition; give feedback to schools but avoid league tables and unfair comparisons among schools; serve for accountability purposes without suppressing autonomy in curricular and pedagogical matters The following action plan attempts to resolve these dilemmas by: reducing the negative consequences of the JL and CE examinations and the end of year examinations in Years 4 and 5 in state primary schools. This is achieved by reducing the strict segregation of pupils into winners and losers as they transfer from primary to secondary education and the rigid streaming as they progress from one year to another in the upper classes of the primary school; keeping the externally set examinations in the English, Maltese, Mathematics, Religion and Social Studies in State schools at the end of Years 4 and 5 to report on individual pupils achievement. extending the externally set examination in English, Maltese and Mathematics at the end of Year 6 in State schools to Church and Independent schools on a voluntary basis, in order to be able to report on individual pupils achievement in a standardised manner. promoting school-based assessment with an emphasis on assessment for learning in a range of curriculum areas including in the oral-aural aspect of English and Maltese; reporting individual students assessments as levels of achievement; strengthening internal and external moderation to ensure fair and consistent use of assessment tools within schools, colleges and other school networks; introducing external monitoring in all curriculum areas using samples of schools and pupils over a cycle of five years. 5.5 Recommendations - Primary School Recommendation 2: The purpose of all examinations held in State primary schools, including the examination at the end of Year 6, should be to report on the achievement of each learner and not to stream or select pupils. 157

158 Recommendation 3: The annual assessment in Year 4 and Year 5 will consist of written examinations in English, Maltese, Mathematics, Religion and Social Studies and school-based assessment in other areas of the curriculum including oral-aural assessment in English and Maltese. Recommendation 4: The end of Primary school examination will be offered to all pupils in State, Church and Independent schools. Written examinations will be held in English, Maltese and Mathematics with a school-based oral-aural component in English and Maltese. There will also be school-based assessment in Religion, Social Studies and other areas of the curriculum. Recommendation 5: There will be internal and external moderation to ensure that schools are fair and consistent in the use of assessment tools. Recommendation 6: External monitoring will be introduced to evaluate childrens learning in the different areas of the curriculum in the upper years of primary (Years 4, 5 and 6) and the lower years of secondary education. (Forms 1 and 2). Recommendation7: Assessments will be recorded as levels of achievement. 5.6 An Action Plan for Assessment in State Primary Schools This section presents the plan of action for State schools in a chronological order. It specifies stages in the development of the assessment system for each of Years 4, 5 and 6 of the primary school. External monitoring is discussed in a separate section. The next section looks into the practical implications for the curriculum, pedagogy and the support system. Another section shows how Church and Independent schools are affected by the recommendations. The plan spans six years after which the new assessment system would be in place, classes would be organized by mixed-ability groups, and teaching and learning would be clearly learner-centred and in line with the developmental approach rather than the preparatory approach to primary education (refer to Figure 5.1). The plan is incremental to allow for necessary changes and training to take place. 158

159 159

160 160

161 5.6.1 The First Year of the Reform Year 6 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in February as is current practice. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment in Science will continue. The Junior Lyceum exam will take place in May. Written examination in English, Maltese, Mathematics, Religion and Social Studies will be set and marked centrally. Selection still applies for these pupils in their transition to Form 1. Year 5 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June as is current practice. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment in Science will continue. Results will be used for streaming purposes for these pupils as is current practice. Year 4 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June as is current practice. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment of Science will continue. Colleges may provide an alternative assessment scheme for pupils with particular needs. The results of the annual examinations will not be used for streaming purposes. Years 1, 2 and 3 The current system of teacher assessment of pupils in Years 1, 2 and 3 will be retained. However, early screening is recommended to identify pupils encountering serious difficulties, especially in literacy and basic numeracy and to give them appropriate support The Second Year of the Reform Year 6 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in February as is current practice. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment in Science will continue. The Junior Lyceum exam will take place in May. Written examination in English, Maltese, Mathematics, Religion and Social Studies will be set and marked centrally. Selection still applies for these pupils in their transition to Form 1. This will be the last year that the Junior Lyceum examination will take place for selection purposes. 161

162 Year 5 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment in Science will continue. School-based assessment in other areas of the curriculum may be introduced. Colleges may provide an alternative assessment scheme for pupils with particular needs The results of the annual examinations will not be used for streaming purposes. Year 4 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June. Papers will be set centrally and marked by class teachers. The listening comprehension examination in English and Maltese and the school-based assessment of Science will continue. School-based assessment in other areas of the curriculum may be introduced. Colleges may provide an alternative assessment scheme for pupils with particular needs. The results of the annual examinations will not be used for streaming purposes. Years 1, 2 and 3 The current system of teacher assessment of pupils in Years 1, 2 and 3 will be retained. However, early screening is recommended to identify pupils encountering serious difficulties, especially in literacy and basic numeracy and to give them appropriate support The Third Year of the Reform Year 6 The end of Primary school examination will be offered to all pupils in State, Church and Independent schools. In effect, the end of Year 6 examinations in English, Maltese and Mathematics become a national benchmark to determine the achievement of all pupils in State schools, and in Church and Independent schools that decide to participate. The written papers will be set and marked centrally and the oral-aural assessment in the languages will be assessed by the schools. The written papers will be designed to allow all pupils to demonstrate their level of achievement. There will also be school-based assessment in Religion, Social Studies and other areas of the curriculum. All pupils can proceed to secondary education. Colleges may organize secondary classes as mixed ability classes and for certain subjects introduce setting or organise them in bands such that where there are four or five classes, these may be divided into two bands and where there are six or more classes these may be divided into three bands. Pupils whose achievement in the core subjects is low are to be given special support. This support should be school-based but there may be students who 162

163 are linked to after-school support of the type currently offered by the Ħilti, Id f Id and Nwar programmes. Appendix 11 presents a brief description of each of these programmes. The support system may also be extended to the high achievers who need to be challenged further. Year 5 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June. Papers will be set centrally and marked by class teachers. School-based assessment in other areas of the curriculum is in place including oral-aural skills in English and Maltese. Colleges may provide an alternative assessment scheme for pupils with particular needs. The results of the annual examinations will not be used for streaming purposes. Year 4 Annual examinations in English, Maltese, Mathematics, Religion and Social Studies will be held in June. Papers will be set centrally and marked by class teachers. School-based assessment in other areas of the curriculum is in place including oral-aural skills in English and Maltese. Colleges may provide an alternative assessment scheme for pupils with particular needs. The results of the annual examinations will not be used for streaming purposes. External monitoring is introduced. Years 1, 2 and 3 The system of teacher assessment of pupils in Years 1, 2 and 3 is strengthened. Early screening is recommended to identify pupils encountering serious difficulties, especially in literacy and basic numeracy and to give them appropriate support The Fourth, Fifth and Sixth Years of the Reform In the fourth year of the reform, external monitoring will start from Year 4 with a few subjects and representative samples of pupils and schools. All subjects and skills of the primary curriculum will be monitored by rotation over a number of years. This external monitoring serves to establish benchmarks for the different curriculum areas. In the fifth year of the reform, external monitoring will be extended to Year 5 in a few curriculum areas with representative samples of pupils and schools. Similarly, in the sixth year of the reform, external monitoring wil be further extended to Year 6. The same pattern of assessment including examinations, school-based assessment, internal and external moderation and external monitoring will apply in subsequent years. All subjects and skills that are assessed by the schools in Years 4, 5 and 6 will be externally monitored by rotation over a cycle of five years. 163

164 External monitoring will be carried out on a representative sample of pupils and schools and in a sample of subjects, which may vary from year to year. It may consist of written or oral tests, performance tasks, practical work and other forms of assessment, which are appropriate for the subject or skills that are being assessed. Monitoring will be coordinated centrally and the results will be used to obtain information about national levels of achievement. A follow-up activity of the results of the monitoring programme with teachers could serve as an opportunity for their professional development. The main aim of the activity would be to obtain a better understanding of results of the monitoring exercise, the needs of pupils and teachers, and the action needed to improve achievement. A more detailed explanation of the external monitoring programme is given in a separate section below Starting from the fourth year, level descriptors are to be used to report on the individual learners levels of achievement (see Appendix 12) in Year 4. This reporting system is to be extended to Year 5 in the fifth year and to Year 6 in the sixth year. Initially, both marks and levels are to be reported. Decisions on levels of achievement take into consideration continuous assessment and performance in examinations. Internal and external moderation is to be carried out to ensure that assessment tools and level descriptors are used fairly and consistently within and across schools Although the brief of the Review Committee was to focus on transition from Primary to Secondary education with a main focus on assessment in the upper years of Primary schools, in the following section it extends its recommendations to secondary education as well to ensure smoothness, consistency and continuity though it does not tie them to a particular timeframe. 5.7 An Action Plan for Assessment in the State Secondary Schools Recommendation 8: The system of assessment including school based assessment, internal and external moderation and external monitoring will be extended to at least the first two years of secondary education Written examinations and other modes of assessment such as orals, practicals, project work and coursework should allow all students to demonstrate their level of achievement in the various subjects of the curriculum. This means that the examination papers should allow for the assessment of multiple levels of 164

165 achievement and this may be achieved by setting papers at different levels. The papers will be set centrally, though the colleges or network of colleges may set their own examinations. Since the setting of multi-level papers requires particular expertise, the centrally set papers may be shared by all schools in the State, Church and Independent sectors Level descriptors are to be used to determine the students levels of achievement (see Appendix 12) in secondary schools as well. Internal and external moderation ensures that assessment tools and level descriptors are used fairly and consistently A system of external monitoring of the core areas of the curriculum in Forms 1 and 2 is to be developed on similar lines to the system recommended for primary education, which will be discussed later. 5.8 Curriculum, Pedagogy and Support Systems Introduction The main aim of this review is to suggest how the transition from primary to secondary education can be smoother and more beneficial to all pupils, however, the Review Committee realises that the success or failure of its recommendations depends to a large extent on the pupils learning experiences before and after the transition. This observation leads to the need to reconsider the curriculum as well in the light of the recommendation to adopt a developmental approach, grouping of pupils in mixed ability classes and a new system of assessment. In this respect, besides assessment, the curriculum is seen as comprising a statement of aims and objectives, subject content, and the teaching and learning strategies and activities. This section discusses the changes that need to occur in these areas of the primary and secondary curricula in order to complement the changes in the assessment system. It also discusses the human and physical resources required to support the changes The current system of primary and secondary education is producing results since a fair proportion of pupils continue with their education after compulsory schooling and achieve creditable results. Yet, as highlighted in previous sections, an unacceptably high proportion of pupils are disaffected with formal education at an early age and resist attempts to improve their knowledge and skills during schooling and sometimes even after they have left school when they are 165

166 offered courses to improve their chances of employment. Changes in the assessment system are not enough to counter this trend. These changes must be accompanied by changes in the curriculum and supported by appropriate resources. Only the synergy between syllabi, pedagogy and assessment can bring about a change whereby a much higher proportion of pupils enjoy learning, strive to succeed and achieve results that encourage them to become lifelong learners. This section addresses the changes in the curriculum at the primary and the secondary level and the necessary support system that should accompany the changes in assessment. Specific recommendations are presented at the end of this section after a discussion of these issues Primary Education A common criticism of the traditional primary school curriculum is that it places too much emphasis on academic achievement and cognitive skills, which can be assessed by written tests, and pays little attention to the development of positive attitudes towards learning and social skills, which are more difficult to assess objectively. In other words, it values what can be measured instead of measuring what is valued. Another criticism is that the syllabi are overloaded, teachers have to rush through subject content leaving behind the pupils who need more time to understand and others who get lost because they just miss one point and cannot make the connection between the new knowledge and what they have learned already Faced with these criticisms and the challenge of suggesting alternatives, the Review Committee turned to the document Tomorrow s Schools (1995) of the Consultative Committee on Education, which asked the following question: What are considered to be the basic requirements of a quality education one that is meaningful, worthwhile, responsive to individuals and social needs and does each and every student, without fail get those requirements, regulated as these are by the principle of entitlement? (page 8) Quality is not an easy concept to qualify. According to the Education For All: Global Monitoring Report The Quality Imperative (EFA:GMR): Two principles characterize most attempts to define quality in education: the first identifies learners cognitive development as the major explicit objective of all education systems. The second emphasizes education s role in promoting values and attitudes of responsible citizenship and in nurturing creative and emotional development (p.17). 166

167 This report goes on to suggest that quality determines how much and how well children learn and the extent to which their education translates into a range of personal, social and developmental benefits. Goal 6 of the Dakar Framework for Action (2000) in education reported in EFA: GMR (p.28) emphasizes the need of improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills. This suggests that the curriculum is not enough; it must be accompanied by a stimulating pedagogy. Indeed, the EFA:GMR states that the teaching and learning process brings the curriculum to life. This process determines what happens in the classroom and the quality of the learning outcomes. According to the GMR, good practice requires the attention to six policy issues with direct impact on teaching and learning: 1. relevant aims policy dialogue must arrive at a relevant balanced set of aims describing what learners should learn and why; the development of cognitive, creative and social skills and values; respect for human rights, the environment, peace and tolerance and cultural diversity. These put citizenship, democracy and human rights at the fore. 2. subject balance how subjects are defined, how many are taught and the time allocated to each. Central to the core curriculum is the teaching and learning of reading. 3. good use of time Positive correlations are noted between instruction time and student achievement at both primary and secondary levels. Between 850 and 1000 effective hours (not necessarily official hours) of schooling per year is broadly agreed as a benchmark pedagogic approaches for better learning teacher dominated pedagogy placing students in a passive role is undesirable. Child-centred active pedagogy, co-operative learning and the development of critical thinking and problem-solving skills need to be developed. Structured teaching is recommended. 5. language policy Language of instruction is a policy choice affecting curriculum, content and pedagogy. A balance needs to be struck between 19 In Malta the number of contact hours per year from to are 920.5, 931.5, 915.0, 920.5, and

168 enabling people to use local languages in learning and ensuring that they have access to global languages. 6. learning from assessment Regular, reliable, timely assessment is a key to improving learning achievement. The goals are to give learners feedback and improve learning and teaching practices. Formative assessment is needed as a complement to formal examinations. The Global Monitoring Report also identifies support as essential for quality education. The Report identifies the key concepts relating to quality education as follows: understanding the diverse needs of learners and supporting reforms that focus on better teaching and learning outcomes relevant content, enough learning time, structured teaching in child-centred classroom and assessment. It is critical to support teachers and ensure that schools are well-managed. Strong political will and sufficient resources must back up reforms. Given the Maltese educational context, how can quality education be provided? A detailed answer to this question is beyond the brief of this Review Committee, however, the Committee would like to make the following observations in an attempt to give direction in the search for quality primary education. Curriculum The current state primary school curriculum comprises a commendable mix of areas of knowledge that offer opportunities for the holistic development of individual pupils. The curriculum areas include English, Maltese, Mathematics, Religion, Science & Technology, Social Studies as subjects that are assessed in a formal manner. The other areas include Physical Education, Expressive Arts (Drama, Art & Design, Music, Movement) and Personal and Social Development, which are not formally assessed. All these areas are important as they cover the multiple intelligences identified by leading educational psychologists (Gardner, 1983) The current curriculum also recognizes Information and Computer Technology (ICT) skills as tools which can be used to access knowledge in all the other areas of the curriculum. These skills need to be emphasized even more given their increasing importance in the acquisition and communication of knowledge and given that Malta aims to become an important regional hub for ICT. For this to happen, the primary curriculum should enhance the existing strategies for incorporating ICT skills in the specific areas of the curriculum. The availability of 168

169 necessary resources also needs to be considered because while the internet could be used in various areas such as English, Mathematics and some aspects of Social Studies, other areas may require the development of appropriate software for the local curriculum. Other needs include continuing professional development of teachers in this area and the development of forms of assessment to monitor the progress of pupils in the acquisition of ICT skills. Syllabi The primary school syllabi, which were published in 2005 are very useful documents to promote a developmental approach. The syllabi specify the learning outcomes in each subject with a clear learner-centred focus. The evaluation of the syllabi in Chapter 4 of this review indicates that in general the new syllabi for state schools are commendable as they present the learning outcomes quite clearly. Some of the comments of the evaluations suggest improvements. These need to be discussed by the competent subject experts keeping in mind the developmental approach advocated by this review. Curriculum specialists also need to judge whether all the learning outcomes can be achieved within the given timeframe. This professional judgment is needed to allow time for meaningful learning to take place and to avoid undue pressure on teachers and pupils and the parents criticism of an overloaded curriculum. Pedagogy Given that formal examinations will have a less determining role on the pupils progress and that teachers will have less pressure to prepare for the test, the Review Committee recommends that teaching should place greater emphasis on important knowledge and affective skills which cannot be easily assessed by written tests. The reforms provide space for teaching to emphasize application of knowledge, analysis, synthesis, and evaluation besides the usual recall and simple understanding. They also open up opportunities for activities and learning experiences where pupils develop personal and social skills and attitudes as they interact by sharing, discussing, acting responsibly, using different forms of communication, and accepting diversity. With the freedom from the constraints of high-stakes examinations, teachers can explore the use of cross-curricular, problemsolving and thematic activities, which would allow pupils to experience the association between different areas of knowledge. In general, the reforms should gradually lead to more interesting, enjoyable and meaningful teaching and learning experiences. 169

170 Support The extension of mixed ability grouping to Years 5 and 6 presents new challenges to teachers, especially those in large entry primary schools who have been used to teaching rigidly streamed classes. These challenges need to be recognized and addressed by offering teachers effective professional training on mixed ability teaching. Good resources for training are found in the recent publications of the project DTMp- Differentiated Teaching Module primary, which are meant for trainee teachers and which contain contributions from educators from seven European countries (Bartolo ed., 2007) 20. Teachers may also need supplementary curriculum materials and practical advice on how to use them. Whilst further training and appropriate materials may help teachers to cope with the new challenges, there may be pupils in each class who need special support. This can be provided by teacher assistants and by continuation of after school initiatives such as the Hilti, the Nwar and the Id f Id programmes. The roles of the learning support assistant and the facilitator should be clearly differentiated. In particular, the need to have more than one facilitator in a class should be avoided. The Review Committee affirms that learning support assistants are essential, however, this resource should be deployed more equitably and effectively (Spiteri et al., 2005). Some schools seem to have a larger share of pupils with social problems usually because of family circumstances. While teachers and Heads of school are usually able to give these pupils the required attention, there may be cases where further professional help is required. The Review Committee recommends the employment of trained care workers, including psychologists, social work professionals, counsellors and occupational therapists, who will be able to look after the needs of students at risk. The role of these workers is to guide and counsel students and help them to overcome the obstacles that may interfere with their development and education. Leadership College principals and heads of school have a crucial role to play in the implementation of the reforms. Without their driving force, it is difficult to envisage how the reforms can be implemented successfully. They are the professionals who face the many practical day-to-day problems that are bound to arise in any educational change of this magnitude. They are the ones who need support to find solutions for these problems, therefore they need to have an appropriate allocation of Assistant Heads and clerical assistance. They also must have a fair degree of 20 The publications are Responding to Student Diversity: Tutor s Manual and Responding to Student Diversity: Teacher s Handbook edited by Paul Bartolo and published in June

171 autonomy provided that they adhere to the general principles of the reform. Their professional judgments on what is best for the pupils in their school should be esteemed It is imperative that a more distributed form of leadership is encouraged to involve even teachers within and across networks and other stakeholders. Dispersed, delegated and democratic forms of leadership would imply greater empowerment where and when it matters (Bezzina, 2003; 2005). A strategic approach is recommended that would see the two newly-established Directorates establishing greater links with schools and Colleges in particular. This would allow for a stronger link between support, development, internal review and external evaluation. It would also nurture a culture of autonomy based on greater responsibilities at the school/network level. This would mean that networks would become slowly more responsible and accountable for matters dealing with school life Secondary Education Curriculum At the secondary level, the spread of abilities and interests among pupils becomes wider. Therefore, while it is desirable to offer the same curriculum to all pupils, it is also important to offer an appropriate curriculum, that is, one which takes into account the diversity of the pupils needs. This may be achieved by offering a group of core subjects to ensure a basic entitlement for all and a choice of other subjects that allows adaptation to the pupils needs. This recommendation is different from the core of twelve subjects for all pupils as stipulated by the National Minimum Curriculum (2000). The Review Committee would like to present the following curriculum scheme as an example of a scheme with an in-built flexibility that allows College principals and Heads of school to adapt it to the needs of their pupils (Tables 5.1a and 5.1b). The scheme assumes a timetable of 40 lessons of 40 minutes per week; the number of lessons per week for each area is shown in brackets. It is understood that individual schools may modify the scheme, adjust the number of lessons and the options on offer according to the students needs and the available resources. 171

172 Table 5.1a Curriculum Scheme for Secondary Education Forms 1 and 2 Forms 1 and 2 Core Subjects (25 lessons) Maltese (3) English (5) Mathematics (5) Religion (2) Integrated Science (4) Personal, Social & Health Education (1) Physical Education (2) History/ Geography/ Social Studies (2) ICT (1) Options (15 lessons) Pupils take five options of 3 lessons per week from the following list: International language/s (3) Home Economics (3) Design and Technology (3) ICT (3) Extra Maltese (3) Extra English (3) Extra Mathematics (3) Extra Maltese/English/Mathematics (3) Music (3) Art (3) Drama (3) Music/Drama/Art (3) Physical Education (3) Table 5.1b Curriculum Scheme for Secondary Education Forms 3 to 5 Forms 3, 4 and 5 Core Subjects (24 lessons) Maltese (3) English (5) Mathematics (5) Religion (2) One Science (Biology, Chemistry, Physics, Coordinated Science) (4) Environmental Studies (1) Personal, Social & Health Education (1) Physical Education (2) ICT (1) Options (16 lessons) Pupils take four options of 4 lessons per week. The options could be all academic or all vocational or a mix of both. All subjects offered at SEC level apart from English, Maltese, Mathematics, Religion and the one science subject offered as core, will continue to be offered as options with four lessons each. Technical and Vocational subjects (4) including: Performing Arts (4) Art/Design/Crafts (4) Social Care and Community Studies (4) Beauty, Hair and Health Studies (4) 172

173 Agricultural Studies (4) Media Studies (4) Tourism Studies (4) Mechanical/ Electrical/Engineering Studies (4) Home Maintenance (4) The curriculum for Forms 1 and 2 reflects current practice with some modification to allow more flexibility. The curriculum for Forms 3, 4 and 5 offers the opportunity to all pupils to obtain certification at SEC level. Some may prefer to aim at certification in basic competency level in the subjects identified by the MATSEC Review (2005), that is, in Maltese, English, Mathematics, one Science, ICT, to which Religious Knowledge may be added In Forms 3, 4 and 5, some students may prefer to follow a technical or vocational path and for them the curriculum scheme proposes the introduction of a number of options. Others may opt for a mix of academic and vocational subjects. This innovation entails the development of syllabi, the training of teachers, the allocation of resources, and the development of a system of assessment and certification complementary to the SEC system of examinations. Foreign examination boards already offer similar areas of study at GCSE level. Their publications, including detailed specifications and assessment schemes, can be studied and adapted to the local context. The following lists of topics in the proposed areas are presented in order to illustrate the potential of the recommended vocational and technical subjects. Performing Arts: Drama, dance, music, singing, movement, stage management, cultural background and appreciation. Arts/Design/Craft: Art techniques, wood and stone carving, Maltese crafts such as lace making, embroidery, filigree, costume jewelry, pottery; stained glass, parchment production, printing; sketching and graphical representation, basic design and technology. Social Care and Community Studies: Child care, care of the elderly, community services, health and safety in the home and at work, first aid. 173

174 Media Studies: Video and still photography, audio/visual effects, film and TV, journalism and print media, advertising, marketing and consumer education. Beauty, Hair and Health Studies: Nutrition, exercise, basic human biology; beauty therapy, cosmetics, skin care and hair dressing. Agricultural Studies: Soils and soil fertility, water catchment and irrigation, crop production and rotation, prevention and treatment of crop diseases; animal farms, animal husbandry, energy saving devices. Engineering Studies (Mechanical/Electrical): Studies conducted in one sector of engineering could include: electrical and electronic devices, computer devices, chemicals and pharmaceuticals, automobile repair and servicing, production of food and drink. Studies include design specifications, tools and machinery, manufacturing processes, quality control, maintenance, health and safety, customer service. Home Maintenance: basic woodwork, painting and plastering, tiling, plumbing, masonry and construction, electrical installation, servicing electrical appliances, health and safety. Pedagogy The developmental approach to teaching and learning emphasizes the development of competences and skills more than the preparatory approach. The teaching methods used in the various areas of the secondary curriculum should aim at developing the skills and competences of individual pupils to enable them to become autonomous and effective citizens in a democratic society. The pedagogy should therefore aim at developing: (a) cognitive skills such as observing, sorting, grouping, finding patterns and many other similar skills relating to the acquisition of knowledge; (b) communication skills such as explaining to others, expressing an opinion, giving instructions; (c) personal skills that relate to knowing oneself, the ability to face change and to decide, developing confidence and self-esteem; (d) affective skills such as empathizing, appreciating others, respecting other peoples values, accepting diversity; (e) social skills such as learning to work in a team, debating, listening to others, understanding norms, and recognizing rights and duties; 174

175 (f) entrepreneurial skills such as creative thinking, planning and researching, financial planning, making a product, time management, marketing the product These are only a few of the skills that can be developed during secondary education. Appendix 13 presents a fuller list of skills and an explanation of the measures needed to improve students learning in secondary schools including ideas about cross-curricular teaching and learning and the important role of formative assessment. Support At the secondary school level, it is quite likely that the number of students who need support will increase. At this stage, the range of abilities among students becomes wider and their interests become more diverse. Additionally, the personal and social problems of some students may become more complex as they enter the phase of adolescence. Therefore, it is essential that secondary schools, like the primary schools, are given the support of learning support assistants, and specialists in psychological and pastoral care. Although this may entail added costs, it is far better to invest financially in preventive measures than to invest greater sums of money on remedial action later, which may not be so effective. Leadership College Principals, Heads of school, Assistant Heads, Heads of Department and all teachers with special responsibilities all have a determining role in the success of any initiative. Their style of management and the school ethos they build contribute inestimably to the development of positive attitudes and values. However, the school administration needs adequate support to look after purely clerical and maintenance tasks so that they have more time for their leadership role in educational matters. They are the professionals who best know the needs of the pupils under their care and whose judgment and autonomy should be respected. The need to establish schools as learning communities would help to develop the appropriate culture policies and structures that are needed to create leadership opportunities when and where needed The central authorities must take a leading role in curriculum development and innovation. The National Curriculum Council and the department responsible for curriculum represent the central authorities and carry responsibility for policy making in the field of curriculum. Their leadership is essential for 175

176 establishing an appropriate policy and driving the required curriculum development and innovation. The subject specialists are responsible for implementing curriculum innovation and development in their area of expertise, for advising teachers on new methods of teaching and assessment, and for keeping abreast with local and international developments in the teaching of their subject specialization. The Faculty of Education also has a role in providing advice besides offering the essential pre-service teacher education and continuing professional education to practising teachers. The Review Committee would like to reiterate that the success of the reform in assessment could be jeopardized if it is not accompanied by a change in curriculum and this needs the support of all concerned Recommendations The observations in this section lead to the following recommendations: Recommendation 9: The commendable mix of areas of study at Primary level of the current curriculum offers opportunities for holistic development and pupils can develop necessary skills, attitudes, knowledge and understanding. Teaching should put greater emphasis on important knowledge, skills and attitudes that are not necessarily assessed by written examinations. The primary curriculum should ensure daily physical activity and enhance the existing strategies for incorporating ICT skills in the specific areas of the curriculum. Recommendation 10: The current primary school syllabi specify the learning outcomes in each subject area and are therefore in line with a developmental approach to education. However, subject specialists working in conjunction with schools need to determine whether all the learning outcomes can be achieved within the given time-frame and adjust the syllabi where necessary. Recommendation 11: Mixed ability classes bring additional challenges to teachers who need to feel empowered to deal with such classes. For this reason, continuous professional training needs to focus on preparing them for diversity and multi-level teaching. Recommendation 121: Other stakeholders apart from school staff need to be prepared for differentiated teaching, its implications, benefits and challenges. Information seminars for parents and pupils need to be organised at school or college level. 176

177 Recommendation 13: Mixed ability grouping brings with it the need for a robust support system. Such a system needs the employment of full-time professional care workers in the schools. These could be college-based or provide services for a number of schools. Classroom support in terms of learning support assistance is also recommended in classes with pupils with special requirements. Recommendation 14: Early screening of pupils in Primary School is considered necessary to identify those pupils in need of additional support in literacy and numeracy and appropriate support is to be given. Recommendation 15: At secondary level, an evaluation of the current curriculum and the preparation for changes is needed urgently. Students need to continue to develop a range of functional skills and attitudes as well as knowledge and understanding in order to live an active and independent life within a community. At Form 1 and 2, all students are to have common core subjects and a range of optional subjects that they choose according to their preferences and aptitudes. At Form 3, a range of vocational subjects are to be introduced as soon as possible, once the necessary background work is done regarding curriculum design and assessment procedures, teacher preparation and resources allocation. 5.9 Church and Independent Schools The recommendations in this report have implications for both Church and Independent schools. It is therefore important that all changes be preceded by discussions with all the parties concerned so as to clarify and develop what is being proposed and in order to negotiate any necessary support Admission to Church Secondary Schools Recommendation 1, which concerns the termination of examinations as a means of selecting of pupils in all schools, has immediate implications for many Church boys secondary schools and some Church girls secondary schools. Currently, these schools use the Common Entrance examination as the nondiscriminatory means of admitting students. Once Recommendation 1 is accepted, this would mean the discontinuation of the Common Entrance and other selective entrance examinations. Another system for admitting pupils on a nondiscriminatory basis will be needed. The appropriate system is to be freely established by the competent ecclesiastical authorities, as stated in the Agreement on Church Schools ratified by the Holy See and the Maltese Government on November 28,

178 As proposed above, it would be appropriate for State and Church authorities to discuss and agree on the termination of selective examinations and what provision of support structures are needed subsequently. Working together will give both parties the opportunity to strengthen each other National End of Primary Assessment The next change directly concerning Church and Independent schools will occur with the implementation of Recommendation 4 concerning the national end of primary assessment which is scheduled in the third year of the reform. This assessment will be offered on a national basis for all pupils in State, Church and Independent schools. It will consist of written examinations in English, Maltese and Mathematics with school-based oral and aural assessments in English and Maltese. There will also be school-based assessment in Religion, Social Studies and other areas of the curriculum. Since the written examinations will be set and marked centrally, it is important that the unit responsible for the national assessment would have representatives from the State, Church and Independent sectors End of Year Assessment in Years 4 and Besides participating in the national end of primary assessment, Church and Independent schools may be offered the use of the end of year written examination papers in Years 4 and 5 that are set for state primary schools. The use of common examination papers may help the schools to gauge their strengths in these subjects against the national levels of achievement and to take action as necessary. However, it is important that the examinations do not stifle the autonomy of schools in curriculum development and innovation. It is imperative also that no attempt is made to create league tables Internal/External Moderation and External Monitoring The fourth year of the reform also envisages the strengthening of internal and external moderation and the introduction of external monitoring in Year 4 classes in a sample of subjects or areas of study using representative samples of pupils and schools. Internal moderation of school-based assessment is carried out within state Colleges or other school networks. This exercise primarily involves the moderation of samples of pupils work carried out during the school year, which is used as evidence of the level of achievement reached by each pupil. External monitoring is organized centrally and concerns a small number of areas each year. Details are given in a separate section below. Eventually, monitoring will be extended to the 178

179 upper years of the primary school and to the first two years of the secondary school. All Church and Independent schools are invited to participate in this exercise since its purpose is to monitor national levels of achievement in the chosen areas of study. Before this can happen, it is important that all teachers receive training in the use of level descriptors. Some schools already use level descriptors when reporting students progress and it would be useful if they could share this valuable experience during training sessions and other networking opportunities Support Church and Independent schools that have continuity between primary and secondary education are already experiencing the challenges of teaching pupils with a wide range of abilities in the same class. With the removal of the CE examination, these challenges are expected to increase in Church secondary schools and in the primary preparatory schools. Teachers will need to be given support and training in order to reorganize the educational programmes in these schools. Therefore, the necessary training of teachers in State schools should be extended to all teachers in Church and Independent schools Schools have pupils with disabilities that need very particular attention, and pupils with emotional and behavioural difficulties. Some of these pupils are statemented and Church schools receive funding from the Ministry of Education to recruit learning support assistants. However, Church and Independent schools have to carry the burden of costs of recruiting other learning support assistants, psychologists and social workers to offer assistance to other pupils with serious difficulties. Independent schools usually offer these services to students in need and the parents cover the costs involved. However, parents of statemented pupils in Independent schools are given a tax rebate on these costs. The Review Committee suggests that the guiding principle should be that once a pupil is identified as in need of special attention and is statemented, support should be given irrespective of whether the pupil attends a State, Church or Independent school and the costs are to be covered by the Ministry of Education Teachers in Church and Independent schools also need further professional training to plan, organize and teach mixed ability groups. Therefore, the training of teachers in State schools should be extended to teachers in Church and Independent schools. 179

180 The diversification of the curriculum to accommodate a wider variety of preferences and needs, especially in the secondary schools, will require more resources, teachers and facilities. More cooperation and flexibility will be required from all concerned Recommendations Recommendation 16: The Common Entrance examination and other selective entrance examinations are to be discontinued. Non-discriminatory means are to be sought for admitting students into church schools. The appropriate means are to be freely established by the competent ecclesiastical authorities as stated in the Agreement of the Church Schools ratified by the Holy See and the Maltese Government in State and Church Authorities are to discuss and agree on the termination of selective examinations and the provision of support structures that are subsequently needed. Recommendation 17: Church and Independent schools are to be offered the possibility to participate in the national end of primary assessment made up of written examinations in English, Maltese and Mathematics and school-based oralaural assessment in English and Maltese. Church and Independent schools will also be encouraged to participate in the external monitoring programme of the different curriculum areas in the different year groups which is anticipated to include representative samples of pupils and schools at the national level. The results of such an assessment will be used to determine and monitor national levels of achievement in the various curriculum areas. Recommendation 18: The challenges associated with diversity and mixed-ability classrooms are as real for Church and Independent schools are they are for State schools. Therefore, all training and support services introduced in the state sector are to be made equally available to schools in these sectors. Since the focus of the reform is the pupils, they should be treated equally in the different sectors. Recommendation 19: The collegiality among schools within and among the sectors needs to be strengthened. This can be achieved in several ways by networking of staff and pupils, sharing of facilities, expertise, good practices and support services. 180

181 5.10 External Monitoring Accountability There is a range of possible rationales for accountability mechanisms in education. At one end of the range, the policing mechanism takes the view that many teachers and schools will only do their job properly if they are tightly directed and carefully monitored. At the other end, the formative collegial mechanism takes the view that teachers and schools need helpful, well-informed feedback that they can reflect on and respond to improve their practices (Crooks, 2006). The concept of intelligent accountability attributed to O Neill is defined as follows: An intelligent form of accountability would need to offer to the public, parents and pupils evidence which they can use as a basis for placing trust in teachers and in schools. Such evidence would need to allow people to make informed judgments about where to place trust (O Neill, 2005, p.15) Intelligent accountability leads to better quality education (Crooks, 2006). Six criteria form the foundations of such a system: 1. Intelligent accountability preserves and enhances trust among the key participants in the accountability processes. 2. It involves participants in the process, offering them a strong sense of professional responsibility and initiative. 3. It promotes deep, high quality learning in the domains to be assessed. 4. It recognizes and attempts to compensate for the severe limitations of our ability to capture educational quality in performance indicators. 5. It provides well-founded and effective feedback that promotes insight into performance and supports good decision-making about what should be celebrated and what should be changed. 6. As a consequence of the accountability process, the majority of participants are more enthusiastic and motivated in their work Intelligent approaches to assessment processes at primary and secondary school levels recognize that there is a huge variability in performance among students at any selected age or grade level. This variability makes nonsense of setting a target performance level for all students at an age or year level (Crooks, 2006). The implications of this insight apply to assessment and accountability at the different levels of the education system (see Crooks, 2006 for details). 181

182 National Educational Monitoring System National monitoring of educational outcomes is an accountability mechanism used in many countries. We recommend that such a system be introduced in the local educational system at both primary and secondary levels. Such a system would provide information on how well overall national standards are being maintained and where improvements are needed. A national assessment unit is to be set up in order to organise and manage this enterprise. Its function would be to monitor pupils progress and achievement in all areas of the curriculum. The main message to be given to all schools is that all children can and must make progress during their schooling For reporting purposes, the intention is to develop and use the level descriptors prepared by the Department for Curriculum Management. These level descriptors, which indicate levels of achievement, inherently imply that there is a journey involved in educational progress and achievement and that individual children travel at a different pace and arrive at different destinations. Teachers are the people who can make the most accurate and detailed judgments on their students progress and development. For this reason their judgments must be trusted and valued. In practice, teachers will report their judgments of pupils progress in terms of levels of achievement. The external system of monitoring will be used to calibrate and consolidate teacher judgments The external monitoring system will therefore have two main purposes: (1) It reassures teachers that they are making a good job of assessing their pupils; it will help them feel confident about the value judgments they are making and therefore enable them to trust their competence. It may also identify areas of concern where teachers need further support and training and make a claim for such services. (2) The system would also reassure parents, Heads of school, Principals and Educational Authorities that what is being reported about learning in classrooms is credible and trustworthy Internal and external moderation is also being recommended for additional support and may be used to bridge the gap between decisions taken by individual teachers at classroom level and decisions being made by a national monitoring team. Teachers in the same College teaching the same year group or subject or teachers from a network of schools teaching the same year groups or subject are encouraged to carry out internal and external moderation to gain confidence in using the levels of achievement and reassure their colleagues that 182

183 they are making accurate judgments. This process should also involve Heads of Department and Educational Officers. The system of internal and external moderation is aimed at helping to build up teacher confidence and professionalism in making value judgments and thereby increase the trust that the direct stakeholders, in particular, and the local community, in general, place in them We recommend that through the monitoring system, we build a national cumulative picture in two ways: by assessing entire cohorts of pupils in a limited number of areas as well as gathering information from small national samples of pupils in different areas of the national curriculum. In practical terms, we are suggesting that all pupils are to be assessed in literacy and numeracy skills (English, Maltese and Mathematics) at the end of the primary cycle. 21 All schools in the three sectors are invited to participate in the national end of primary assessment in order to be able to monitor national standards. This system of assessment would provide feedback to individual pupils and their parents, as well as to teachers, schools and colleges. This information is to be used to inform the teaching and learning process of individual pupils and groups and to assist schools in making decisions about their educational programmes and support needed. The information would also be used to work out the value-added of particular schools in relation to student progress and achievement. It may also be used to assist in the distribution of resources and support services. The results would also allow policy makers and politicians to observe particular trends and monitor standards At the same time, we would also recommend regular monitoring, using a five-year cycle, of the different areas of the national curriculum in the different year groups, using representative samples of pupils from representative samples of schools. The sampling of schools and pupils and areas of the curriculum will be done on a random basis. A number of the tasks will be kept constant from one cycle to the other and these are not made public in order to allow trends in achievement over time to be observed and reported. The remaining tasks will be released, making them available for teacher use and allowing detailed and clear reporting of students responses. Such a system would provide a snapshot of children s knowledge, skills and motivation and a way to identify which aspects of the curriculum are improving, staying constant or declining nationally. Participant schools will be informed of the performance of their pupils in relation to the national standards. The results would help schools and systems in planning, developing, 21 Such assessments may be considered at an earlier or later stage of schooling in future. However, at this point in time, they must be seen to replace the current high stakes examinations at the end of the Primary cycle. 183

184 evaluating and modifying their educational programmes. The results of such a monitoring system would influence educational practices in schools and the development of national education policies. Such systems exist in a number of countries, such as New Zealand (NEMP), USA (NAEP), Scotland and Wales. Detailed reports of the design of these monitoring systems and subsequent annual performance reports are accessible on their websites Here we provide one possible scenario of how such a system would be implemented locally as an illustration. We have a five-year monitoring cycle covering ten curriculum areas (Religion, Social Studies, Science, PSD, PE & Health, Expressive Arts, ICT, and aspects of English, Maltese and Mathematics denoted by letters A to J in Tables 5.2 and 5.3). Two curriculum areas (e.g. Social Studies and Expressive Arts) will be chosen annually and each area will be assessed in different year groups (Years 4, 5 and 6). This facilitates the construction, piloting and production of tasks to assess different levels of achievement For each area, there will be a random sample of 480 pupils, which amounts to about 10 per cent of the national cohort at each group, from a random sample of 40 schools. In effect, 12 pupils from each of the 40 schools will be randomly selected from the year group that is being monitored. All of the 100 or so primary schools in Malta and Gozo are to be monitored every year for one or more subjects depending on the size of their pupil population involved Each year there will be 1440 pupils to be monitored in two curriculum areas. With ten trained assessors to carry out the monitoring, each assessor would typically be assigned four schools and would assess all selected pupils in those schools for each area. Therefore, each assessor would need to assess 36 pupils (12 pupils in each year group) in each school or 144 pupils in all. The time required to complete the exercise depends on the length and nature of the tasks; whether they are individual or group tasks and whether computers, station tasks or performance tasks are used. With careful planning, the whole data collection should be completed in four weeks, preferably in May; marking will be in June and analysis in July Table 5.2 presents an example of an annual plan in which two curriculum areas A and B are monitored. The selection of the areas may be different and need not be advertised beforehand. Table 5.3 presents an example of planning over a period of five years. During this period all ten curriculum areas (A to J) can be monitored in each of Years 4, 5 and 6. The cycle can then be repeated. The 184

185 sequence of curriculum areas to be monitored is given only for illustrative purposes. It may be necessary to monitor certain areas more frequently and others less frequently than indicated. Table 5.2 Example of an Annual Plan for External Monitoring Curriculum Areas A, B A, B A, B Year Group Year 4 Year 5 Year 6 Pupils sample size Schools sample size Table 5.3 Distribution of Curriculum Areas for Monitoring over a Five-Year Cycle Year 4 Year 5 Year 6 First Year A, B A, B A, B Second Year C, D C, D C, D Third Year E, F E, F E, F Fourth Year G, H G, H G, H Fifth Year I, J I, J I, J The assessors themselves or a separate team needs to design, construct and trial the assessment tasks before they are used. In addition, teachers are needed for specific periods of time to mark the tasks. The performance of the pupils will be reported using levels of achievement and results will be calibrated against teacher assessment. Follow-up meetings will be organised where materials and standards are shared among teachers Such a system needs to have a budget for five years to cover the expenses associated with a complete cycle. Expenses include the employment of personnel involved in the management of the system, task development and review, technical analysis of sampling and data and report writing. Other expenses are related to material resources needed and to the printing of the reports. Professional training for teachers associated with the outcomes of the results needs to be considered in addition to an external review of the process that looks into fitnessfor-purpose, impact and backwash effects International Testing Programmes At this point in time, we also recommend that Malta participates in international testing programmes such as PIRLS, TIMSS, and PISA which provide comparative, cross-national information on the educational achievement of national 185

186 cohorts of students in different countries. Such participation would help put us on the map and allow an external form of accountability to be in place. At the same time, we acknowledge that such programmes have their limitations (see Crooks, 2006). They are based on paper-and-pencil tests in a comparatively narrow range of curriculum areas, usually reading, writing, mathematics and science. Issues of comparability of translations, match between the tests and national curricula, and varying levels of test-wiseness of students in different countries limit the validity of such inter-country comparisons. The biggest risk with these assessments is that poor rankings may undermine public trust in the quality of education, without adequate account being taken of the relevance and validity of the information for the local context. Despite these limitations, we believe that there is value in participating in such programmes at this point in time when much effort is being concentrated on ensuring that our educational system is at par with other systems of education Associated Needs We need an accountability system that supports and confirms teacher professional judgment and not replace it. We need to come together to look at how standards might be shared and how the accountability agenda be met. Significant effort is needed to increase and improve teacher assessment and moderation practices, bringing in professional help where necessary. Evidence about national standards needs to be gathered through rich samples of students work. Systematic change of practices needs to be facilitated by the adoption of ideas about transformational change, prioritizing the need for a shared vision with all constituents being involved in the process (Cambridge Assessment, 2007). The system must be seen to have important systemic integrity, which wins the trust of different stakeholders and increases our enthusiasm for quality education Recommendations Recommendation 20: A system of national monitoring of educational outcomes is to be introduced in the local educational system at both primary and secondary levels. Such a system would provide information on how well overall national standards are being maintained and where improvements are needed. Recommendation 21: A national assessment unit is to be set up in order to organise and manage this enterprise. Its function would be to monitor pupils progress and achievement in all areas of the curriculum. 186

187 Recommendation 22: Malta is to extend its participation in international testing programmes such as PIRLS, TIMSS, and PISA which provide comparative, crossnational information on the educational achievement of national cohorts of students in different countries Concluding Observations On the basis of the evidence at hand, it appears that currently the rate of success at the end of compulsory schooling reaches 50 per cent of the age cohort. This estimate is based on achievement in academic subjects since practically this is the only clearly measurable scale that we possess. Following the main objective of the policy document For all Children to Succeed, this review has looked into the current situation of transition from primary to secondary education with the aim of increasing the current rate of success for a significantly higher proportion of students when they are at the end of compulsory schooling. It is our hope that our evaluation and recommendations lead to an education system that allows our children to develop and grow in a school environment that is supportive and at the same time challenging. We also hope that our education system develops in students values that enhance social cohesion. All members of our local community need to be individuals who are confident and have good self-esteem and are therefore willing to strive forward yet at the same time feel a sense of responsibility towards others in their community. We believe that all our schools, in the State, Church and Independent sectors, should be places where children of different abilities develop, learn and grow together. The range of educational experiences on offer should allow all children to find stimulating areas of study that motivate them to learn and achieve a variety of competencies and skills that are beneficial to them. In the foregoing report, we propose assessment practices that focus on individual children s levels of progress and achievement so that realistic targets are set accordingly. We know that teachers are the best judges of their children s learning styles, needs, progress and achievements and therefore we have proposed a system that values and respects teacher judgments. Complementary to this, we feel the need for an accountability system that provides teachers with the necessary support in stabilising their judgments and one which highlights successes and indicates areas where improvement is needed. Research, training and evaluation of systems need to be felt to be part and parcel of the way forward being proposed to ensure that we remain on target to implement our vision. It is imperative that all 187

188 stakeholders are given time to understand and strengthen their belief in this vision. Furthermore, it is equally important that teachers, parents and schools learn to accept that the changes are primarily of benefit to all our children. All changes require effort, time and commitment on the part of the individuals concerned. It is hoped that the necessary support and networking will be set up that strengthen reciprocal trust within and among the different educational sectors. This is necessary to transform our vision into a tangible reality. 188

189 Bibliography Agreement on Church Schools between the Holy See and the Republic of Malta (28 November 1991). In Scerri, D.V. (2000). The Saga of the Church Schools in Malta ( ) a historical portrayal of events. Malta: MUT Publications. pp Bartolo, P. A. (ed.) (2007) Responding to Student Diversity Teacher s Handbook. Malta: Faculty of Education. Bartolo, P. A. (ed.) (2007) Responding to Student Diversity Tutor s Manual. Malta: Faculty of Education. Bezzina, C, (2003) Devolution and Deregulation of Education in Malta: an introductory report. Malta: Foundation for Educational Services, Ministry of Education, Youth and Employment. Bezzina, C. (2005) Networks for nurturing professional learning communities. Keynote presentation at the Learning Through Peer Collaboration EXCEL Network National Conference, Malta, May. Black, P. (1998) Testing: Friend or Foe. London: Falmer. Black, P. and Wiliam, D. (2005) The reliability of assessments. In John Gardner Assessment and Learning. London: Sage Blyth, W.A.L. (1965 ) British Primary Education. London: Routledge Blyth, W.A.L. (1984) Development, Experience and Curriculum in Primary Education. New York: Croom Helm. Bonnici, B. and Soler, L. (2002) Repeating Year 6: Views and Perceptions of Different Stakeholders. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Borg, C. and Mayo, P. (2006) Learning and Social Difference: Challenges for Public Education and Critical Pedagogy. London: Paradigm Publishers. Brincat, S. and Mamo, C. (2001) Occupational Stress amongst Year 5 Maltese State School Teachers. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Brooks, R. and Tough, S. (2006a) Assessment and Testing: making space for teaching and learning. London: Institute for Public Policy Research. Available online: Brooks, R. and Tough, S. (2006b) Pupil Attainment: time for a three Rs guarantee. London: Institute for Public Policy Research. Available online: Cambridge Assessment (2007) Proceedings from the 2nd House of Commons Research Seminar 25 th April. Cameron, J. (1970) Malta: Educational Planning. Paris: UNESCO Cassar, M. (2003) Stress of 11+ exam. The Sunday Times, 16 th February. Malta 189

190 Cassar, R. (1991) Stress and the Junior Lyceum-oriented Pupil. Unpublished B.Ed.(Hons.) dissertation, University of Malta. Catania, A. (1991) Traditional Examinations why the need for change? Unpublished B.Ed.(Hons.) dissertation. University of Malta. The Common Good and the Catholic Church s Social Teaching. Catholic Bishops Conference of England and Wales, Available online Cauchi, N. and Scerri, G. (2003) Investigating Methods of Assessment in Maltese Primary State Schools. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Commissioner for Children (2007) Children are now! Annual Report Malta Crooks, T. (2006) Excellence in Assessment for Accountability Purposes. Keynote address presented at the Northumbria EARLI SIG Assessment Conference 31 August 2006 Curriculum Department (2003) Level Descriptors. Malta: Education Division. EFA Global Monitoring (2005) Education for All: The Quality Imperative. Paris: UNESCO Debrincat, J. and Falzon, M. (1996) The Impact of Private Lessons on Teachers Esteem in the Community. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Department for Curriculum Management (2005) Primary School Curriculum. Malta: Education Division. Duca, J., Gauci, J.B., Muscat, C., Muscat, M.A., Pace, N. and Pace, P. (1999) Parental Perceptions Regarding issues Pertinent to the Primary Education in Malta. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Educational Assessment Unit Reports on the Junior Lyceum Entrance Examinations. Malta: Education Division. [Reports for the years 1994 to 2003 have been consulted.] Federation Parent Teacher Associations Private Schools (1983) Il-Genituri u l- Iskejjel Privati. Malta. Galea, T. (1991) The Bottom-less Buckets: An experience of Year 6 lower-streamed primary pupils. Unpublished B.Ed.(Hons.) dissertation. University of Malta. Gardner, H. (1983) Frames of Mind: the Theory of Multiple Intelligences. New York: Basic. Garrone, G.M. and Javierre, A.M. (1977) The Catholic School. Congregation for Catholic Education, Rome. Available online Gatt, C. and Vassallo Agius, S. (1988) A Study of the Knowledge and Attitudes that Primary School Pupils and their Parents have on Streaming. Unpublished B.Ed (Hons.) dissertation. University of Malta. 190

191 Grace, G. (2003) First and Foremost the Church offers its educational services to the poor : class, inequality and catholic schooling in contemporary contexts. International Studies in Sociology in Education, 13, 1, Grace. G. (2002) Catholic Schools: Mission, Markets and Morality. London: Routledge Falmer Grech, L. (2003) Parents and the assessment of children at the end of Year 6. The Sunday Times, 1 st June. Malta. Grima, G. and Farrugia, J. (2006) Transition From Primary to Secondary in Malta. Time to Break the Mould? FES Educational Publications Series Families and Schools. Malta: Allied Publications Ltd. Grima, G., Camilleri, R., Chircop S., Ventura, F. (2005) MATSEC: Strengthening a National Examination System. Malta: Ministry of Education. Guidance and Counselling Services Tracer Studies 2003, 2004 and Malta: Education Division. MATSEC Support Unit SEC Examinations Statistical Reports. Malta: MATSEC Examinations Board. [Reports for the years 2001 to 2006 have been consulted]. Ministry of Education ( ) Junior Lyceum Entrance Examinations Reports. Malta. Kelly, A.V. (1999) The Curriculum: Theory and Practice. (4 th Edition). London: Paul Chapman Publishing. Laghi, P. and Martins, J.S. (1997) The Catholic School on the Threshold of the Third Millennium Congregation for Catholic Education, Rome. Available online con_ccath... Lewis L.J. (1967) Malta Education Planning (Preparatory Mission). Paris: UNESCO. M.U.T. The Teacher, April-September, Malta. Mallia, G. (2003). Child hardships and Junior Lyceum exams. The Sunday Times, 26 th January.Malta. Mansueto, R. (1997). Examinations and Stress: childhood and education subverted. In R.G. Sultana (Ed.) Inside/Outside Schools. Malta: PEG. Mercieca H. (1991). Streaming in the Primary School: The Maltese Experience. Unpublished M.Ed. dissertation. University of Malta. Mifsud, C., Milton J., Brooks G. and Hutchinson D. (2000) Literacy in Malta: The 1999 National Survey of the Attainment of Year 2 Pupils. National Foundation for Educational Research and the University of Malta. Mifsud, C., Milton, J., Hutchison, D. and Brooks G. (2000). Do Schools Make a difference? A report on using the Malta national literacy survey as a basis for a school effectiveness or value added study. University of Malta Literacy Unit Monograph Series, No

192 Mifsud, C., Grech, R., Hutchinson, D. Morrison, J. Rudd, P. Hanson, J. (2004) Literacy for School Improvement: Value Added for Malta. Malta: Media Centre Publications. Mifsud, C., Richard, S., Hutchinson, D. and Grech, R. (2005) Mathematics in Malta: The National Mathematics Survey of Year 1 Pupils. Education Division/Literacy Unit and NFER UK. Ministry of Education (1998) Draft of the National Minimum Curriculum. Malta: MEYE. Ministry of Education (1999) National Minimum Curriculum: Creating the Future Together. Malta: MEYE. Ministry of Education (2001). Strategic Plan: National Curriculum on its Way. Malta: MEYE. Ministry of Education (2005) For all Children to Succeed. Malta: MEYE. Ministry of Education (2005) School Attendance Improvement. Malta: MEYE. Muscat, S. (2002). From primary to secondary education. The Sunday Times, 21 st April. Malta. Notice in the Government Gazzette (04/02/05) O Neill, O. (2005). Assessment, Public Accountability and Trust. UCLES: Cambridge Assessment Conference proceedings: A Question of Confidence: Maintaining Trust in National Examination Systems October. Organisation for Economic and Development Cooperation (OECD) (1998). Overcoming Failure at School. France: OECD. Organisation for Economic Co-operation and Development (OECD) (2004). Learning for Tomorrow s World First results from PISA France: OECD. Pace Moore, E. (1999). Modifying the Primary Curriculum to suit Mixed Ability Classes. Unpublished B.Ed. (Hons.) dissertation. University of Malta. Private Schools Association Reports on the Common Entrance examinations. Malta. [Reports for the years have been consulted.] Pullicino, C. (2001). Second Class Pupils? A Case Study of Ten Primary School Pupils. Unpublished B.Ed. (Hons.) dissertation. University of Malta. Reports on the Working of Government Departments: Education. Malta. [Reports for the years and from 1968 to 1983 were used.] Savona, S. (1883) Report on the Educational Institutions. Malta: Government Printing Office. Scerri, D.V. (2000) The Saga of the Church Schools in Malta ( ) a historical portrayal of events. Malta: MUT Publications. Sollars, V. (2006) Children s Right to Play: a study of Maltese children s perceptions on cultural and recreational activities. Malta: Office of the Commissioner for Children. 192

193 Sollars V. (2006). Children s Right to Play a study of Maltese children s perceptions on cultural and recreational activities. Malta: P.E.G. Ltd. Spiteri, L., Borg, G, Callus, A.M., Cauchi J. and Sciberras, M. (2005) Inclusive and Special Education Review. Malta: MEYE. Sukhnandan, L. and Lee, B. (1998). Streaming, setting and grouping by ability: a review of the literature. UK: NFER. Sultana, R.G. (ed.) (1997). Inside/Outside Schools: Towards a critical sociology of education in Malta. Malta: PEG. Sultana, R.G. (1989). Streaming: a sociological perspective. Education 3, 3, Sultana, R.G. (1992) Education and National Development: Historical and Critical Perspectives on Vocational Schooling in Malta. Malta: Mireva Publications. Sultana, R.G. (1996) Under-Achievement: Are Students Failing School or Are Schools Failing Students. In F. Ventura (Ed.): Secondary Education in Malta. Malta: MUT Publications. Tough, S. and Brooks, R. (2007) School Admission: fair choice for parents and pupils. London: Institute for Public Policy Research. Available online: Tough, S. and Reed, J. (2006) Curriculum,Assessment and Pedagogy: beyond the standards agenda. London: Institute for Public Policy Research. Available online: Guidance and Counselling Services Tracer Studies 2003, 2004 and Malta: Education Division. Vassallo, J.P. (1957) Report on the Department of Education for the scholastic Years In Reports on the working of Government Departments during the Financial Year Malta, pp Wain, K., Attard, P., Bezzina C., Camilleri R., Darmanin M., Farrugia, C., Psaila A., Sammut J., Sultana R., Zammit L., (1995) Floriana: Ministry of Education and Human Resources. Wain, K. et al. (1995). Tomorrow s Schools: Developing Effective Learning Cultures. Malta: Ministry of Education. World Bank ( Zammit Mangion, J. (1992) Education in Malta. Malta: Studia Editions. Zammit Marmarà, D. (2001) The ideological struggle over Comprehensive Education in Malta. In R.G. Sultana (ed.) Yesterday s Schools: Readings in Maltese Educational History. Malta: P.E.G., pp

194 Appendix 1 Discussions on Transition From Primary to Secondary 1. Meetings with Stakeholders Interviews with Yr. 6 Pupils of Floriana Primary School Interviews with Yr. 6 Pupils of Zurrieq Primary School Interviews with Yr. 6 Pupils of Paola Primary School Interview with Yr. 6 Pupils of St Francis School, Cospicua Interviews with Yr. 6 Pupils of Mosta Primary School Interview with Yr. 6 Pupils of Stella Maris School, Balzan Interview with Yr. 6 Pupils of Laura Vicuna, Gozo Meeting with Parents at Laura Vicuna, Gozo Meeting with Parents of candidates sitting for JL and CE Examinations at NCC, Hamrun Meeting with Parents at Victoria Primary School, Gozo Interviews with Yr. 6 Pupils of Victoria Primary School, Gozo Meeting with Teachers and Head of Garendon School, Zebbug Interview with Yr. 6 Pupils of Garendon School, Zebbug Meeting with Administration of De La Salle Primary School and Principal of De La Salle College Meeting with Head of School and Yr 6 Teachers of Convent of the Sacred Heart, St Julians Meetings and interviews with Teachers and Students of San Benedittu Secondary School, Kirkop Meeting with Head of school, Assistant Head and Yr. 6 teachers of Zabbar Primary school Meeting with Head of school and Yr. 6 teachers of San Gwann Primary school Meeting with Gozo Primary Teachers and Facilitators attending Inservice Training Meeting with Education Officers responsible for formulation of Syllabi of the Junior Lyceum Examinations Meeting with Private Schools Association Examinations Board 194

195 Meeting with Education Officers Curriculum Meeting with members of the Faculty of Education Meeting with Private School Association Meeting with AKS and APCCS Meeting with Prof Gerald Grace and Heads of Church Schools Meeting with Professor Kenneth Wain Meeting with Private School Association Meeting with College Co-ordinators Meeting with Mr Joseph Zammit Mangion Meeting with Malta Labour Party Officials Meeting with Malta Union of Teachers Officials Meeting with Alternativa Demokratika Officials Meeting with Prof Ronald Sultana Meeting with Officials of Education Division Meeting with Mr George Borg, College Co-ordinator Meeting with College Co-ordinators Meeting with Heads of Independent Schools Meeting with Officials of Education Division Meeting with College Co-ordinators Meeting with Maltese Bishops & Church Officials Meeting with Council of Church Schools Meeting with Minister Louis Galea Meeting with Provincials of Religious Orders and Heads of Church Schools Meeting with Mr George Borg, College Co-ordinator 195

196 2. Educational Programme on Channel 22 Personalities interviewed by Ms Annalise Ellul regarding the issue of transition from Year 6 to Form Dr Grace Grima: Chairperson, 11+ Review Committee Mr Marjoe Cassar: Guidance Teacher Dr Carmel Borg: Dean of the Faculty of Education Mr Colin Calleja: Lecturer, Faculty of Education Dr Victor Martinelli: Child Educational Psychologist Ms Bernie Mizzi: Head of an Independent School Mr Mario Cardona: Teacher, FES Mr Peter Vassallo: ADE (Assessment) Fr David Cilia: PSA Chairperson 3. Workshops Four workshops held as part of the activities held on the occasion of the launch of the publication: Transition from Primary to Secondary in Malta: Time to break the mould? by Grace Grima & Josette Farrugia. 196

197 Appendix 2 PERSPECTIVES ON THE 11+ EXAMINATION Joseph Fenech Introduction This paper treats the issue of transition from the primary to the secondary sector of schooling in the Maltese educational system and is organized as follows: The first section gives a brief history of the 11+ examination and embeds it in the culture of selection which has always played a determining role in Malta s educational development. The second section reviews the curriculum traditions of primary schooling while the third section uses the responses of a teacher questionnaire to assess the impact of the 11+ on the curriculum enacted in primary schools and classrooms. The concluding section raises a few questions and offers some ideas to enable future policy decisions. I. A brief history of the 11+ Examination Paul Black s (1998:22) statement that the assessment and testing system within any given society can only be understood in relation to the history and culture of its educational and social systems cannot be gainsaid. It is a distillation of the insights and reflections of one who has spent so much time researching educational assessment both in the U.K. and in other countries. It is with this observation in mind that I want to trace briefly the history of the 11+ in Malta s educational system. The 11+ is deeply embedded in a culture of selection which has characterised educational development in our Island. It needs hardly pointing out that Malta s educational system emerged first at higher levels (secondary and tertiary) and, subsequently seeped down, so to say, at lower levels (elementary). This phenomenon has generated and reinforced what Blyth (1965) called the preparatory tradition in the schooling of primary school children. The fact that the Collegium Melitensae established in 1592 had a preparatory class to prepare pupils for secondary education which set the pattern for future development is a highly significant historical occurrence. 197

198 It can be said that, at least at the level of ideas, this tradition continued to be reinforced with the introduction of mass schooling in the 1850s. In 1865 Savona delivered a lecture to the Maltese establishment entitled On the Necessity of Educating the People in which he began to articulate his rationale for selective examinations. In 1881 he chaired a Commission, which he himself established, to suggest a scheme for the organization of the Lyceum. Their report was approved by the Governor in April In his report as Director of Education in 1883 he expressed his belief in the opening statement of his account of the Lyceum: This institution, he wrote was intended to complete the education given at the Government Primary Schools or in private schools, and, at the same time, to prepare candidates for admission to the Arts Faculty in the University (Savona, 1883:14). It is this firm belief in the preparatory function of primary schooling that is captured in the confident tone evident in his report of 1883: It is absolutely necessary to alter the present regulation concerning the admission of students to pursue their studies at the Lyceum, in order to maintain to that institution its character as a school for secondary instruction. No one shall be admitted therein, except after passing a searching exam in all the subjects taught in the higher classes of the city primary schools. (p.17-18). Savona legitimated this conviction, which, as Director of Education, he translated into actual policy, by referring to four important reasons which reflected both concerns and contingencies: Selection by examination was the current policy in countries like Britain, Prussia, Austria, Holland and Italy; non-selection would reduce a considerable portion of the Lyceum to a primary school; there was lack of teachers to cope with increasing numbers of students; and, the existing accommodation placed limits on the number of students who could be admitted. This policy continued to be strengthened by the educational developments in the decades that followed. The following are the most conspicuous: the detachment of the Lyceum from the University of Malta in 1913; the introduction of the scholarship examination in 1915; the establishment of the Central School (with an examination at point of entry) to offer secondary level of education to prospective primary school teachers in 1923; 198

199 the amalgamation of the Elementary Schools Department with the Secondary Schools Department in 1932; and the establishment of the Preparatory School by Dr Laferla in However, the competitive dimension of the 11+ began to be glaringly apparent after the Second World War when the demand for education increased sharply. The Committee appointed by Dr Ganado, the then Minister of Education, to report on and recommend developments in the primary sector of education in 1948, reaffirmed the selection of pupils at 11 to ensure that successful candidates are able to follow a Grammar School course (Department of Education, :xxxv). They also recommended the extension of the opportunity to sit the examination to those beyond 11 years of age but not yet 14 in order to offset the handicap which pupils may have suffered through part-time attendance in the primary schools ( :xxxv). It must be said that, although they recommended that the concession was an emergency measure and should be withdrawn when a return to normal conditions has been affected ( :xxxv), it continued to be established policy until 1970 when secondary education for all was introduced. Discussion about selection at 11 re-emerged at this time. But, in the circumstances, the tripartite structure in the secondary sector was retained, although some innovations in the selection mechanism, in line with the recommendations of Tuppen (1969), were introduced. However, this was to be a temporary measure. Cameron, the educational expert who was advising the government on the implementation of secondary education for all at the time, was very explicit on this: The secondary education for all scheme is only one stage in a long process of development. It is a big one, but it merely continues the process of expanding secondary education which began at least two decades ago. Nor does it end the process. It is not the final solution. It will be amended and redefined again and again in the light of experience let us first put secondary education for all into practice, ensuring that it remains flexible enough to be easily adapted to meet the further need for changes which will inevitably occur (Cameron,1970:para.3b.2) This was, in fact, the official position of the government as evidenced by the ministerial statement during the parliamentary debate (Parliamentary Debates,23/3/70:5742) and the minister s own intervention during a seminar organized by the MUT where he was reported to have said that the form we have 199

200 chosen will lead in its own good time to a comprehensive system. (The Teacher, April-September 1969: 3). That good time came barely two years later with the change of Government in June 1971, when the 11+ was abolished and the comprehensive system of education was introduced. This situation was to last only a decade because, the Labour Government which introduced a comprehensive secondary school system in the early 1970s abolished it and re-introduced selection and a tri-partite structure in The only changes in this system were made in the late 1980s when Social Studies and Religion were added to English, Mathematics and Maltese as examinable subjects at 11+. The decision to abolish the comprehensive system and re-introduce tripartism and selection in 1981 highlights the cultural embeddedness of selection in our educational system as well as the inability of policy-makers to implement a successful policy which is not based on strict selection at the age of 11. II. The Primary Curriculum The way the curriculum is enacted in today s primary schools and classrooms is a legacy of the nineteenth century when mass elementary schooling was introduced. It was at that time that the governing principles, traditions and the curriculum system itself were laid. These constituted the educational legacy which continued to define the work and culture of the primary schools to the present day. A brief review of these will lead to a better understanding of the curriculum and pedagogy of primary schooling in Malta today. Governing principles The socio-political context, demographic features, Malta s status as a British colony as well as the administrative culture of the Catholic Church made centralism the most congenial principle for the administration of the system of mass schooling in Malta. Decisions were taken at the centre and for many decades also by a single individual, about the form and manner in which elementary schooling was to be provided in the evolving system of schools and classrooms. This centralist tendency determined to a large extent important aspects of the educational process such as the curriculum system, the pedagogy, the organization of the primary schools and the training of teachers. 200

201 A second principle which the central administration established from the earliest years was the principle of uniformity. It was a highly valued principle, strengthened by the belief that there was one best way to teach, for teaching and learning in the schools to be as uniform as possible. Uniformity was ensured through centralist strategies like the prescription of syllabuses and texts, annual testing and inspections of schools and pupils, the allocation of resources and departmental circulars detailing the professional activity of teachers. In conformity with the norms of the times, and rooted in the philosophy of the encyclopaedists, the curriculum was thought of in terms of clearly demarcated subject matters portraying the areas of knowledge classified at the time. This led to a bureaucratized curriculum process, with strict time allocation for subjects, prescribed content and pedagogy and close monitoring of its enactment from the centre. The last highly regarded principle was the principle of teaching efficiency believed to be achieved through the establishment of bureaucratized institutional structures. In the previous paragraph, I have already remarked about the bureaucratization of the curriculum process. The form of school organization can be added to this. Children were classified into homogeneous learning groups, organized into classes labelled as standards, initially from I to IV and, with the extension of elementary schooling, from I to VI on the basis of test results. It must be pointed out in this regard that this was the result of borrowing policies and practices from European countries besides Britain. Traditions It is the characteristic of institutional structures to generate customs and norms of behaviour as they evolve and develop over the years. These constitute what are known as traditions and they serve a number of purposes, which include: channelling policies in a particular direction; giving activities a particular quality; establishing continuity with the past; regulating how innovations are taken up; and providing a framework for the definition of problems. 201

202 As has already been mentioned in the previous section, three curricular traditions have emerged in primary schooling. They are (after Blyth, 1965): the elementary, the preparatory and the developmental. All three are distinguished by clearly identifiable characteristics. The elementary nature of the curriculum of primary schooling, besides being reflected in the qualifier elementary school, was evidenced by its narrow emphasis on reading writing, arithmetic, and in our case, religion. It was based on simple, rudimentary knowledge, reserved for the lower classes of society, concerned with moral rescue, and organized in terms of standards for each year. In their report to the Minister of Education the committee chaired by J.P. Vassallo (Department of Education, 1948:vi) described it with acceptable accuracy: In the work of the primary schools, therefore, after the character formation and the teaching of the principles of Catholic living, the teaching of English and Arithmetic must constitute the most important part of the work. Although more than half a century has passed since then, it may not be surprising to find that there are remnants of this tradition in certain schools and classrooms today. What is more glaringly evident in the way the work of primary schools today is conducted, however, is the preparatory tradition. In tracing the history of the 11+ earlier on, I have pointed out that it emerged within, and eventually reinforced, the preparatory tradition. A review of its characteristics is in order at this point. As has already been remarked, the preparatory quality of primary schooling was legitimated by a particular view of the provision of education in a society. As against the elementary view which thought of schools as self-enclosed institutions providing basic education necessary for work requiring low skills, the preparatory view looked at primary education, as its name implies, as a stage in a longer process leading to the secondary and tertiary sectors. The curriculum, therefore, was planned and designed in sequential form based on an increasingly more complex selection of knowledge as the pupil moved up from one standard to another and one sector to another. In many European countries it was not only attached to grammar schools but also made accessible to the upper classes of society (Blyth, 1965). The concept of a preparatory curriculum is an improvement on the elementary in a number of ways: it conceives of education as consisting of stages with particular structures, learning as sequential and progressive and knowledge as increasingly 202

203 more complex. However, because of the ideology of selection through which it was legitimated, it made streaming and examinations the predominant features of primary schooling both in Europe and the Anglo-Saxon world. While, for a few decades after the late 1950s and 1960s these features came under attack by psychometricians like Vernon (1957) and sociologists like Halsey and Gardner (1953) and Halsey (1972) and comprehensive schooling introduced with a good degree of success, the voices of selection, especially in Britain, had not been completely silenced and the impetus to provide for selective schooling still survives. In Britain, for example, secondary school heads have been granted discretion to determine a percentage of their entry by aptitude and ability in order to boost the school s performance in the league tables. The developmental tradition was the last to appear on the educational landscape in many countries. Its origins are traced back to Rousseau s writings which, in turn, influenced educationalists like Pestalozzi and Froebel. It emerged in the wake of the discovery of childhood in the nineteenth century, the emergence of psychology as a discipline, the dissemination of emancipatory ideas in the early twentieth century, the concomitant economic development in many societies and the enactment of social legislation, especially regarding child labour. Developmentalism places the child at the centre of the educational encounter, highlights the importance of the process of learning and looks at the curriculum in holistic terms rather than as fragmented domains of knowledge. In its emphasis on the child and the learning process, it relegates the curriculum and the teacher to a secondary position and consequently, in its extreme forms, has attracted the criticism of many writers ( Cox and Dyson, 1971; DES, 1981; Alexander, 1994). In the development of primary schooling in Malta, developmentalism made its appearance at two levels: the level of ideas and the level of practice. One did not always lead to the other; however, a cursory glance at this development will find evidence of both levels. The ideas of child-centredness in education began to form part of educational discourse in Malta in the early twentieth century, largely as a result of our connections with Britain. In those days there were two major channels: teacher education provision and the Malta Union of Teachers. In the years following the Second World War, the teacher training institutions also made a significant contribution. In the last two decades of the nineteenth and for a long time in the 203

204 twentieth century, four of the best teachers from the Training School were sent annually to Britain to upgrade their professional education and training. At that time teacher training institutions in Britain were imbued by the philosophy of childcentredness, especially Froebel s. Some of them began to disseminate these ideas on their return to Malta in their schools as headteachers. One of them was E B. Vella who used the MUT publication The Teacher in order to disseminate these ideas more widely. In an article he wrote in The Teacher in the late twenties he expressed himself in the language of the lecture hall: The child, he wrote, is no longer to be regarded as a passive recipient but he is considered as an active subject well worth of careful and serious study.. the new teacher first studies the child and then adopts his methods accordingly. As an outcome of psychological research we now hear of new classification, order, of concrete methods: of discovery, of playway in teaching of learning by doing (MUT, 1928:13) The MUT, through its contact with the NUT in Britain, access to educational writings and its publication The Teacher served also as a source for the dissemination of child-centred ideology. Quoting from the Hadow Report (Board of Education,1931), it asked rhetorically in its editorial (The Teacher, Vol.3 (9), 1945): Can it be reasonably said that the curriculum is sufficiently realistic and thought of in terms of activity and experience rather than of knowledge to be acquired and facts to be stored? Is school work inspired by the needs and possibilities peculiar to the definite phase in child-life under review? This child-centred ideology was found a fertile soil in the infant classes of the primary school. It was in a spate of articles in The Teacher in the early twenties that this was articulated. One of these writers articulated it in a Froebelian idiom: The infant teacher must become acquainted with the child s mind and its peculiarities; she must know the material at hand before she works on it. She must have discovered that, not only is the young child most receptive of all created things, but that the germs of all his future possibilities lie dormant in him, only waiting for the hand that will tend and develop them (The Teacher, Vol.1 No. 10 December, 1920:5). 204

205 Later on Laferla, in establishing the Preparatory School, mingled with this the ideas of Montessori to qualify the practice in the infant classes: It (the school) aims at developing the powers of observation of a child in an atmosphere of joyous activity. (Department of Education, :501) and a year later in this school we aim at making the children happy and we can claim that we have been successful in doing so. Not only do we avoid any strain on the young mind, but we study every child individually and, as far as possible, follow its bent (Department of Education, :437). After the Second World War especially, child-centred ideas began to reach the level of practice through the work of the Teacher Training Colleges. This could happen because the Religious Orders who ran these institutions disseminated these ideas and, more importantly, they exhorted students to put these ideas into practice in their classrooms during teaching practice. This is the legacy of the child-centred philosophy which still prevails in certain classrooms, the Education Division and the Faculty of Education. The curriculum system The planning and design of the curriculum in the Maltese educational system has always been a centralized activity. The schools were only seen as implementation sites of a curriculum that needed to be carefully supervised and strictly controlled. In the earliest years it had to be so. The system was of a very small size, expertise at the school level was non-existent, the teacher quality was low, teacher training was very primitive and teacher associations had not yet appeared on the educational scene. It eventually became a system that embedded itself so strongly in the administrative culture that it would take more than a re-educative exercise to lessen its pervasiveness. To give a brief description, in the early years it consisted of a one-man task: the prerogative of making curriculum decisions rested with the Director of the Elementary Schools. He wrote the syllabuses, selected or produced the textbooks, planned the time-table, inspected the schools and examined the pupils. Eventually, after Savona, the task remained at the centre but devolved to the inspector of the elementary schools. When the number of inspectors grew in the1930s, Laferla chose a team of inspectors to produce textbooks for the teaching of Maltese, History 205

206 and Geography and made arrangements for an English textbook to be adapted to Maltese primary pupils. (Figure 1 gives an overview of the Malta s curriculum system) With educational development after the Second World War, including increasing expertise in the department of education, the establishment of teacher training institutions and higher administrative skills in the schools, decision-making in the curriculum became more collegial. To be sure, the tasks continued to be those of syllabus writing, compiling or prescribing textbooks and providing time-tables, they were performed, however, through the structure of ad hoc committees called, in the jargon of the administration Syllabus and Textbook Committees. On these committees sat representatives from the inspectorate, the teacher training institutions and headteachers of the primary schools. On the completion of their task, the committees were made to disband. Although, after the 1990s and the publication of the National Minimum Curriculum, decentralization as a policy value entered educational discourse in Malta and the centre has been encouraging the peripheries to take more initiatives in the design and planning of curricula to meet the needs of pupils, the dependency culture generated by decades of centralized decision-making is preventing the schools from becoming more enterprising in curriculum development. 206

207 III. Impact of the 11+ on the curriculum and pedagogy of primary schools Introductory It cannot be denied that tests and examinations exert an influence on the curriculum and pedagogy of schools, especially when they are high stakes ones. The 11+ is one of these. Depending on the outcome, students are placed either in the academic route or not. If they miss the first, their path to a satisfactory educational achievement is often laid with enormous (often insurmountable) obstacles. Thus, there is general agreement among educationalists internationally that the influence of the 11+ cannot be but harmful. With reference to this, the Plowden Committee nearly forty years ago, argued that it leads to a narrowing of the primary curriculum, an excessive emphasis on the acquisition of measurable skills and rigid streaming (CACE, 1967:153). Although, in our educational culture, both in the past and more recently, the 11+ examination has been the target of criticisms for bringing about this situation (e.g. MUT, 1981:2), many still regard it as an integral part of the process of primary schooling and as providing both teachers and pupils a worthy end in view. What a teacher of a Year 6 class said in an ethnographic study of primary schooling is shared by a good number of her colleagues: If there is no 11+ what are our efforts for? It gives us a sense of where we should be going and what we should be doing. Examinations lend seriousness to teaching, motivate pupils to work hard and give us some satisfaction at the end. Besides, a good result is a reward to those pupils who work hard during the year. (Fenech, 1992:326). This perception of the status of the 11+ is also shared by those in the administrative echelons of the educational system. In the study quoted above, the headteacher of one school found the 11+ examination to be a valuable instrument in his attempt to raise the status of his school in the community. When he took up the administration of the school, this is what he said he did: I assembled the staff and made them aware of the school situation. I asked them to help me change it. Increasing the number of passes in the 11+ examination became my overriding educational goal. It was kept at the focus of the school s concern. I tried to rally the support of the whole staff to achieve this. (Fenech, 1992: 295). 207

208 The 11+ enjoys high status in our educational system indeed. But what impact does it exert on the professional practice of teachers? To be able to assess this, with the cooperation of the Education Division, the views of the teachers of Years 5 and 6 were surveyed through a questionnaire distributed to the schools in all sectors of the system. (See Appendix 3). The responses were processed by Dr G Grima and the following observations and reflections are based on her report as well as on my reading of the free responses. Use has also been made of data from interviews conducted by members of the Ministerial Committee which Dr Grima kindly submitted to me (See Appendix 1). Observations and reflections 1. Both interview and questionnaire data show that both teachers and headteachers hold conflicting views about the 11+ examination. Those who favour the continuation of the examination argue that it helps to maintain standards, enable teachers to follow learning objectives and it is a reliable indication of what students have learned. It is, therefore, providing a direction and rationale for the work of the primary schools. Those who want it abolished, however, refer to the harmful psychological impact it is having on teachers, children and their families. Teachers holding this view consider that it is unsuitable for the low streams and hinders educational opportunities for children with learning difficulties. In some cases, it demotivates teachers too. Any recommendations, therefore, by the Report Committee to abolish or modify the status of the 11+ examination should include provisions to reassure those who want to keep it that the primary schools will not become institutions without direction and a rationale for their work. 2. There is quite a broad agreement, however, on the impact of the 11+ on the teaching and learning process. The evidence is that teachers who are preparing children for the examination find themselves in a dilemma, namely, the masterycoverage one. Teachers who strive to cover what the syllabus prescribes (and because of examination pressure this is what they are tempted to do) do so at the expense of mastery of important concepts and skills. This leads teachers to place more emphasis on rote-learning and continuous revision and working ditto exercises than on the provision of opportunities for the development of understanding of subject matter. 208

209 3. Where the 11+ examination is not kept as an end in view specifically in Church and lay private schools having both primary and secondary sectors the curriculum is enacted in a context where teachers exercise more professional discretion over both curriculum content and pedagogical strategies. The consequence is that, in these instances, curricular work is more enterprising and creative, involving the children in activities like the writing of magazine articles, essays on a wider range of topics, project work and research in knowledge sources beyond the content of prescribed texts. The fact that we find the teacher in this situation saying that if she had the 11+ as an end in view she would lose her motivation to teach is not to be wondered at. The assumption also is that, without this examination as an end in view, the children themselves will be able to learn in an atmosphere devoid of anxiety, stress and insecurity. 4. The 11+ examination is not only having an undesirable impact on the curriculum and pedagogy of the primary schools; it is also determining the way the children are organized for learning, especially in the junior classes. It is leading to a rigid streaming policy based on marks obtained in nationally set examinations where children enter a race for places in the upper streams from the beginning of Year 4. Although commendable steps have been taken in the late 1980s to defer streaming first from Year 2 to Year 3 and then from Year 3 to Year 4, it is a lamentable fact that the last three years of primary schooling are characterized by an atmosphere of competition for the top streams. Unsurprisingly, as on the issue of the abolition or not of the 11+, opinion on this matter, both among the general public and in the teaching profession itself, is conflicting. The survey on streaming, showing the support of close to 90% of respondents, conducted in the late1980s reflects the selection culture which prevails in our system and there is no evidence that there has been a dramatic change of attitude since then. What can be said is that an increasing number of teachers are beginning to question the wisdom of streaming policies, but at the same time are ambivalent in their thinking about teaching mixed ability classes, even though there are so many mixed ability classes in our schools. The belief that there is more teaching efficiency in terms of results obtained in streamed classes is hard to eradicate. 5. It is encouraging, however, to see evidence that there are teachers who are unhappy about the impact the examination is having on their professional practice. They lament the fact that they are forced to teach to the examination with all that this implies, using pedagogical strategies of which they are critical but with which they have to comply. While they point out a degree of professional satisfaction 209

210 derived from the limited opportunities they have to employ inquiry-based pedagogies, they lament the fact that the situation does not allow them wider discretion to exercise their professional judgement. IV. Recommendations and Conclusion 1. Teachers, especially those with an enterprising professional attitude, must not be placed in a situation where they have to teach to an examination syllabus. If this situation continues to prevail, their complaints regarding this matter will be entirely justified. And if the syllabus, as they say, is so excessively vast that subject-matter coverage is at the expense of mastery, then it has to be reviewed to allow them more professional discretion. 2. On the other hand, it cannot be denied that an examination can serve as a compelling device for teachers to teach important curriculum areas or topics and adopt innovative pedagogical strategies like those which are aimed at the development of understanding rather than the mere exercise of memory. And there is some evidence that, through some questions at least, the 11+ is doing this. Of course, this does not mean that the 11+ is the only means of bringing this about. If it is to be removed, however, some other means must be devised to see that desirable curriculum and pedagogical innovations are introduced into the teaching and learning of primary schools. 3. The traditional curriculum areas of language and mathematics should remain at the centre of curriculum practice in the primary schools. Closer attention must also be paid, however, to Science and ICT. It is also extremely commendable for some assessment mechanism to be put in place to check that a satisfactory level of achievement in these areas is reached. This may entail a review of the present curriculum of primary schools in line with the recommendations of the National Minimum Curriculum. 4. It is important that information about the progress in learning of children at the end of their last year in primary school should be obtained both through appropriate tests and continuous assessment by class teachers. This information serves three purposes. First, it enables receiving schools to organize students into learning groups; second, it enables them to plan the students future learning (if continuity and progression in the curriculum experiences of students moving from primary to secondary schools are not to suffer); and, third, it gives an account to 210

211 students and their parents of where their children stand in their learning. It must be added that, besides offering parents an opportunity to make a contribution to their children s development, it also introduces an element of accountability in the work of schools. Receiving secondary schools would also do well to monitor both the transition process and the students learning in the first year and provide feeder primary schools with the necessary information to enable them to improve the assessment of children in Year 6 and minimize any problems of transition. 5. In any future reorganization of the educational system, the institution with the regulation portfolio should have as one of its prerogatives the task of assessing the achievement of pupils in the primary schools and the discretion to take the necessary measures to be able to deal with instances where a satisfactory level of achievement has not been achieved. 6. The structure of school networks which is being introduced (Ministry of Education, Youth and Employment, 2005), can be a commendable way of (i) removing the high stakes attached to the 11+ examination, thereby eliminating the stress and anxiety experienced by children preparing for it; (ii) providing opportunities for more creative curricular work in the primary schools; (iii) dismantling social division; and, as a consequence, (iv) making improvement in the quality of educational provision in schools and classrooms possible. Of course, improvement will not automatically ensue with the removal of obstacles to it. More positive interventions on the part of teachers and administrators at all levels of the system will have to follow. Some of these will entail steps aimed at offering teachers opportunities to develop those understandings and skills which are essential for teaching and learning in a changing culture. 7. Finally, rethinking the place of the 11+ in the system is an exercise which should be conducted within the wider framework of state and private (specifically Church) educational provision in Malta. Leaving two systems, the grammar (Church schools) and comprehensive (State schools) running in parallel has social as well as educational implications, and what happened in the early 1970s should not be forgotten. There are things to be learnt from the past. In conclusion, policymakers should consider the need to develop a policy discourse aimed at introducing into the culture of summative assessment, which has historically prevailed in Malta s educational system, the values of an assessment culture which takes the assessment process to be a means of promoting the 211

212 learning rather than the selection and classification of students. It is understandable that, because of the embeddedness of assessment with high stakes in Malta s educational culture, the move from discourse to practice will take time and cannot be brought about quickly through edict or legislation but through a gradual normative-re-educative process. The shift, however, is worthwhile making: both the educational system and the achievement of students stand to gain if and when this happens. References Alexander, R. (1994) Primary Teaching, (London, Holt, Rinehart and Winston). Black, P. (1998) Testing: Friend or Foe, (London, Falmer) Blyth, W.A.L. (1965) British Primary Education, (London, Routledge) Board of Education, (1931) Report of the Consultative Committee on the Primary School (Hadow Report), London, HMSO. C.A.C.E. (1967) Children and their Primary Schools (Plowden Report), (London, H.M.S.O.) Cameron, J. (1970) Education Planning, Malta (Paris, Unesco). Cox, C.B., Dyson, A.E. eds Goodbye, Mr Short, (London, The Critical Quarterly. Department of Education, (1937) Report on the Workings of Government Departments, (Malta: Government Printing Office). Department of Education, (1948) Report on the Primary Schools (Malta: Government Printing Office) D.E.S. (1981) The School Curriculum, (London, H.M.S.O.) Fenech, J. (1992) Primary Schooling in Malta: An Historical and Ethnographic Study, unpublished PhD dissertation, University of London. Halsey, A. (1972) Educational Priority, Vol.1, EPA Problems and Policies, (London, H.M.S.O.) Halsey, A.H., Gardner, L. Selection for Secondary Education, in British Journal of Sociology, March, Ministry of Education, Youth and Employment, 2005 For All Our Children to Succeed, (Floriana, Ministry of Education, Youth and Employment,.) M.U.T. The Teacher, Vol.1, No. 10, M.U.T. The Teacher, Vol. 8, M.U.T. The Teacher, Vol.9 (3)

213 M.U.T. The Teacher, April-September, M.U.T. The Teacher, 1981 Savona, S. (1865) On the Necessity of Educating the People, (Malta, Government Printing Office). Savona, S. (1883) Report on the Educational Institutions, Malta, Government Printoing Office. Tuppen, C.J.S. (1969) Educational Assessment in Schools, Paris, Unesco, 1970 Vernon, P.E. ed. (1957) Secondary School Selection, (London, Methuen). 213

214 Appendix 3 TEACHER QUESTIONNAIRE FOR THE 11+ EXAMINATION REVIEW Grace Grima and Frank Ventura The teachers questionnaire was created by Dr Joe Fenech in collaboration with the 11+ Review Committee (see final version at end of this document). The aim of the questionnaire was to gauge the impact of the Junior Lyceum and Common Entrance examinations on the teachers curricular and pedagogical practices during the last two years of primary schooling. The questionnaire was sent out from the Curriculum Department of the Education Division to all primary schools in Malta and Gozo, via the internet in mid-october All Heads of Schools were encouraged to invite all Year 5 and Year 6 teachers to complete and return the questionnaire to the Educational Assessment Unit of the Education Division by mid-november. In total, we received questionnaires from 77 schools: 57 state schools, 17 church schools (6 that have a primary school only; 5 that have both primary and secondary and 6 that did not provide this information) and 3 independent schools (1 that has a primary only and 2 that have both primary and secondary sectors). There were 356 respondents out of a cohort of 450 teachers (79.1%): 85 males and 268 females; three respondents did not identify their gender. There were 276 teachers who were targeting the entrance examinations. There were 278 completed questionnaires that were received from the state schools (S) out of 301 (92.4%); 71 from the church schools out of 97 (73.2%), 22 from schools that had a primary school only (C1); 25 from schools that had both primary and secondary schools (C2) and 3 from schools that did not inform us whether they had primary only or primary and secondary (C3) and 7 from independent schools out of 52 (13.7%), 4 from schools which have a primary only (I1) and 3 from schools that have both primary and secondary (I2). Initially the teachers were asked to indicate which examinations were targeted in the school. Their results indicated that 77.2% of the respondents targeted the Junior Lyceum examination and 48.9% targeted the Common Entrance examination. Table 1 presents a breakdown of the data. 214

215 Table 1: Targeted Examination/s at the end of Primary School JL CE Other None No answer S C C C Total Church I I Total Indep Grand Total Table 1 shows that the Junior Lyceum and Common Entrance examinations are the targets of substantial proportions of teachers. It is interesting to note that 59.6% of the teachers in the state schools target both the Junior Lyceum and the Common Entrance examinations. A few (8.5%) also target other examinations for specific church schools. Most of the respondents from the church schools (82.9%) targeted both examinations as do all teachers in the Independent schools that have primary only. As expected, the teachers from the Independent schools that have both primary and secondary schools do not target either of these examinations. Next, the teachers were asked to identify the sources they used to plan their lessons. Their responses showed that all teachers (356) used the syllabus and the textbooks as sources of planning and that 82% (292) also used past papers. Other sources identified were: the internet: 30.9% of respondents (110); workbooks and books: 21.9% (78) and self-made resources: 17.4% (62). We were also interested in finding out what the teachers expected the children to achieve through the work that they assigned to them. Almost all the respondents: 98.6% (351) identified progress in learning as their main objective. The responses for the other categories were problem-solving skills: 324 (91.0%) and exam-tested skills: 80.0% (285). Other skills identified by the teachers included self-esteem, life skills and decision-making. Most teachers 97.2% (346) said that they used cross-curricular tasks which demand the use of skills across subjects. Most of the respondents use these tasks sometimes: 56.5% (201) although there were 34.3% (122) who claimed that they used them often. Few teachers: 6.5% (23) admitted that they rarely use such tasks 215

216 and 6 even said that they never use them. In addition, there were 10 respondents who did not answer this question. When asked to specify the areas in which crosscurricular tasks were used, the largest category of teachers: 26.4% (94) suggested English and Maltese in Religion and Social Studies. There were a few teachers (23) who suggested that they used cross-curricular tasks in all subjects and some others (20) who said Mathematics with English. The issue of homework is directly related to the preparation for these entrance examinations and the impact of the preparation on the pupils life in upper primary. Almost half of the teachers 47.8% (170) and 45.3% (125) of those targeting examinations said that they gave pupils one hour of homework a day and the other half 47.5% (169) and 48.9% of those targeting examinations said that they gave pupils two hours of homework a day. There were few teachers (13) 11 of whom are those preparing pupils for examinations who said that they gave pupils three hours of homework a day. No-one said that they gave them more that this and 4 did not answer this question. This amount of homework relates to school only and does not include the work given as homework during private lessons. We asked the teachers to quantify how often they assigned tasks which required children to undertake research using the following resources. Their responses are tabulated below: Table 2: Tasks that require research Often Sometimes Rarely Never No answer Library Internet Other publications Interviews In three cases (library, internet and other resources), the highest category chosen by teachers was sometimes. There were 60.7% of the respondents who sometimes used tasks that required research in the library and another 57.0% who sometimes used tasks that required research on the internet. There were 40.4% of respondents who sometimes used tasks that require research using other publications. Tasks requiring interviews were less popular and 42.5% of the teachers admitted that they rarely used these tasks. Another 22.0% said that they never use them and a further 22.0% did not provide an answer for this part of the question. The nature of these tasks was varied. The most popular answers were projects and charts: 27.2% (97); 216

217 Social Studies tasks: 25% (89); pictures and information:17.1% (61), and the more general answer topics in the syllabus: 15.7% (56). We were also interested in finding out how often teachers use commercially produced material in assigning work in the different subjects. Their responses are tabulated below: Table 3: Use of commercially produced material in examinable subjects Often Sometimes Rarely Never No answer Mathematics English Maltese Religion Social Studies Once again, sometimes was the largest category chosen for all of the subjects. However, often was another popular choice especially for English, Maltese and Social Studies. When we consider often and sometimes together, we found that 70.2% of all respondents used commercially produced materials for English (71.4% of those targeting examinations), 68.3% for Maltese (71.4% of those targeting examinations), 59.8% for Social Studies (58.0% of those targeting examinations), 48.9% for Religion (48.6% of those targeting examinations) and 45.5% for Mathematics (44.6% of those targeting examinations). In fact, 40.4% of the respondents (42.8% of those targeting examinations) claimed that they rarely or never used commercially produced materials for Mathematics and 37.4% (39.5% of those targeting examinations) claimed that this was the case for Religion as well. The teachers were asked to evaluate the extent to which the 11+ examination was influencing what they emphasised in their teaching. The majority of the respondents: 72.2% (257) [220; 79.7% of those targeting examinations] claimed that the examination influenced them a great deal. A smaller category (76) said not so much and 18 others said not at all. There were 5 respondents who did not answer this question. The teachers in the last three categories did not have the 11+ examinations as a target at the end of Year 6. Therefore, when we consider the respondents who are targeting these examinations we find that they are all influenced greatly by these examinations. Most of the overall respondents consider this influence undesirable: 66.3% (236) [202: 73.2% of those targeting examinations]. There were equal numbers of respondents (60) who said that this influence is desirable or who did not provide an answer to this question. The respondents who decided it was an undesirable influence qualified their answer in 217

218 several ways. The most popular responses were the following: the examination is very important and it goes against the principles of education (105), there is a vast syllabus (66), these examinations are killing motivation and creativity (52) and there is a lot of pressure (50). The teachers were also asked to evaluate the extent to which the 11+ examination was influencing how they teach. Again, the majority of respondents: 66.0% (235) [204: 73.9% of those targeting examinations) said that it influenced them a great deal. A smaller proportion: 25.0% (89) said not so much and 23 said not at all. There were 9 respondents who did not provide an answer to this question. Again these were the respondents who were not targeting the 11+ examinations. The majority of the respondents: 69.9% (249) [214: 77.5% of those targeting examinations] considered this influence as undesirable. Few teachers considered it desirable (40) and 67 did not tick either category. The teachers gave several reasons to qualify their position. The most common responses were the following: vast syllabus no time to adopt creative strategies (69); the main focus is the examination emphasis is on the core subjects (48); there is very limited time to meet the required targets (28); stressful process no time to enjoy learning (25); and no time to develop pupils full potential through other subjects such as the crafts (21). The teachers were also asked to evaluate the extent to which the 11+ examination was facilitating the achievement of the following objectives of the NMC in their teaching. Their responses are tabulated below. Table 4: The achievement of the NMC objectives NMC Objectives A lot Not so Not at all No answer Develop children s potential 66 h Stimulate analytical, critical and creative thinking skills Promote a pedagogy that respects diversity and celebrates difference Encourage the holistic development of children The teachers responses show their awareness of the incongruence between the 11+ examination and the implementation of the NMC. More than half of the respondents suggested that the examination did not do much to stimulate analytical, critical and creative thinking skills (63.5%); to help develop children s potential (61.2%); to encourage the holistic development of children (57.9%) and to promote a pedagogy 218

219 that respects diversity and celebrates difference (52.2%). The teachers qualified their responses in several ways. The most common reasons were the following: the vast syllabus does not allow time to adopt creative strategies and analytic thinking skills (127), the examination is very important and very stressful (82), there is very little time to cover the syllabus (28), and the system makes slow learning pupils feel they are failures (20). Finally, the teachers were invited to add any additional comments on the impact of the 11+examination on their curricular and pedagogical practices. Teachers complained about the lack of resources for English and Maltese, the large number of subjects that are examined and the vastness of the syllabi and the unfairness of not having re-sits. Other teachers hoped that something would be done as there is a lot of stress and pupils from lower streams are already being labelled as failures from a young age. The current system was not allowing for discussions and creative approaches to teaching so pupils learning was reduced to studying for examinations. There were some comments in favour of the current system. These teachers asked to see the examination papers on the examinations day and to have stricter marking schemes. They also suggested the possibility of combining the two sets of examinations (Junior Lyceum and Common Entrance) into one set. The general concluding observation from the analysis of the teachers questionnaires was that the teachers were very aware of the strong influence of the 11+ examination. The current system was having undesirable effects on what they taught and how they taught it. Teachers claimed that the current system is made up of vast syllabi and that time which is limited is mainly focused on the core subjects that form part of the 11+ examination. The system was leaving little or no time for additional skills and profitable activities that were fun and that helped children express themselves in different ways and explored their potential. Furthermore, they also felt the clash between the current selective system and the principles of the NMC. Both cannot co-exist in the same educational system and as long as the selective examination remained, the implementation of the principles of the NMC are not able to be developed and infiltrated in our educational system. 219

220 TEACHER QUESTIONNAIRE FOR THE 11+ EXAMINATION REVIEW Tick ( ) or complete as necessary Gender: Male ( ) Female ( ) Type of School: State ( ) Church ( ) Independent ( ) Years of Teaching Experience in the Primary Your class in Target examinations (external to the school) at the end of Primary: Junior Lyceum ( ) Common Entrance ( ) Other ( ) None ( ) 1. What sources do you use to plan your lessons? (More than one item can be selected) 1. Syllabus Yes ( ) No ( ) 2. Textbooks Yes ( ) No ( ) 3. Past exam papers Yes ( ) No ( ) 4. Other sources (specify) 2. What do you expect the children to achieve through the work you assign them? (More than one item can be selected) 1. Exam-tested skills Yes ( ) No ( ) 2. Problem-solving skills Yes ( ) No ( ) 3. Progress in their learning Yes ( ) No ( ) 4. Other (specify) 3. How often do you assign cross-curricular tasks which demand the use of skills across subjects (e.g. Maths/English in Social Studies/Art; Maltese in Religion/Social Studies, etc.) Often ( ) Sometimes ( ) Rarely ( ) Never ( ) State in which areas: 4. How many hours, on average, do your pupils spend on homework daily: 1 hour ( ) 2 hours ( ) 3 hours ( ) More than 3 hours ( ) 5. How often do you assign tasks which require children to undertake research using : 1. Libraries: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 2. the Internet: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 3. other publications: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 4. Interviews: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) Briefly describe the nature of these tasks: 6. How often do you use commercially produced material in assigning work in: (More than one item can be selected) 220

221 1. Mathematics: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 2. English: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 3. Maltese: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 4. Religion: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 5. Social Studies: Often ( ) Sometimes ( ) Rarely ( ) Never ( ) 7. To what extent do you think that the 11+ examination is influencing what you emphasise in your teaching? A great deal ( ) Not so much ( ) Not at all ( ) Do you consider this influence: Desirable ( ) Undesirable ( ) Explain why 8. To what extent do you think that the 11+ examination is influencing how you teach? A great deal ( ) Not so much ( ) Not at all ( ) Do you consider this influence: Desirable ( ) Undesirable ( ) Explain why 9. To what extent is the 11+ examination facilitating the achievement of the following objectives of the NMC in your teaching? A lot Not so much Not at all Develop children s potential. ( ) ( ) ( ) Stimulate analytical, critical and creative thinking skills. ( ) ( ) ( ) Promote a pedagogy that respects diversity and celebrates difference.. ( ) ( ) ( ) Encourage the holistic development of children ( ) ( ) ( ) 10. State why the 11+ examination is either facilitating or hindering the achievement of the objectives mentioned in Question Add any comments you may wish to make on the impact of the 11+examination on your curricular and pedagogical practices. 221

222 Thank you for your co-operation. Please return the completed questionnaire in a sealed envelope to the Head of School. 222

223 Appendix 4 An Evaluation of the Junior Lyceum and Private Schools Association Entrance Examination Papers English , 2005, and 2006 A report commissioned by the 11+ Examinations Review Committee Dr Doreen Spiteri 9 th January

224 Table of Contents 1 Introduction An examination syllabus The Primary Schools Syllabus 3 Internal validation of the examination Examination paper analysis: Communicative Language Abilities 4 Examination paper analysis- Junior Lyceum Language Use Test Method The response required of candidates Elements of language ability tested The meaningfulness of the test input Topic choice Difficulty level Cultural bias Rubrics, visuals, layout 4.2 Reading Comprehension Test Method and response required Which aspects of the skills are tested The suitability of the test input Topic choice Difficulty level Cultural bias Rubrics, layout, visuals 4.3 Writing Test Method The number of options and response required of the candidates The suitability of the prompt Topic choice Cultural bias Rubric 5 Examination paper analysis Private Schools Association Language Use Test Method The response required of candidates The elements of language tested The meaningfulness of the test input Topic choice Difficulty level Cultural bias Rubric, visuals, layout 5.2 Reading comprehension Test Method and response required Which aspects of the skills are tested The suitability of the test input Topic choice Difficulty level Cultural bias Rubric, layout, visuals 224

225 5.3 Writing Test Method The number of options and response required of the candidate The suitability of the prompt Topic choice Cultural bias Rubric, layout, visuals 6 Recommendations

226 1 Introduction This evaluation is organized in two sections, one for the Junior Lyceum Examination (henceforth JL) and another for the Private Schools Association Examination (henceforth CE). Within each section, the organizing principle is determined by the sections of the examination. A general section precedes both and deals with the syllabus governing the two examinations. The report ends with a summary of the recommendations made throughout. 2 An examination syllabus A pre-requisite for any examination is an examination syllabus that performs a different function to a teaching syllabus, and which delineates in as clear a way as possible the knowledge and skills that candidates are expected to demonstrate. It also specifies: the duration of the examination the sections of the examination paper the various components and their relative weighting the types of materials candidates will be expected to engage with the use or otherwise of authentic texts the test methods and response formats the language elements tested (grammatical items, lexis) the language skills and enabling skills tested a description of what preparation would be appropriate for the examination a description of the view of language underpinning the examination. The Junior Lyceum examination does not have a syllabus but relies on the State Primary Schools syllabus. The CE examination does not have its own syllabus either and it too relies on the state Primary School syllabus. A description of this syllabus will follow. 2.1 The Primary Schools Syllabus Both the 1997 and 2006 syllabi are intended for use in classrooms. The 1997 syllabus includes the following areas - Reading, Usage, Reading Scheme, Poetry, and Project Work. This is followed by sections headed Listening and Speaking, Reading, and Writing. For each of these, suggestions are given for Years 4, 5, and 6. Rather unorthodox is the inclusion of grammar structures under the heading of 226

227 Writing. No specific mention of vocabulary is made. For the Year Six section only, it is recommended that teachers are to follow the set syllabus. While candidates expected writing skills are delineated in some detail, there is no reference to the reading skills candidates are expected to demonstrate. Indeed, the scarce information available deals with some reading strategies and refers to pronunciation issues rather than reading skills. It is not clear how this syllabus can function as the examination syllabus. 1. It has no reference whatsoever to assessment and testing. 2. It includes work on listening and speaking skills that are not assessed during the examination. 3. It has references to poetry and project work that also are not assessed in either Entrance examination. 4. Most of the aspects typically associated with an examination syllabus (v. 2) are missing. It is also worth noting that the Primary Schools Year Six examination tests listening skills which are not tested in the JL Entrance examination. This makes their exclusion from the Entrance Examination even more serious. If all the Primary school children are tested on their listening skills, it should be feasible to replicate the process for the JL examination. It also amounts to sending an ill-advised message to Year Six teachers and students, that listening skills are not so necessary because they are not tested in the high-stakes examination. The 2006 Primary Schools Syllabus is a significant development and improvement on the previous syllabus. Learning outcomes are broken down into small units and the subskills for the four language skills are described in greater detail. However, it remains a teaching syllabus and cannot adequately replace an examination syllabus. There was no evidence that the 2006 examinations were influenced by this new syllabus. The lack of a proper examination syllabus effectively means that there is no document that describes what candidates will be assessed on and how they will be assessed. For example, though the CE too makes use of the State syllabus the component Language Use is weighted at 50%, writing skills at 30% and reading comprehension at 20% (Figure 2.1). Compare this to the JL examination where (in 2005, 2006) 40% of the marks are allotted for language use, 30% for writing skills 227

228 and 30% to reading skills (Figure 2.2). The rationale for the unequal weighting is not provided yet both examinations purport to be based on the same syllabus. Figure 2.1 Weighting of Examination Components, JL Language Use 40% Reading 30% Writing 30% Figure 2.2 Weighting of Examination Components, PSA Language use 50% Reading 20% Writing 30% Another example can be found in the 2004 JL paper where the testing of tenses was omitted from the Language Use part of the paper. This imbalance was redressed in subsequent sessions. The absence of an examination syllabus is a serious lacuna particularly when it is coupled with an absence of test specifications for use by the test writers. These need to know what content is to be covered, what skills to be tested, what methods to be used. Such information is also invaluable for the revising stage or some form of checking process carried out by persons other than the test constructors. The task of drawing up a specification grid should be an a priori exercise against which the examination paper is designed. An examination syllabus should specify the test methods that candidates can be expected to find in the examination paper. This is required even more at this level 228

229 when candidates are young and can be easily thrown by a new type of exercise simply because of its newness. In the absence of such a syllabus, paper setters might be reluctant to vary the question/exercise format and re-use ones that have been used in previous sessions. This state of affairs effectively freezes an examination into a shape that changes very little. And, naturally, it freezes the teaching of the subject. In addition, an effective validation of the test can best be achieved when a syllabus is available which expounds, among other practical aspects mentioned above, the theories of language underlying the test. Teachers coaching students for the examination should have an indication of the vocabulary items or word lists which candidates will be expected to recognise and use. Failing this, examination preparation may result in a hit-and-miss exercise, as teachers cannot be expected to teach all vocabulary areas. In a worse case scenario, this absence of information could result in unreasonable demands being made on students who are pushed into memorizing lists of numerous useless words. An example of the need for a word list is evidenced in the CE 2006 paper in which one of the writing tasks consists of a picture composition showing a couple of children cut off by the tide. Cultural barriers aside, no teacher could have anticipated that candidates would need to use that word. It does not appear in Cambridge s Young Learners English word list, and does not appear among the first 5000 words in Brown s corpus (the Brown Corpus comprises 1,015,945 words with 47,198 unique words. These lists reflect general non-academic English as it is used in newspapers, magazines, and books). 3. Internal validation of the examination In this section, a two-fold analysis of the examinations content will be carried out. The first concerns the communicative language abilities represented in both the examinations and analyses whether these abilities constitute a representative sample of the relevant language skills. The second concerns test methods and refers to test items and test passages and will deal with the JL and CE examinations separately. 229

230 3.1 Examination paper analysis: Communicative Language Abilities The most striking aspect in both the JL and CE examinations is that only two skills are tested: reading and writing. This has serious implications for the washback effect on the teaching and learning that takes place in the sixth year of primary education. Test users are likely to bow to pressure and give priority to the skills on which they are going to be tested. It is not improbable that this will result in a scholastic year in which listening and speaking skills are sidelined to the detriment of the test takers language development. What is certain is that these examinations have given the construct of language ability a shape that is not consonant with the teaching and learning that precedes the examinations and neither with that which follows it. This point being made, the rest of this section looks at how the two Examining Boards have conceptualized their examinations. Table 3.1 shows the marks allocated to the three components of the examination for both entrance examinations. The figures show that while the CE has maintained a fixed weighting year on year for the three components tested, the JL examination has seen fluctuations but appears to have stabilized the marks allotted from 2005 onwards. Table JL CE JL CE JL CE Language Use 34% 50% 40% 50% 40% 50% Reading 36% 20% 30% 20% 30% 20% Writing 30% 30% 30% 30% 30% 30% 100% 100% 100% 100% 100% 100% The most noteworthy difference, however, lies in the different views the two Examining Boards have of language ability and the different interpretations they have given to the same syllabus. In effect, the different weightings given transform the examinations into two different tests: they measure the same abilities differently and they hold a different view of language from each other. Both need rethinking. If the PSA had to write an examination syllabus, their description of the purpose of the examination and their view of language would read differently from that of the JL 230

231 and, what is more striking, very differently from most other examinations of English language for this age group. The scant attention given to the receptive skill of reading does not reflect the position the skill holds in language learning and teaching in neither the Primary school nor the Secondary school for which this examination is a selection tool. Neither does the weighting given reflect the usefulness of the reading skill vis-à-vis the other skills, particularly at this age when the receptive skills should take priority. There are consequences also to the balance of the examination paper in terms of the focus on accuracy. Given that writing skills are also assessed for accuracy and range of grammar, syntax, and lexis, this effectively means that around 80% of the marks on the CE examination are awarded for accuracy. The JL examination reflects a more balanced approach to the relative importance of the assessed skills, however, the question of excessive overall concentration on accuracy prevails here too. 4 Examination paper analysis - Junior Lyceum 4.1 Language Use The questions / test items in this part of the examination are evaluated in terms of: o test methods o the response required of candidates - production or recognition, o what element of language ability is tested (grammatical structures, vocabulary etc), o the meaningfulness of the test input o topic choice o difficulty level o cultural bias o rubric, visuals, layout Language Use Component Test method The range of methods used to test candidates in this part of the examination are mainly multiple choice and selective deletion. Occasionally a matching exercise and a gap-fill exercise are set (v. Table 4.1). The following describes each type of exercise and what the exercise tests. 231

232 Table 4.1 Test method Test type Language element tested A multiple choice B selective deletion gap-fill & 2006 Recognition Production Recognition Vocabulary and grammar, cohesion, reading for meaning 2004 expectancy grammar and vocabulary and ability to carry meaning across sentences vocabulary and grammar 2006 grammar C 2004 matching and Production morphology gap-fill 2005 selective deletion & 2006 Recognition grammar D 2004 gap-fill Production word formation comparatives and superlatives selective deletion & 2006 Production grammar - tenses F 2004 / selective Production Grammar and vocabulary deletion & 2006 Sub-total Exercise A 10 marks The first exercise is common to all three years under review. It consists of a multiple-choice task and consists of a series of sentences, which make up a story. Each sentence has a word or phrase missing and candidates are required to choose one option from four. It is a recognition task and tests knowledge of vocabulary and grammar. Candidates are required to carry meaning across the sentences and for this reason it is a fairly challenging first task. It is advisable to set as first task a less demanding exercise in a candidate-friendly examination. Exercise B 10 marks, 6 marks, 5 marks In 2004 this exercise consisted of a selective deletion task in the form of a conversation between two children. Candidates were expected to fill in the missing words from their repertoire of language. Such tasks have an element of reading ability and are valid, as long as the parts of the sentences deleted are carried out with great attention. 232

233 In 2005 and 2006, this changed to a recognition task rendering it easier. An equal number of gaps and responses were given making the last matches predictable through a process of elimination. This works against the average student inasmuch as candidates cannot mistake one answer only. In such exercises, there should be at least one more response than gaps. Exercise C 10 marks, 4 marks, 5 marks In 2004 this exercise consisted of a matching and a form of gap-fill task. Candidates were shown posters (which would have been more accurately described as notices ) and asked to (a) match the notice to a sentence and (b) form a word from one appearing in the notice. The task tested word formation skills in a meaningful context requiring comprehension skills. The new words formed were adverbs. This is a useful exercise with realistic aspects and should be retained. In 2005 and 2006, the switch to recognition tasks appeared once again and candidates were provided with words to insert in a text from which these words had been selectively deleted. The words were prepositions of place and conjunctions respectively. Once again, the number of gaps matched the number of responses provided. Exercise D 4 marks, 10 marks, 10 marks In 2004 this was a short exercise focussing on the formation of superlatives and comparatives of commonly used words. The exercise is useful, meaningful and candidate-friendly. In 2005 & 2006 this took the form of a text from which the verbs had been deleted. Candidates were provided with the verb and required to conjugate it according to the context. The tenses expected were within the abilities of suitably prepared candidates. Exercise F 0 marks, 10 marks, 10 marks In 2005 and 2006 an additional exercise was included consisting of a text from which words had been selectively deleted and candidates were required to construct the response by providing the missing words. Again a useful, interesting way of testing expectancy grammar and reading skills, but the deleted words must be chosen with great care to make sure that candidates can reasonably be expected to anticipate what is missing. Lying (no.52) was too difficult for the candidates. 233

234 4.1.2 Language Use Component - The response required of candidates The Examiners Report states that half a mark is to be deducted overall in each exercise where candidates are required to copy the response. Contrast this with the policy of marking a constructed response as wrong if it is misspelled. It would be fairer if the practice were reversed: more marks should be deducted if candidates copy wrongly rather than when candidates construct the response from their own bank of language. A trend seems to have developed regarding the Language Use section with respect to whether candidates are required to choose from provided responses or construct their own. Three-quarters of the marks on this section (30 of 40 marks) of the examination are achieved on the basis of the candidates producing the correct response from their internal language ability. This has the effect of rendering the examination somewhat more difficult and creates issues for marking. If this section of the examination is accuracy based, and if 75% of the responses are candidate-constructed, the more likely it is that candidates will lose marks. When other parts of the examination are taken into consideration, this focus on accuracy has implications for the construct the examination purports to test and for the purpose of the examination. Clearly, the greater the number of production items, the more the examination will discriminate among candidates. However, at this level, it would be advisable to increase the percentage of recognition items in this part of the examination to 40%. Table 4.2 shows the balance between recognition and production tasks over three years. Table 4.2 Analysis of Language Use Component Recognition Production Mark % Mark % Language Use Component - Elements of language ability tested There is a variety in the elements of language tested in that candidates are tested for their ability to cope with English beyond the word and sentence level and this is 234

235 commendable. However, there does not appear to be a systematic coverage of the kinds of vocabulary and grammar tested, they seem to be random. For example, punctuation has not been tested for three years in a row; neither has question formation. The availability of a specification grid which lists all the elements of language that the examination can test would stop this from happening Language Use Component - The meaningfulness of the test input All the test items in the Language Use component are presented in a meaningful chunk of text, which creates a mini-story. Language, therefore, is presented in a realistic, contextualized way which helps learners achieve meaning. This approach seems to have been firmly established and should be continued Language Use component Topic choice The topics reflected in the test input are all appropriate and suitable to the candidates age. They generally involve children and themes related to children Language Use component - Difficulty level There are instances when the language used is above the candidates expected level. It is unreasonable to expect candidates to know the meaning of the verb to found (2004 Ex A, no.8) which ranks 4,814 on Brown s corpus. Words such as unsuitable do not appear in the first 5000 words (2005 Ex A, no. 8). The Examiners Report (2004) expresses surprise that candidates had problems with words such as irritate, annoy and upset. However, these three words do not feature among the 5000 most frequently used words. Care must be taken not to expect candidates to produce language to which they have not been exposed. The Examiners Report for 2006 comments that in exercise D candidates could not construct the present perfect. The present perfect is an awkward tense generally for Maltese learners of English; in fact it does not feature among the tenses stipulated in the 1997 syllabus. The 2006 syllabus does not list specific tenses. Attention should also be paid to the difficulty level of texts chosen for selective deletion tasks. The Space Shuttle text for instance (2005 Ex C) is classified as suitable for an 11-year-old native speaker of English. The fact that the candidates have to process a challenging text in order to fill in the conjunctions makes the task more difficult than it needs to be and confuses the issue; if the candidate mistakes 235

236 the conjunction is this because she does not know its function or is it because the surrounding text was too difficult? Language Use section Cultural bias There were no instances of cultural references in the Language Use component section Language Use section Rubrics, visuals, layout The rubrics are generally clear and useful, however, the word suitable should be replaced with best and complete might be simplified to finish. As a rule, examples are included and this is good practice. The use of metalanguage is avoided, though this is listed in the syllabus. There is no harm in using metalanguage such as noun, verb, adverb and adjective as long as this does not interfere with allowing the candidates to show what they can do with language. The use of visuals even in this section of the examination is commendable and should be continued. 4.2 Reading comprehension The questions / test items in this part of the examination are evaluated in terms of: o o o o o o o o test methods, the response required of candidates, the subskills of reading ability tested the suitability of the test input topic choice difficulty level cultural bias rubrics, layout, visuals Reading comprehension component Test Methods and response required 2004 The question formats used are varied - True/False, Multiple-Choice, Short Answer Questions (henceforth SAQs) - but there is a predominance of SAQs requiring candidates to answer in complete sentences. 236

237 Table Test method Test type Reading skill tested marks marks marks True / False Objective vocabulary at word level, Multiple choice Objective vocabulary at sentence Short Answer Questions Quote from text level, Subjective reference devices, inferencing, Objective scanning for specific information / / 5 SAQs allow the tester to glean valid information on candidates reading skills, which might not be achieved on multiple-choice items if there is not a sufficient number of them. Such constructed responses mean that the candidate takes on more responsibility for the answer and more demands are made on candidates writing abilities. This interference blurs the measurement, and the measurement is further complicated when candidates are required to answer in complete sentences as is the case in this examination. In principle, the testing of a skill should not be contaminated with the inclusion of other skills and though this is difficult to attain when testing the receptive skills of listening and reading, the solution generally lies in: awarding marks if candidate shows comprehension, requiring test takers to write as little as possible, ignoring grammatical accuracy if they do have to write, or requiring answers in the first language. In the case of this examination, the practice of penalising incomplete sentences and syntactical errors (Examiners Report 2006) should be abandoned immediately. The predominance of SAQs has implication for marking procedures as it requires substantial standardization among markers. Having said this, the method should be retained for reasons mentioned above, but the marking policy should be rethought and 2006 The observations made on the 2004 examination with regard to test methods hold for the 2005 and 2006 papers; however, the 2006 paper sees an increase in the number of objectively marked items with the inclusion of questions requiring candidates to quote from the text. This is a step in the right direction not only because it adds more objectivity to the marking process, but it allows the testing of different reading skills. 237

238 The number of marks allotted to each question are quite high: some questions are allotted 4 marks on the basis of requiring a two-part answer, however, several others are allotted 3 marks when one piece of information was required. This approach does not necessarily favour candidates, indeed it tends to be an unfair strategy. If a candidate gets a wrong answer, s/he looses several marks. If, on the other hand, there were more comprehension questions pitched at the right level, carrying fewer marks, the risks are smaller and the possibility of building up marks greater Reading comprehension component - Which aspects of the skills are tested The reading skills tested are mainly at word and sentence level and these include the ability to infer and to understand referring expressions. These are appropriately tested; however, the last two reading comprehension components saw a reduction in the number of questions requiring candidates to infer meaning and to understand referring expressions and an increase in questions requiring candidates to scan for specific information. The former requires higher order reading skills and this move makes the comprehension task easier. The downside to this, however, is that the range of reading subskills tested is shrinking. The understanding of meaning at word and sentence level occurs frequently; however, the method of testing specific vocabulary (Questions 36 and 38 in 2004; 48 in 2005) is unsound. Candidates are required to give the meaning of four vocabulary items whose meaning cannot be deduced from the context or through morphology. This renders the task as a pure vocabulary test and not a test of reading skills. Moreover, requiring candidates to define the meaning of vocabulary items in their own words is a most unusual task in tests of reading. Moreover, judgement has to be used when choosing which vocabulary to test. Question 38 in 2004, requires candidates to show they have understood the sentence He darted at the biggest crumb with a lightning peck by paraphrasing it. The choice of sentence, loaded as it is with infrequently used words, does not warrant testing. Similarly, checking that candidates can locate the synonym musty for stale and figures for persons in a drawing (35, 2006) can be replaced with higher-frequency words and phrases found in the text. 238

239 Equal care should be given to clarity of purpose when setting questions; a case in point is question 42; it is not clear whether this tests reading skills or writing skills. It is ostensibly a reading task as it forms part of the reading comprehension component, however, the marking scheme does not list the things that candidates should mention in their answer, but guides markers to grade the writing as excellent, competent, or weak indicating that more than comprehension skills are required. This amounts to another instance of an inadequately defined construct which confuses the balance of skills in the paper. The marks allotted to this question are also questionable; to allot 12 marks (out of 36) to this one task is excessive and effectively means that it has taken the place of the testing of other reading skills Reading comprehension component - The suitability of the test input The sources used for the testing of reading comprehension are invariably extracts from children s literature. This repeated use of the same text type effectively translates into a narrowing of the syllabus. The message communicated to teachers by the Examining Board is to always expect a narrative with its inherent characteristics of chronology and past tenses. The range of texts that learners do and should come across as indicated in the syllabus - should be reflected in the examination and alternative non-literary sources should also be used Reading comprehension component - Topic choice The topic of the texts for the years under review were all appropriate in that they all involved children or a child. The themes perhaps could be more contemporary and more realistic than for example a story about a sea monster snatching a girl s model boat (2005) or children stumbling upon a cave covered in pre-historic drawings (2006), but this is not a major issue Reading comprehension component - Difficulty level The 2004 reading comprehension text is pitched at a higher level than that which Maltese 11-year-olds can be expected to have. Readability statistics describe it as requiring 8 years of education (Gunning Fox Index) to understand it and suitable for a native speaker American fifth grader (age 10 to 11). Most of the words do not 239

240 appear in a list of the 5000 most frequently used words, and it is not surprising that candidates had problems giving the meaning of accustomed to, and could not understand perch. (Examiners Report 2004). The 2005 text is also a challenging read: it scores 7.72 on the Gunning Fox Index meaning it requires as many years of education to read and understand it, and is suitable for a native speaker American in Grade 5. This text too has a fair amount of difficult words for second language learners at this level of education and once again, it is these words which are selected for candidates to define. Phrases like pitch and rock do not warrant testing. In 2006 the level of difficulty was slightly lower: statistics show that the text is within the grasp of learners with 6.45 years of education and a native speaker fourth grader would understand the text. Still, it appears that all the texts were written for native speakers and not for second language learners. This thinking runs counter to that underpinning the choice of reading schemes for primary school learners. There, the policy is for reading material that has been carefully graded to achieve a balance between known and new language. The same thinking should direct the choice of comprehension passage. These should be chosen from graded readers or material that has been simplified for second language young learners Reading comprehension component - Cultural bias The reading comprehension text used in 2004 relies considerably on the reader s understanding of the word attic. Although there are visual and lexical clues, these might not be sufficient to bridge the cultural divide. Research has shown that lowlevel learners cannot decode such symbols and depend on bottom-up analysis to reach an understanding of the text. If there is any possibility that a major feature in a text could pose problems of cultural understanding and could stand in the way of linguistic understanding, this should be discarded Reading comprehension component - Rubrics, layout, visuals The questions were clear. The layout was helped by dividing the text into parts or paragraphs. The visuals were useful and attractive. 240

241 4.3 Writing The questions / test items in this part of the examination are evaluated in terms of: test methods the number of options and response required of candidates the subskills of writing ability tested the suitability of the prompt topic choice cultural bias rubrics, layout, visuals The necessity of an examination syllabus and a specification grid is particularly evident in this part of the examination for one of the years under review. In 2004, 42 marks were in effect allotted to the writing skills through the setting of a question, which, while testing reading comprehension placed considerable emphasis on written accuracy. This casual attitude of changing the physiognomy of an examination should not take place. Increasing the weighting of one component of the examination by 12% to the detriment of another component amounts to: a disregard for the balance of skills (such as there was) a lack of consideration towards the test takers and test users who were not expecting this unannounced change difficulties in comparing performance on the examination over the years In the following two years, 2005 and 2006, the situation seems to have stabilized and the weighting given to writing skills maintained at 30% Writing skills component Test methods The methods used are generally short written prompts or a visual prompt Writing skills component - The number of options and response required of the candidate Candidates are offered five writing prompts. Offering such a wide choice might appear good practice; in fact, it gives rise to problems of reliability in marking which ultimately is not in the candidates interest and also it has a negative washback effect. The problems of reliability arise because of the different demands different types of writing make on the candidates. A narrative type piece of writing calls for an adept handling of tenses while a descriptive writing task requires a wide range of 241

242 vocabulary. Candidates therefore are not being tested on the same skills because the demands put on them are different. Offering five writing tasks and setting a pattern of narrative and descriptive writing skills translates into a very predictable examination and those preparing for the examination can safely hone their narrative skills knowing that there will definitely be one prompt out of five that tests these skills. This results in a narrowing of the curriculum and is pedagogically undesirable. For greater reliability in marking and a positive washback effect, the prompts should be reduced in number and the subskills testing in each should be similar. Candidates are required to write not less than 150 words. Given that marks are forfeited for writing that falls short of this, candidates should be warned against committing this error. The mark scheme makes no reference to an upper word limit for consistency this should be set. The length of writing expected of candidates taking the JL entrance examination and of Form 1 students in Junior Lyceums in the Above Average band is both 150 words. It would make for greater validity and reliability if candidates were required to produce two shorter pieces of writing such as a letter or a short account and a message or a note totally 150 words between them. The syllabus lists these as learning outcomes Writing skills component - The suitability of the prompt All the fifteen writing tasks in the three sessions under consideration tested descriptive or narrative writing skills. It appears that a pattern has taken shape within the five writing prompts: each examination paper offers four narrative and/or descriptive titles and one letter-writing task. Among the fifteen writing prompts, there was only one recognisable text type - the letter-writing stimulus. The others simply prompt candidates to produce a piece of writing exemplifying descriptive or narrative writing skills. However, these are not representative of the types of writing tasks that candidates should be able to perform and indeed the syllabus prepares learners for wider writing skills including how to write brief reports on various situations such as a recent event, or a TV 242

243 programme, a book, a poem, a news item, a personal diary. Varying the text types will make for greater construct validity as the writing skill will be more fully represented over the years. Furthermore, offering writing tasks that approximate towards real-life use of language will bring the examination closer to the next logical stage secondary schooling where the teaching of English builds on the communicative approach intimated in the primary schools syllabus Writing skills component - Topic choice Candidates should have no problem relating to and identifying with the topics of the writing tasks; the background knowledge required is within their grasp Writing skills component - Cultural bias There is only one instance of cultural bias in the writing tasks for the three years under review. This occurred in the picture compositions, which tend to suffer somewhat when they cross borders. The pictures showing a large garden complete with wooden shed and nest box up in an old tree (2005) depict a way of life that is unfamiliar to most Maltese learners. Supplying the candidates with words such as shed, does not necessarily mean that they will know which part of the picture the word refers to. Generally, such pictures should be avoided and replaced with locally commissioned ones portraying characters and situations the average 11-year-old can relate to Writing skills component - Rubric The rubric is in certain instances rather brief and this could lead to problems of reliability in the marking stage. Writing prompts such as A Shining Old Coin (2005) can be interpreted in any way the candidate fancies, as can The Old Tree in the Valley (2004) and A rainy day (2006). It is this variety of responses which poses problems of standardization in marking. Moreover, a degree of guidance at this level would not be out of place; it is after all a strategy adopted at SEC level. Some suggestions would help both the candidate and the marker. 5 Examination paper analysis Private Schools Association 5.1 Language Use The questions / test items in this part of the examination are evaluated in terms of: 243

244 o o o o o o o o test method, the response required of candidates - production or recognition the elements of language ability tested the meaningfulness of the test input topic choice difficulty level cultural bias rubric, visuals, layout Language Use component Test methods A range of methods are used to test candidates in this part of the examination (see Table 5.1). The following describes each type of exercise and what the exercise tests. Ex. 1, 3. 4 and 6. These exercises cover vocabulary and are all matching exercises. Candidates are required to match collocations, antonyms, or synonyms. They are all at word level and no attempt is made to contextualize the vocabulary. Ex 2 This is not quite a gap-fill exercise as indicated in the table, however, it essentially consists of candidates producing a vocabulary item from the lexical domain of families which is defined in the test input. For example: Peter s mother s brother s son cousin. This is more of a conundrum than a vocabulary exercise and having it as the first task in the examination paper could not have helped candidates. The task could have been rendered less confusing if Peter were replaced with His so that the question reads as follows: How are the following related to Peter? His mother s mother.. Ex. 5 This is a multiple-choice exercise consisting of sentences with missing words, which candidates fill in from four responses. It tests a variety of vocabulary and grammar items. The sentences are not linked to each other. Ex 7 and 8 are both gap-fill exercises however, in 7 candidates are to construct a response by supplying the synonym. In Ex 8 candidates fill the gaps by choosing from a list of prepositions. In both instances, the exercise consists of separate sentences. Ex 9 appeared in the three sessions of the examination and consists of some form of prompt to which candidates are required to form questions. It is a challenging task requiring candidates to think backwards, as it were. In certain instances, an 11-year-old might not have the thinking skills required to do this especially if the 244

245 context given is rather slim and does not contain a verb that candidates can focus on and build a question round it. For example writing a question to which: Student: I m sorry sir. I won t do it again was the answer could be unnecessarily perplexing for a learner and the problem might not stem from linguistic ability. It is also an open-ended task to quite an extent as some questions can take various forms. For example, what sort of question are candidates expected to write if the response is: He is a doctor who words in a private hospital? Is it sufficient for a candidate to write: Who is he? Or are candidates expected to write: Who is he and where does he work? This has implications for marking and in the absence of a published mark scheme, it is not known whether the purpose of this exercise is to test syntax and neither is it known what weighting, if any, is given to punctuation and spelling. Ex 10 and 11 are both selective deletion tasks requiring candidates to construct their own response. The first tests knowledge of vocabulary and grammar while the second focuses solely on tenses. Ex 12 is a sentence completion exercise; candidates read a sentence which they are required to paraphrase using the prompts supplied. This exercise tests comparatives and superlatives. The example given, however, does not illustrate what the candidates are expected to do. Ex 13 tests candidates knowledge of vocabulary adjectives and consists of stand-alone sentences which candidates read and show they have understood by choosing the most suitable adjective to match the sentence. Ex 14 requires candidates to complete a compound word for which one part is provided. Ex 15 tests candidates word formation skills. Candidates are given words in a particular form and required to form new words from them. The transformation is often from noun to adjective. Ex 16 is a punctuation exercise consisting of five sentences to which candidates have to add the appropriate punctuation marks. Ex 17 tests candidates range of vocabulary by requiring them to write down words associated with various places. 245

246 Table 5.1 Test method Test type Language element tested 1. matching Recognition vocabulary collocations 5 / 2005, A 2. gap-fill Production vocabulary lexical set , A 3. matching Recognition vocabulary - antonyms 10 / / 2004, B 4. matching Recognition vocabulary synonyms / 5 / 2005, B 5. multiple choice Recognition vocabulary and grammar C, 2005 D, 2006 D 6. matching Recognition vocabulary - collocation 10 / / 2005, A 7. gap-fill Production vocabulary - synonyms 10 / / 2004, E 8. gap-fill Recognition prepositions / 2004 F, 2005 G 9. SAQ Production question formation G, 2005 E, 2006 C 10. selective deletion Production vocabulary and grammar / 2004, H 2005 F 11. selective deletion Production grammar tenses I, 2005 H, 2006 G 12. sentence completion Production sentence formation / 10 / 2005, G 13. gap-fill Recognition vocabulary adjectives / / , B 14. gap-fill Production vocabulary word / / , E formation, compound words 15. gap-fill Production vocabulary word / / , F formation 16. SAQ Production punctuation / / , H 17. grid-filling Production vocabulary lexical sets / / , A Language Use component The response required of candidates The trend in the CE examination papers is towards an increased number of tasks requiring candidates to construct a response rather than recognize the correct response from a number of alternatives. The production tasks in the Language Use component have increased from 50% to 75%. In the absence of an Examiners Report one can only speculate about the reason for this trend. It might be due to a policy decision to make the examination more challenging; it might be a bid to increase the differentiating power of the examination

247 Table 5.2 Analysis of Language Use Component Recognition Production Mark % Mark % What is also unknown, in the absence of a marking scheme, is the degree of accuracy expected on the production items. Number of responses provided. In the following exercises candidates were provided with an equal number of responses as were required: 2004, B,D,F; 2005 A; 2006 B. Good practice would provide more responses than are necessary to complete the task for reasons explained in Language Use component - The elements of language tested There is an appropriate range of language elements tested in the examination. The inclusion of sentence formation and question formation, and the sociolinguistic appropriacy embedded in these, collocations, punctuation and compound words make for content validity over the years. However, the adherence to word and sentence level exercises has limitations on the testing of higher order language abilities such as the ability to carry meaning from one sentence to the next. It does not permit, for example, the inclusion of exercises where candidates choose the best alternatives (from multiple choice) based on their understanding of how sentences are linked to each other and how the meaning of one influences the meaning of the other Language Use component The meaningfulness of the test input This is the weakest aspect of this component of the examination; seventy percent of the test input (85% in 2005) is at sentence level and consists of de-contextualized sentences which bear no meaningful relationship to the preceding and following sentences. Candidates trawl from one sentence to the next, creating meaning out of each only to abandon the process and start afresh with each new sentence. People are introduced in one sentence, The thief stole my father s mobile phone and my brother ran after him only to be replaced by another set of people: After the party Paul and Kim went back home on foot in the next sentence. This approach is a throwback to less enlightened times in language teaching and testing and should be 247

248 rethought. Sentences should be replaced with short texts which create a mini context and which are coherent. This will not only make for more sensible and realistic examination input, but will allow for the testing of a wider range of language elements such as reference devices and discourse markers. In those exercises where the test input is a meaningful paragraph (H, I, 2004 and 2005; F, G, 2006) the input is suitable for the age of the candidates and works well Language Use component Topic choice The topics reflected in the test input are all appropriate and suitable for the candidates age. They generally involve children or themes that children can relate to Language Use component Difficulty level Requiring candidates to work with vocabulary items which are presented out of context should be avoided. De-contextualized words are not a feature of real language use and raise difficulty levels unnecessarily. Moreover, the choice of certain words seems groundless and based on personal impressions of frequency; words such as thrilled, vanish, outgoing, worthless, miserable, shocked, furious, located, insignificant, deserted, situated, haunted, sore, steep, pebbly, icing, sites, lotion, shallow etc. do not appear among the first 5,000 words in Brown s corpus (v. 2.1 above). One should also look for authentic, contemporary material; the text for a selective deletion task (G, 2006) appears to have been written by a second language user of language and contains outmoded words such as whilst and unidiomatic words such as pursued when ran after would have been simpler and more natural Language Use component Cultural bias There were no elements of cultural bias in this component Language Use component Rubric, visuals, layout There are no pictures at all in this section of the examination paper for any of the years under review. Considering the age of the candidates, some pictures would help to relieve some of the strain. 5.2 Reading comprehension component The questions / test items in this part of the examination are evaluated in terms of: o test methods and response required of candidates 248

249 o o o o o o the subskills of reading ability tested the suitability of the test input topic choice difficulty level cultural bias rubrics, layout, visuals Reading comprehension component Test Methods and response required The question formats used over the three years are varied (True/False, Multiple- Choice, SAQs, Quote from the text) and in the last two sessions of the examination the balance tilts towards objective type questions. This balance should be retained. Response required. Candidates are not told to write in complete sentences, however, in the absence of a published mark scheme, the line adopted by the markers is not known. It is hoped that the policy is to ignore spelling, punctuation and syntactical mistakes and look for signs of comprehension and reward it. If this is not the policy, then the observations made in connection with the JL (4.1.2) examination paper are relevant here. Table Test method Test type Reading skill tested marks marks marks True / False Objective vocabulary at word level, Multiple choice Objective vocabulary at sentence 1 / / Short Answer Questions Quote from text level, Subjective reference devices, inferencing, scanning for specific Objective information reading for gist There is a variety of responses required of the candidates such as to explain in their own words, make a list, write facts, and quote from the text. The practice of requiring candidates to give the meaning of words (Q 13, 2004) should not be continued unless the meaning of the words can be worked out from the context. In such cases, it amounts to a reading skill and is therefore valid. The question in which candidates are provided with meanings and asked to find the corresponding word from the text (Q 12, 2004) is valid but candidates should be directed to which part of the text the words are found in. 249

250 5.2.2 Reading comprehension component - Which aspects of the skills are tested The reading skills tested are mainly at word and sentence level and only one question sought to test candidates understanding of the main thrust of the text (Q14, 2004). In the years under review, there were no questions that tested whether candidates could identify the type of text they were reading. Yet this is a skill listed in the syllabus. The ability to understand referring expressions is appropriately tested and is a consistent feature year on year. However, the ability to infer meaning is undertested, as most of the questions require candidates to locate specific information from the text. The former requires higher order reading skills and including such questions not only widens the range of reading subskills tested in the examination but makes for better discriminating items. Moreover, clarity of purpose has to be maintained. This component of the examination tests reading skills and to insert a question asking candidates to locate three examples of similes (Q 3, 2006) is unexpected in a language examination. It is not a valid question in this test and to include it amounts to a lack of consideration for the test takers and test users who could not have anticipated such a question. Judgement has to be used when choosing which vocabulary to test. Question 1, 2005, requires candidates to define the phrase ground-up beef. The minimal usefulness of this phrase to language learners should not warrant testing. Similarly, checking that candidates can locate the word merchant for people who buy and sell goods (5 v 2005) can be replaced with higher-frequency words and phrases found in the text Reading comprehension component The suitability of the test input Two of the three texts used for the testing of reading comprehension come from literature and one is a non-fictional magazine article. It is good practice to vary the text type to represent better the range of reading that learners come across as suggested in the syllabus. Greater construct validity is also achieved over the various examination sessions. Some more consistency in length of texts is required; the 2006 text is twice as long as the 2004 one. This may be due to the fact that there is no examination syllabus which sets down the length. 250

251 5.2.4 Reading comprehension component Topic choice The influence of topic choice of the reading comprehension text on candidates is considerable, particularly at the age of the candidates that the examination targets. The 2004 text is taken from literature and deals with an unstable, depressed doctor who knocks on the door of another doctor whom he has never met in his life and with whom he engages in a life-changing discussion. This topic is unsuitable on several grounds: it is far removed from children s lives, the characters are adults, the mental state of one is suspicious, they talk about career moves, and the message given out to candidates is that it is perfectly all right to open your door to a complete stranger and give him advice about his life. The topic of the 2005 comprehension text is also questionable on several levels: it exalts hamburgers at a time when on a national level there is a drive towards healthy eating, and more worrying still, the text traces the international success of a popular fast-food chain in glowing terms as if the establishment was the epicentre of haute cuisine. This is not ethical and such texts should be discarded. The 2006 comprehension text deals with a wife who has been given a puppy as a present from her husband. Candidates can be expected to relate to this Reading comprehension component Difficulty level The 2004 reading comprehension text is pitched at a higher level than that which Maltese 11-year-olds can be expected to have. Readability statistics describe it as requiring years of education (Gunning Fox Index) to understand it and is suitable for a native speaker American sixth grader Flesch Reading Grade level. Most of the words do not appear in a list of the 5000 most frequently used words, such as faltering, desperately, profession, injected, agitated, state of mind, urgent. The 2005 text is also a challenging read: it scores 9.82 on the Gunning Fox Index meaning it requires as many years of education to read and understand it, and is suitable for a native speaker American in Grade 10. In 2006 the level of difficulty was only slightly lower: statistics show that the text is within the grasp of learners with 8.26 years of education and a native speaker fifth grader would understand the text. The source of the text is not acknowledged, but it appears that all the texts were written for native speakers and not for second language learners. The third text may 251

252 have been written by someone for whom English is a second language. The reservations expressed in are pertinent here also Reading comprehension component Cultural bias There is one instance of cultural bias in the reading comprehension component. The text (2004) refers to Spain as being a sunny place, different from here. Clearly, a mature reader would understand that here must be a country other than Malta, and probably a cold country too. An 11-year-old might not be able to make that leap (particularly after having read about manic depressives) Reading comprehension component Rubric, layout, visuals The rubrics are clear and useful. They guide the candidate into giving the right amount of information, for example, Write down four facts about... The layout is neat. The addition of pictures would help anchor candidates attention and serve not only to help comprehension but also to lighten the text load on the page. 5.3 Writing The questions / test items in this part of the examination are evaluated in terms of: o o o o o o o test methods the number of options and type of response required of candidates the subskills of writing ability is tested the suitability of the prompt topic choice cultural bias rubrics, layout, visuals Writing skills component Test Method The methods used are generally short written prompts and visual prompts Writing skills component - The number of options and response required of the candidate The evaluation here is identical to the one for the JL examination and is being reproduced for ease of reading. Candidates are offered five writing prompts. Offering such a wide choice might appear good practice; in fact, it gives rise to problems of reliability in marking which ultimately is not in the candidates interest and also it has a negative washback effect. The problems of reliability arise because of the different demands different types of writing make on the candidates. A narrative type piece of writing calls for 252

253 an adept handling of tenses while a descriptive writing task requires a wide range of vocabulary. Candidates therefore are not being tested on the same skills because the demands put on them are different (v ). Offering five writing tasks and setting a pattern of narrative and descriptive writing skills translates into a very predictable examination and those preparing for the examination can safely hone their narrative skills knowing that there will definitely be one prompt out of five that tests these skills. This results in a narrowing of the curriculum and is pedagogically undesirable. For greater reliability in marking and a positive washback effect, the prompts should be reduced in number and the subskills testing in each should be similar Writing skills component - The suitability of the prompt All the fifteen writing tasks in the three sessions under consideration tested descriptive or narrative writing skills. It appears that a pattern has taken shape within the five writing prompts: each examination paper offers four narrative and/or descriptive titles and one letter-writing task. Among the fifteen writing prompts, there were only two recognisable text types - a letter and a school magazine article. The others simply prompt candidates to produce a piece of writing exemplifying descriptive or narrative writing skills. However, these are not representative of the types of writing tasks that candidates should be able to perform. and indeed the syllabus prepares learners for wider writing skills including how to write brief reports on various situations such as a recent event, or a TV programme, a book, a poem, a news item, a personal diary. Varying the text types will make for greater construct validity as the writing skill will be more fully represented over the years Writing skills component - Topic choice Candidates should have no problem identifying with the writing tasks set in the three sessions under review; the background knowledge required is within their capabilities. The one task which does not fit this description requires candidates to write about an attack from outer space; but in such a task, one should think, virtually any description is accepted Writing skills component - Cultural bias When examining linguistic ability it is imperative to achieve validity; one way to achieve this concerns checking the examination for cultural bias and background 253

254 knowledge. Candidates should not be prevented from demonstrating what they can do with the language because of cultural issues. The syllabus makes no mention of gradually introducing learners to cultural knowledge and in language testing it should not interfere with candidates performance. For this reason, test input should be screened for cultural references which might appear harmless and incidental but which might have a conscious or unconscious effect on candidates and prevent them from showing what they know and can do. The picture composition (2006) relies entirely on background knowledge that many 11-year-olds cannot reasonably be expected to have. The pictures show two boys looking for a spot along the coast to play football in. They find a tiny cove and after a while get cut off by the tide which blocks their return path. They light a fire and are rescued by a passing fisherman. There are two possible problems here: first, a candidate attempts this writing task, misinterprets the chain of events and writes a piece that does not fit the pictures. The second concerns candidates who spend time puzzling over the pictures because they would like to carry out this particular writing task, they realize they cannot make sense of it and give up, wasting time and possibly feeling defeated in the process. There might even be candidates who understand the concept of tide but do not know the word for it. Markers would need to standardize their judgement when marking such writing tasks Writing skills component Rubric, layout, visuals The rubrics for all the writing tasks are clear and contain additional information which guide the candidates. This makes the task more candidate-friendly and has the added benefit of achieving a greater degree of reliability in marking. 6 Summary of recommendations It is assumed that aspects of good practice present in these examination papers and which I have remarked upon in this report will be retained. Throughout the report several recommendations are made, however, for easy reference I am grouping them here as follows. 254

255 6.1 Recommendations for both entrance examinations i. The rationale, if any, for the existence of two parallel examinations based on the same syllabus, should be stated. There is significant overlap between the two examinations. ii. The feasibility of combining the two into one examination should be studied and current resources used better for a more valid representation of skills tested, i.e. the inclusion of the testing of speaking and listening skills. iii. The reasons for excluding both the listening and speaking skills should be known. This has been a major policy decision with far-reaching effects on the teaching and learning of English in Malta, and warrants an explanation. Administrative difficulties should not override the importance of testing these skills as their exclusion has a negative impact on the teaching of English in primary school. iv. The drawing up of an examination syllabus must take top priority (v.2, p.5 for contents of such a syllabus) so that the examinations are driven by clarity of purpose at all levels. v. The use of frequency lists and readability statistics (applied to second language learning) should be a constant guide in paper setting for all parts of the examination paper. Failing this, the level of difficulty of the examination will become an exercise in approximation. vi. In marking reading comprehension skills, the penalization of errors in syntax, grammar, and spelling should be rethought and brought in line with accepted practice. vii. The sources for the reading comprehension passages should be better representative of the kinds of reading in which 11-year-olds can engage. Literary sources should not predominate. viii. The weighting allocated to recognition and production type exercises in the Language Use section should be revised in view of the skills that are assessed in other components of the examination and also in view of the age of the candidates. 255

256 ix. The methods for testing vocabulary should be revised so that vocabulary items are not presented in isolation and candidates required to define their meaning. x. Clarity of purpose should be maintained so that testing vocabulary is not confused with testing reading skills. xi. A wider range of reading subskills should be tested. The predominance of referential questions and vocabulary checking items are overrepresented to the detriment of other reading subskills. xii. The writing prompts (composition titles) options should be reduced in number; there should not be a predictable pattern of narrative and descriptive skills in every examination session; they should test a wider range of writing skills; and they should include more text types than letter-writing. xiii. Greater validity and reliability will be achieved if candidates are tested on two writing tasks and not one, which together make up the 150-word length as stipulated in the syllabus. xiv. Marking procedures should be more transparent; if, for example, in the marking of subjective parts of the examination paper each script is marked by three different people, it should be known by the public as this will increase public faith in the reliability and fairness of the marking procedure. 6.2 Recommendations CE Examination xv. It is imperative that an Examiners Report is published. xvi. It is imperative that a marking scheme is published. This is indispensable for test users to know what the markers expectations and policies are. xvii. The topics of the reading comprehension texts should always be relevant to candidates age and maturity levels. 256

257 xviii. The length of the reading comprehension texts should be largely similar year on year. If there is a change in policy, the rationale for this should be communicated to stakeholders at least one year in advance. xix. Examination exercises made up of de-contextualized sentences and words should be avoided and replaced with chunks of connected text that form a mini-context. xx. Test input should be screened for cultural bias; cultural elements should not be so prominent that they hamper candidates from showing what they can do. xxi. Some visuals should be included not only to mitigate the anxiety associated with this examination but also as a tool for setting more effective examination tasks. 6.3 Recommendations JL Examination xxii. The elements of language tested in the Language Use component should be done more systematically so that a valid representation of most of the language elements in the syllabus is achieved over 3 or 4 years. 257

258 Appendix 5 Analysis on the Junior Lyceum and Common Entrance Examination Papers for Maltese and their attendant Syllabi Sandro Spiteri - January Remit The 11+ Review Committee requested the author to undertake an analysis of the Junior Lyceum and Common Entrance examination papers for Maltese for the years 2004, 2005 and The objectives of this analysis were: To compare the two sets of examination papers in terms of construct validity, syllabus fulfillment, levels of difficulty, and language skills required; To compare the two sets of papers in terms of their appropriateness for their stated functions; To compare papers within the same examination system to do so the following examination papers were also included in the review, so as to get a sense of the development of both syllabi and papers: Junior Lyceum (JL) Examination: 1981, 1989, 1999 Common Entrance (CE) Examination: 2000 To problematise the appropriateness of both syllabi and examination papers of both systems in terms of the education criteria of the National Minimum Curriculum and an Assessment for Learning perspective. This Report will first present the data and then the discussion, closing with recommendations. The data will focus on: The language section The comprehension section The idiomatic loading of the grammar and comprehension sections The composition creative writing section The layout and design of the papers 2. Language Section Analysis An item analysis of the language section for all examination papers under review was undertaken, with reference to the syllabus. Table 2A overleaf presents this data by comparing the JL and CE papers of the same year. The sub-categories have been grouped in 7 categories: Lexis, Morphology, Nouns, Creative Writing, Proverbs/Idioms, Syntax, and Various Grammatical Items. This analysis looked at both the percentage mark presence of each syllabus item or cluster of items, as 258

259 well as the actual number of times such items appeared in the exam paper. Information from the 1981 and 1989 papers is limited since the marking scheme was not available, nor, indeed, was such information in the exam papers themselves. 259

260 Tiswir ta' Nomi Nom plural Nouns Għadd imtenni Stat kostrutt % of total mark in paper % No. % No. % No. % No. Creative Writing Zvilupp ta' paragrafu 1981 JL / / / / / 1989 JL 5 / / / * 1999 JL / 1 1 / / * 2000 CE / JL / 2 2 / / / CE / / JL / 2 2 / / / CE / / JL / / / / CE / / The following observations may be made from Table 2A: In the JL papers there was a shift from easier grammatical elements, such as insertion of pronoun suffix, definite article, adjectives and consonants to more complex items such as sentence transformations. However, this shift was accompanied with a decisive move away from noncontextualised language exercises, still in use in the 1989 JL paper, to exercises that gave a sentence, mini-text or other rhetorical device as a meaning-making context. All the CE papers under review presented language exercise in context. 260

261 The JL papers generally gave more space for creative writing, in the form of prompted paragraph/sentences or sentence continuation than the CE papers, although there seemed to be a convergence on this since 2004, both in terms of number of items as well as in terms of the relevant percentage of the total mark of the exam paper. CE papers tended to give more importance to lexical knowledge and general vocabulary. Both exam systems are almost completely focused on the semitic element, with only minimal reference to the non-semitic element in the 2006 JL paper 22. Constructions from Mamma and Għerq remained perennial favourites, although more so in CE papers. There was a clear shift over the years from questions presenting proverbs and idioms without context to presenting them in context, from sentence continuations to scenarios and pictures. All the relevant items from 2004 onwards were in contextualised format. Although a range of grammatical items (apart from those mentioned above) are present in exam papers, some are much more popular : pronoun suffix, partiċelli with or without the definite article, numbers in words. With respect to the Maltese syllabus 23 : Over the period, the language content of the syllabus was well represented, if at times unevenly, in the JL and CE papers. The 2006 Maltese syllabus explicitly refers to the need to teach grammar in context, and this is also implied in the 1997 syllabus. The need for meaning-making context in the learning and assessment of idiomatic expression, including idioms and proverbs, is also explicitly referred to in the 2006 Maltese syllabus. The level of the individual items on a one-to-one basis is generally comparable for the two types of exam papers, although there is a slight tendency for CE items to go more for the exceptions and the nuances of specific grammatical rules. However, the significant difference in the language exercises lies more in the fact that: There are more CE items to cover in the same time as the JL exam; 22 Apart from the one reference in Table 2A, there were a few other instances which, however, I felt should be more appropriately placed under another sub-category, for example because they were part of an exercise on adjectives. 23 The Grammar section of the 2006 Maltese syllabus is essentially the same as the previous one, with more examples per item. 261

262 More marks in the Maltese CE exam go on fill-in items rather than writing/completing sentences, which allows for a greater variety of items to be tested; JL items tend to be better supported through illustrations; There is a greater use of more complex meta-language in the CE exam, as explained below. The ability to answer correctly the language section in both JL and CE required a significant knowledge of meta-language, from the basic nom, verb aġġettiv, artiklu etc to stat kostrutt, imperattiv, etc, to whole phases and commands, such as: Nissel verb mil-preżent mill-għerq t-j-r (CE 2000) Sib nom mill-għerq x-j-ħ (JL 2004) Verb minn nom pompa (JL 2006) Għaqqad il-particelli min ma l-artiklu (CE 2005) CE papers tended to make heavier use of such higher-end meta-language. CE papers also made greater use of meta-language simply because there were more grammar exercises and more reliance on fill-in word/phrase exercises, as will be seen later on. On the other hand, neither CE not JL papers resorted to less frequently used meta-language such as verbi dgħajfin, plural sħiħ/miksur, verb kwadrilitteru, verbi trilitteri sħaħ, sinonimi, antonimi etc. Indeed, these terms are mentioned in the syllabus for teacher reference purposes only. It is interesting that although the 1997 Maltese syllabus explicitly refers to the need of children to know how to use the key meta-language, this reference is left out in the 2006 syllabus. However, the use of meta-language in the 2006 JL paper is similar to previous JL papers. The language section for the papers under review was also analysed in terms of the type of response requested, of which there were the following: Fill in letter, word fragment, word or phrase Transform or elicit sentence Complete sentence Multiple choice Choose word Table 2B presents this data by comparing the JL and CE papers of the same year. Information from the 1981 and 1989 papers is not given since the marking scheme was not available, nor, indeed, was such information in the exam papers themselves. 262

263 Table 2B Language Section: Analysis by Type of Response Fill in word/phra se Transform/ elicit sentence % of total Complete Multiple Choose mark in sentence choice word paper % No. % No. % No. % No. % No. 99 JL / / CE / JL / / 4 4 CE / / JL / 6 6 CE JL / / CE The following observations may be made from Table 2B: There has been a tendency over the years to increase the number of items and the overall percentage of the marks for fill-in word/phrase answers, for both JL and CE papers. There has been a corresponding diminution of importance to the transformation/elicitation of sentences and to sentence completion. Multiple choice has made a minor appearance in the last two years for CE, and has been present only once but relatively substantially, at 10%, for JL. Word choice (from a bank of available words) has a minor but steady presence. There is a significant difference between JL and CE in the mark load per item for the fill-in word-phrase category. Indeed, the ratio of word items per mark, i.e. how many fill-in items the student needs to complete to get one mark, is almost double for CE papers, as can be seen from the Graph 2C below: 263

264 Graph 2C Comparing Ratio of No. of Fill-in Word/Phrase Items per Mark for JL and CE Exams Ratio JL CE Year Interestingly, this difference in mark load between JL and CE is either not present or else present but to a much lesser degree in the other types of response. 3. Comprehension Section Analysis The comprehension section on all papers under review was analysed in terms of the type of response requested, of which there were the following: Factual information: information clearly available from the text in reasonably close proximity to the lead phase in the text that had been made available in the question. Answer could be in a variety of forms: full sentence, one word or phrase, multiple choice. Inferential information: information not available from the text or not in proximity to the lead phrase in the text that had been made available in the question. Answer could be in a variety of forms: full sentence, one word or phrase, multiple choice. Questions that asked students to choose the appropriate proverb or idiomatic expression that matched the text or situation were considered inferential questions. Vocabulary: this included explanation of words/phrases in text and finding appropriate words/phrases in text. Grammar: this included both explicit grammar items (e.g. What is the plural of with reference to a word in the text) as well as implicit grammar items (e.g.) What does h refer to in warajh with reference to its use in the text). 264

265 Table 3A presents this data by comparing the JL and CE papers of the same year. No data are given for 1981 and 1989 since these JL exam papers did not have a comprehension exercise. The number in brackets in column 2 represented the total marks allocated in that particular exam for the comprehension exercise including related grammar items. The relevant grammar items were included in the JL papers although, in contrast with the CE papers, they are presently separately from the comprehension so as to allow for JL-CE comparison. Table 3A Comprehension: Analysis by Type of Response Factual Answer Inferential Answer Vocab Gram % % % % 1999 JL (of 15) / 2000 CE (of 40) JL (of 23) CE (of 40) JL (of 28) CE (of 40) JL (of 13) / CE (of 40) % of total mark in Comprehension Section (includes grammar exercises based on comprehension text) Graphs 3B and 3C overleaf give the percentage of each type of response from the total mark of the comprehension section for each paper, separately for CE and JL respectively. 265

266 Graph 3B Analysis of Comprehension Questions in CE Exams by Type of Response % of mark for Comprehension % Factual Answer % Inferential Answer Response Type % Vocab % Gram Graph 3C Analysis of Comprehension Questions in JL Exams by Type of Response % of mark for Comprehension % Factual Answer % Inferential Answer Response Type % Vocab % Gram The following observations may be made from Table 3A and Graphs 3B and 3C: The comprehension exercise was not included in the 1981 and 1989 JL papers. In the papers available for review it was included as from Also in the JL papers other comprehension-like exercises were included: To elicit answers in a dialogue based on a short passage (JL 2005 paper exercise GĦ) To identify the meaning of words and phrases used in a short passage (JL 2006 exercise G) To elicit information for a short CV (JL 2006 exercise I) To a lesser extent this was also used in CE papers, specifically to elicit the meaning of idiomatic expressions from a short passage (CE 2005 paper exercise C, CE 2006 paper exercise E) 266

267 In CE there is a clear shift of marks from factual questions to vocabulary-based questions, with inference and grammar-based questions showing signed of marks inflation as well. In JL the trend is not clear. Although the 2000, 2004 and 2005 papers generally follow the CE, it is not clear if the 2006 paper is an anomaly or a sign of a shift in focus. With respect to the Maltese syllabus 24 : The 1997 Maltese syllabus makes explicit reference to comprehension as a reading skill that includes the skills of extracting specific information from a variety of texts. The 2006 syllabus does not explicitly refer to comprehension but this is clearly inferred; the syllabus widens slightly the skill to including extracting of relevant information, and the understanding of words, phrases and expressions in context. Therefore, strictly speaking inferential questions could only be included in the JL and CE papers as from However, in practice the conclusion must be that both paper setters have very similar criteria of what constitutes testing a comprehension passage, although not on what constitutes an 11+ level comprehension passage, as will be shown in the next set of data with respect to the comprehension section. These criteria are not expressed in the 1997 and 2006 syllabi, which are clearly not the guidelines for the JL and CE paper setters in terms of the comprehension section. On the other hand, the greater use of comprehension-like exercises in recent exam papers both in JL and CE may have its justification in the inclusion of a wide variety of comprehension reading skills in the 2006 Maltese syllabus. The actual text of the comprehension sections of the papers under review were also checked for level of difficulty. A number of measures were used: Total number of words in text Average number of words per sentence in text Ratio of number of words to number of idiomatic expressions in text Data is presented below in Graphs 3D, 3E and 3F respectively. No data are given for 1981 and 1989 since these JL exam papers did not have a comprehension exercise. 24 The Grammar section of the 2006 Maltese syllabus is essentially the same as the previous one, with more examples per item. 267

268 Graph 3D Total No. of Words in Comprehension Text No Yr99 Yr00 Yr04 Yr05 Yr06 JL CE JL CE Graph 3E Average No. of Words per Sentence in Comprehension Text No Yr 99 Yr 00 Yr 04 Yr 05 Yr 06 JL CE JL CE 268

269 Graph 3F Ratio of Idiomatic Expressions to No. of Words in Text Ratio 1: Yr99 Yr00 Yr04 Yr05 Yr06 JL CE Year JL CE The following observations may be made from Graphs 3D, 3E and 3F: In CE, there is a general tendency over time for lengthier texts, shorter average sentences, and a higher ratio of idiomatic expressions (less words per idiomatic expression used) In JL there seems to be a trend for longer texts, although again the 2006 text may be inverting this. There is also a clear trend over time for JL comprehension texts to have longer average sentences, reaching the CE level for this measure. There is no clear trend for proportionate use of idiomatic language; the sign for 1999 signifies that in that particular JL exam paper no idiomatic expressions were used at all, and the 2006 JL text is also significantly poorer in this regard than the previous two years of JL comprehension texts. 4. Idiomatic Loading Idiomatic loading refers to the need to understand the nuances of idiomatic expression, including idiomatic phrases and proverbs, to correctly answer specific questions. This idiomatic expression may be part of the question or part of the answer, in a given text or recalled from memory. So, a total (A+B) of 24 in the last column of the Table 4A below would imply that 24% of marks in that paper relied at least partly on a good knowledge of Maltese idiomatic expression. It is important to note that these numbers do not include the composition creative writing component, in which students are expected to show a mastery of idiomatic, often archaic language As is reiterated in both 2004 and 2005 JL examiners reports for Maltese. 269

270 Table 4A below presents the data by comparing the JL and CE papers of the same year. No data are given for 1981 and 1989 since these JL exam papers did not have a comprehension exercise and so a complete comparison with the other papers was not possible. Graph 4B presents the same data for better comparison over time. Grammar Section Table 4A Compreh. Section Total A+B % % % JL CE JL CE JL CE JL CE CE % of 140, JL % of 75 Graph 4B % of Idiomatic Loading in JL and CE % of total marks / JL CE JL CE JL CE % in Language Section % in Comprehension Total A+B The following observations may be made from Table 4A and Graph 4B: The idiomatic loading in the language section has increased over time in the CE papers, surpassing in percentage terms the JL papers that have remained relatively stable. The idiomatic loading for both CE and JL papers in the comprehension section has tended to increase over time, although, as in previous observations, the 270

271 2006 JL paper is not in line with this trend. The CE papers are reaching the 2000 high in this area. Cumulatively, the idiomatic loading for both JL and CE papers has increased over time (excluding the 2006 JL paper). Comparing the overall idiomatic loading of the JL against the CE papers depends on whether the JL 2006 paper was a fluke was intended to set a new standard. However, it is interesting that in the specifications lists in the 2004 and 2005 JL examiners reports for Maltese that rate the importance of 21 different language skills for each exercise in the test paper, the skill understanding of vocabulary, proverbs, and the idiomatic aspect of language in context went up from eighth place in the 2004 list to joint third in 2005, mirroring the increase in idiomatic loading from the 2004 to the 2005 JL paper in Graph 4B previously. This can be taken as a measure of validity for the idiomatic loading measurement in the present report. 5. Composition Section Analysis The composition section on all papers under review was analysed in terms of the types of writing, or genres, being most clearly required or referred to. In identifying the types of writing a rhetorical perspective of genres was adhered to, not because of a belief 26 in the pedagogical efficacy of this taxonomy, but rather in recognition that the examiners were very likely using this approach. A total of six different genres were referred to, which were: Narrative Argumentative Descriptive Expository Extrapolation from an idiomatic expression of proverb Picture essay: single picture or picture sequence The last two can strictly speaking fall under one of the first four, but are being presented separately since they require slightly different pre-writing and drafting skills. Table 5A presents this data by comparing the JL and CE papers of the same year. Note needs to be taken that in the 1981 and 1989 JL papers only three titles were given, instead of five as with all the other papers under review. Graph 5B presents 26 Discredited at least since the work of Britton et al., The Development of Writing Abilities (11-18), School Council Publications, Macmillan Education, U.K

272 the same data by giving holistic scores for each of the composition type for the JL and CE papers. Table 5A Type of Composition Questions in JL and CE Exams Narrative Argument Descript. Exposit JL JL 2 1 Build on Idiomatic Express JL Picture Essay 2000 CE JL CE JL CE JL CE Graph 5B Composition Questions by Type No Narrative Argument Descript. Exposit. Build on Idiomatic Express. Type Picture Essay JL CE TOT The following observations may be made from Table 5A and Graph 5B: The most popular genre in both JL and CE as well as overall is the narrative, followed by the descriptive and the expository. JL papers frequently gave idiomatic expressions or proverbs as a title or prompt, which was not done in the CE papers under review. 272

273 Conversely, the CE papers all gave the possibility for a picture essay, something not done in JL papers. There was a clear shift over the years from titles to prompts. The 1981 and 1989 JL papers had straightforward essay titles; these gave way to titles with some explanation, to general prompts that required the adaptation or creation of a title for the writing piece. No picture essay had a title or further written prompts. With respect to the syllabus: There is some difference between the 1997 and 2006 syllabi in terms of writing skills, although these differences are complementary. The 2006 syllabus presents a slightly wider variety of genres than the However, the range of composition genres for both JL and CE papers is significantly narrower than either of the syllabi. In both the 1997 and 2006 syllabi there is a distinction between personal writing, creative writing and writing for a purpose, in all cases to a real audience and for a real purpose. This may be seen as the motivation for the shift across time from titles to prompts in the JL and CE papers, with a marginally wider range of genres. The 2006 syllabus explicitly mentions the need for pre-writing. Pre-writing is implicit in the 1997 syllabus in the mention of the thematic web in Yr 5; this syllabus explicitly mentions the need for redrafting. One would expect that the result of this would be time being allowed during the exam proper for pre-writing and re-drafting. Now, the 2006 JL paper has practically the same number of items as the 2005 paper with a very similar level of difficulty. Considering: the simpler and shorter text used for the comprehension, and assuming that the paper was already set at the right attainment level for an end-of-primary school assessment of expected age-appropriate learning 27 ; it therefore follows that the 2006 JL paper may well have been structured to allow for more pre-writing and/or redrafting. Indeed, in both the 2004 and 2005 examiners reports, the use of pre-writing strategies by some was highlighted, and the recommendation was made that more students need to do so. The 2006 CE paper has marginally more items than the 2005 paper, but both have very similar levels of difficulty. The 2006 exam was 15 minutes longer than the 2005 one. On the other hand, the 2006 comprehension text is longer, more idiomatically dense, and the 2006 paper has a marginally higher idiomatic 27 I consider this to be a sound assumption, a conviction shared by the writers of the JL examiners reports for 2004 and

274 loading than the 2005 one. Considering that, as will be argued at the end of this report, the CE papers are significantly more difficult at times twice as difficult than the JL papers that are meant to provide a valid picture of native language attainment in Maltese one would reasonably expect from a student at end of primary schooling, it is difficult to see how, all in all, the extra 15 minutes in the 2006 CE paper could have done anything more than provide a bit more of a breathing space for the students. It is unlikely that this extra time resulted in significantly more pre-writing and redrafting as indicated by the syllabi. 6. Analysing the Layout and Design of the Papers Layout In both the 2004 and 2005 JL examiners reports for Maltese, the writers praised the exam papers layout, design and graded content. There is certainly a marked difference between the 1981 and 1989 JL papers, prepared in the pre-desktop publishing era, and the 1999 paper onwards. The present author would agree that the font size, font type, spacing for writing, layout and balance between pictures and text for the JL exam papers is appropriate, although the overall image varies from year to year. There is perhaps too great a fondness for boxing off text in the 2006 JL paper, that may in some cases hamper the student in identifying where to write (vide exercise E) but this is a minor issue. The big plus in the CE exam papers is that they are printed on one side only, allowing students to use the empty side for rough work. The CE have a uniform look and spacing that may look less creative than the JL exam papers but serves to foster a sense of dependability and recognition, especially since students would have worked on past papers, that is helpful in an examination environment. Illustrations The pictures chosen to illustrate the JL exam papers from 1999 onwards are clearly selected to complement and reinforce understanding of the particular exercise they would be accompanying. Most pictures are quite clear, although some suffer from the lack of clarity that comes from greyscale transformation of what were obviously originally full-colour illustrations. There is also a mélange of drawing styles, from ad-hoc manual illustrations, to pasted pictures, to cutand-paste artwork available as software. Some pictures (such as JL 2004 paper exercises C and D, and exercise F although the word xarabank was placed in an improbable position) may not be culturally spot-on. Others may suffer from excessive stylisation, distortion or pixelisation (such as JL 2004 paper exercise 274

275 Ħ, JL 2006 paper exercises H and J1, respectively), all of which may deter recognition and comprehension. The CE papers fail in comparison to the JL papers in terms of illustrations. Some illustrations (such as 2006 and 2004 CE papers exercise A, and 2004 CE paper exercise C) seem to have only a decorative or stress relaxing function rather than to aid comprehension, and this is contraindicated. Others are less recognizable than some of the illustrations in the JL papers, and suffer from the same vagaries of style and provenance. Some pictures (such as 2006 CE paper exercise I) may even be confusing: what is the boy dreaming about a halfdog? All in all, the illustrations in the CE exams papers apart of course from the picture essay give the impression of having been added on after the paper has been constructed and laid out, rather than being an intrinsic part of the meaning-making framework to be presented to the student. This is less the case with the JL exam papers. A significant failing with respect to illustrations in the CE exam papers relates to those used in the picture essays. Here, the picture does not simply aid comprehension, but is meant to provide the totality of meaning. Objections can be raised for each one of the illustrations used from 2000 to 2006 in the CE exam papers under review. The 2000 illustration is very poor, clearly a bad reproduction, so much so that what is presumably the postbox at the street corner had to be added on, in a manner that did not respect the lines of perspective of the picture. The 2004 and 2005 illustrations are clearly outdated and also poorly reproduced. The 2006 picture essay presents a sequence of pictures, which is a significantly more helpful than the illustrations in previous years, but also suffers somewhat from inferior reproduction and tiny sequence numbers. 7. Discussion The Syllabus as benchmark Because of the parameters of this research exercise, a thorough analysis of the syllabus that the JL and CE exam systems overly refer to, including a comparative analysis with other native language syllabi at the same point of compulsory education transition, was not possible. However it is pertinent to point out that as a pedagogical syllabus it has its limitations, such as: There are elements of Maltese syntax and other aspects of grammar that have come out from linguistic research of the last 30 years than can be considered for inclusion; 275

276 There are certainly alternative ways of presenting the inter-relationship between different areas of linguistic knowledge that would be more generative in terms of holistic communicative approaches; The total hegemony of the standard version of Maltese, polemically called pulit, is not problematised, nor is attention given to the issue of the varying status of regional/local variants of Maltese with respect to Standard in terms of different linguistic skills; Although a lot of emphasis is placed on the teaching of grammar in communicative context, there still remains an element of the old-fashioned taxonomic approach; The full implications of a writing process approach on the place of orthography in teaching of writing in not completely coherent; There is need for a clearer policy on the use of meta-language and its relationship to autonomous learning. Notwithstanding all this, the significant development the Maltese syllabus has undergone since the first post-independence syllabus of 1965 is acknowledged. It is now broadly in line with the modern language teaching paradigm at least in the Western World, and operationalises with the caveats mentioned above key aspects of the National Curriculum and its holistic educational vision. This is because it is basically sound in that it is process-oriented, skills-based, scaffolded, with a tried and tested range of content, and a solid base of suggested good practices and teaching/learning strategies that impact positively on test paper design. Therefore, although this syllabus is not strictly speaking an examination syllabus, we can gauge to what extent the JL and CL exams measure up to it. The Syllabus and the Exam Paper Construction A holistic analysis of all the data presented would lead one to conclude that although the JL and CE exams purport to assess the same syllabus, they in fact are driven by different forces that have turned them into significantly different assessment tools with different raisons d etre. Indeed, it is not altogether clear that they are both following the syllabus. Although this would seem to be the case for the language section, it is less so for the comprehension section (and even less so for the CE papers) and for the composition section. It would seem that the paper setters are guided both by the overt syllabus that is common to both papers, as well as by an internalized or hidden assessment syllabus, elements of which have been uncovered by the present research. 276

277 One way how both the JL and CE exams can improve is by being more faithful to the full extent of content and intention of the syllabus especially with respect to creative writing. Not only do more genres need to be introduced, but time and space need to be made for prewriting and redrafting to take place, with this expectation being made explicit in the exam papers. Reading and oracy assessment are glaringly absent from both JL and CE Maltese exams, although they are rightly present in the syllabus and the SEC. Their introduction would only be realistically possible with a transformation of the operant assessment mode to an Assessment for Learning paradigm, within which end-of-primary exams would change into benchmark and diagnostic tools. The need for a more uniform, systematic and professional approach to exam paper illustrations, in JL but even more so in CE exams, has been highlighted. 277

278 Appendix 5a Submissions to the 11+ Review Committee with respect to the Analysis on the Junior Lyceum (JL) and Common Entrance (CE) Examination Papers for Maltese and their attendant Syllabi Sandro Spiteri - January 2007 The following submissions are rooted in the analysis carried out by the undersigned on the JL and CE papers for Maltese as requested by the 11+ Review Committee, but go beyond this analysis to focus on implications for the assessment paradigm and test paper construction at the critical 11+ juncture. 1. Assessment Paradigm and Exam Paper Construction a) My analysis would suggest that the claim that JL exam papers are reflecting the native language attainment of Maltese that, all things being equal, Maltese 10/11-year olds can reasonably be expected to have mastered by the end of primary schooling, can all in all be accepted with some qualifications. This is because the basic conditions and parameters are there: - the exam is norm-referenced; all students reaching pass-mark can enter the Junior Colleges; - the content has been selected, timed, structured and graded over long years of experience in 2006 the JL exams were 25 years old, and the pass rate as a result of all this experience has now passed 60%; - the content is well selected over the range of the syllabus, and changes sufficiently from year to year to present a revolving cast of content and question constructs that are not, however, predictable; - The paper design has all the right elements of contextualization and many of the elements related to illustrations as aids to comprehension as indicated by good practice in this field. b) Indeed, although the JL system is currently based on a competitive exam, it may be considered to be operating within a Benchmark Paradigm again with some qualifications since the exam papers already have the outlines and structures to become, if such a move is made, fully fledged diagnostic tools that fulfill the Assessment for Learning perspective implicit in the National Curriculum. In this the JL system has moved significantly in concept from its original structure, which was strictly competitive with a limited number of places in the original Junior Lyceums. 278

279 c) If therefore the JL exam papers with further adjustments are taken as the gold standard for assessment of Maltese at the primary/secondary transition point, it follows that the CE papers, with: - significantly more work per mark; - higher idiomatic loading; - use of higher-end meta-language; - longer comprehension texts, and - higher-order comprehension exercises, with less factual answers and more inferential ones, are designed around a different assessment paradigm than that of the JL. d) If the JL system is operating within/working towards a Benchmark Paradigm, the CE can be said to be operating within the Selection Paradigm. This is of course mostly grounded in the need to differentiate between applicants for a highly competitive, highly selective exam. Even with exam papers that are significantly harder than the JL ones, entry into one of the schools participating in the Common Entry exam can only be contemplated with an average mark, when considering all five exams 28, that is well in the 70-80% bracket 29. e) However, the CE Selection Paradigm of assessment has deeper roots. Some of the Church schools that participate in the CE exam system consider it to be an integral part of their religious charism to select academically gifted students who would be trained as Catholic leaders so as to bring about the greatest possible change in society from positions of service through power. Apart from this, all Church schools see each new intake in terms of a potential harvest of new vocations, in which academic ability is a pre-requisite. These considerations are partly counterbalanced by the dilemma many in Church schools face that the inevitable CE selection system has lessened the Church s ability to reach out to the poorest of the poor through its schools. And yet the situation remains as it is. f) So, although the JL and CE exam systems may, on analysis of the papers themselves, not seem too far apart, they are in fact fundamentally different because their philosophical foundations are on different tectonic plates. Superficial attempts to harmonise the two systems and bring them in line with an Assessment for Learning approach to exams at the primary-secondary transition point will quickly uncover this difference. g) The high-stakes scenario in the CE system has a number of implications on exam paper design. Although it clearly has not led to content inflation in the 28 Maltese, English, Maths, Social Studies, Religion. 29 Vide successive summary reports at the back of CE past paper booklets. 279

280 new 2006 Maltese syllabus, that is essentially a further elaboration of the 1997 version, it has led to the CE exam paper-setters for Maltese opting for measures to reduce discretionary marking to a minimum thus the raise over time in marks for single-word/phrase items over longer syntactical items. h) The data suggests that this in turn may have led to a similar paper design inflation in JL design, through progressively longer and more complex comprehension texts, for example, although it could be that the 2006 JL paper marks a turn in a new direction for JL exam paper design. i) On the other hand, it is worth noting that this paper design inflation has not led to the ditching of good practices such as language and idiomatic expression exercises in context, and a spreading use of the comprehension format. 2. The Way Forward a) The cost to students and families of the pressure exerted by the CL exam, not only the Maltese paper but the whole system, cannot be underestimated. Grima and Farrugia 30 have reviewed the research on the stress induced by the JL system. The stress induced by the CE system, which is objectively grounded in the very design of the exam papers, is of a significantly higher order of magnitude. b) The preceding discussion has served to indicate that certain ways forward are non-starters as solutions to the dilemma of the 11+ exam system: - Redesigning the syllabus, while useful, worthwhile and necessary, will not impact on the issue at hand; - Redesigning of the JL exam within present parameters will, again, not address the issue. - scaling down the CE exam to make it less of a distortion of expected 11+ attainment is not an option since the present level sometimes struggles to provide the necessary differentiation for selection purposes. c) Nor would the solution be to simply change the mechanics of the two examination systems. I refer specifically to the idea doing the rounds that the solution would be to fuse the JL and CE exam systems and have one exam system, in which students would list in inverse numerical order their preferred secondary schools, and attend according to their exam placing. Although this solution may sound attractive because it would reduce examination time for over 1,500 boys by half, it would, in fact make things worse. This is because it does not problematise the powerful centrifugal forces that have shaped the CE 30 Transition from Primary to Secondary in Malta: Time to Break the Mould?, Grima G. and Farrugia J., FES Educational Publications Series: Families and Schools, Allied Publications, Malta,

281 exams into what they are today, that is the dilemma of how to select only 1 in every 4 of applicants, from a field that every year includes amongst the most academically gifted boys in the cohort. If this underlying justification for the CE system as it is today is not removed, the chances are that a simple fusion of JL and CE exam systems will result in exam papers that will inexorably have a significantly higher content and construct level than the present JL papers, simply because the level of the present JL papers is not high enough to allow for the discrimination between candidates required by the CE system. d) This is not to say that a harmonization between the JL and CE systems such that boys spend less time on exams is not possible. However, it has to be a critical harmonization, not simply an administrative one. A critical harmonization would never loose sight of the goal of transforming the 11+ examination into a national benchmarking and diagnostic tool. This is within reach for the present JL system, but still conceptually distant with respect to the CE. e) Thus, the real challenge is for the Church and Church schools in Malta to discern deeply what the CE system is doing not only to their schools but to the rest of education in Malta at the critical 11+ juncture, and to judge this in terms of their mission and calling in Malta today. The Church today needs to do the next bold step, and look at its educational provision not only in terms of its indisputable right to provide education, which is not in question, but its duty to capitalize on the symbiotic relation its schools have with the rest of the educational system in Malta to ensure that education in Malta is an effective tool for social justice and empowerment for all. 281

282 Appendix 6 An Analysis of the Content and Cognitive Demands of the Maltese 11+ Mathematics Examinations Cettina Axiak Introduction The purpose of this paper is to look into the quality of the most recent Mathematics Papers in the 11+ examinations, namely the Junior Lyceum Entrance (JL) examination and the Common Entrance (CE) examination. This has been carried out for the examinations held in 2005 and in The focus of the study relates to the validity of the examinations and constitutes an examination of the content and cognitive demands of the questions used. The data used includes the relevant syllabus, the examination papers, the marking schemes and the markers reports. Over the last 25 years, I have come to recognise with increasing conviction that there are questions relating to the 11+ examinations that go beyond the quality of the examinations themselves. I have experienced this personally both as a lecturer in mathematics education and as a parent of two children who have sat for these examinations. Selection at such an early age raises pressing questions of equity; for example, do all our students get an opportunity to develop educationally to their full potential? While such issues go beyond the scope of this paper, I also recognise that educational reform is necessarily a lengthy process needing to take the many stakeholders (pupils, parents, teachers, educational authorities) on board in a carefully directed manner. Changes overnight simply don t work and eliminating the 11+ examinations overnight would not be a solution. On the other hand, written tests will always continue to serve as an important assessment tool and we should strive to improve their quality. While we still have the 11+ examinations, it is important that these reflect both the mathematics content that is useful for everyday life and that which helps our pupils gain a good foundation for the subject. This is particularly important for the 11+ examinations where teaching to the test is an inevitable consequence of their very high stakes nature. The paper is divided in six sections. The first section takes a look at the syllabus, while the presentation of the examination papers is treated in the next section. Section 3 includes a description of how the papers were analysed for distribution of content and for cognitive demand. Since the cognitive demand of the items in the 282

283 examination papers were analysed following the TIMSS 2007 framework for assessment, this section also incorporates a description of the categories employed by TIMSS in their analysis of the cognitive demand of their assessment items. In sections 4 and 5, the distribution of content across the four examination papers and the cognitive demand of the papers are discussed in turn. At the same time, the actual results obtained in coding each item in each of the four examination papers for content and cognitive demand are presented separately in Appendix 6a. The last section includes the main findings as well as suggestions for improvement of the examinations. The Syllabus The syllabus is undoubtedly a most important guide as to what is taught and tested. The 11+ examinations under discussion were all governed by the same syllabus, the Syllabus for the Primary School: Years 4, 5, 6 (Curriculum Management Department, 1997). This is true for both the JL entrance examination and the CE examinations. Schmidt, Wang & McKnight (2005) identify coherence as a necessary condition for a syllabus to be of good quality. For coherence, syllabi need to be articulated over time to respect the logical, and where appropriate, hierarchical nature of the discipline under study. The Primary syllabus content is described in Sections 2-5 on Number, Measurement, Shape and Space and Data Handling. These sections are retained through Years 4, 5 and 6. For each section, there is a progressive development of the content through the three years, respecting the logical and hierarchical nature of the mathematics involved. How does the content covered compare with that expected elsewhere at this level? A comparison with the content of the upcoming Trends in International Mathematics and Science Study (TIMSS) mathematics tests in 2007 at Grade 4 can throw some light on this question. The TIMSS is an ongoing longitudinal curriculum development project in Mathematics and Science of the International Association for the Evaluation of Educational Achievement (IEA) (Mullis et al., 2005). Three cycles have already been completed; TIMMSx1995 involving 41 participating countries, TIMMSx1999 involving 38 countries and TIMMSx2003 involving more than 50 countries. About 60 countries are expected to participate in the upcoming TIMMSx2007 assessments. Testing within the TIMMS is carried out in Grade 4 and Gradex8. It is important to note here that TIMSS Grade 4 testing is carried out with pupils a year younger than those normally sitting for the 11+ examinations. 283

284 This comparison is particularly useful because before publishing the content domain for the assessment, the participating countries were consulted about their curricula to ensure that the TIMSS 2007 assessment framework reflects the goals of mathematics and science that are regarded as important in many countries (Mullis et al., 2005). In comparing the Primary mathematics syllabus with the content domains published for the TIMSS 2007 Grade 4 mathematics testing, it is evident that: There is plenty of overlap between the two syllabi. The Number Section of the local syllabus is very similar to the Number content domain published for TIMSS, except that a few topics feature in the local syllabus, but not in TIMSS 2007 Grade 4 content domain. These include percentages and the recognition of prime and of rectangular numbers which are neither mentioned nor implied in the TIMSS content description. The content described in the Measurement section of the local Primary syllabus is all covered by the TIMSS 2007 content domain for Gradex4. The content described in the Shape and Space section of the local Primary syllabus is included in the TIMSS 2007 content domain for Gradex4. Moreover the TIMSS document includes the identification of parallel and perpendicular lines and rotations of figures, to include simple cases for both two- and three- dimensional shapes, while the local Primary syllabus does not. On the other hand, the Primary syllabus requires the construction of circles and the use of the term circumference, but this is not required by the TIMSS content domains for Grade 4. Moreover, while the Primary syllabus requires candidates to know and use compass directions, the comparable TIMSS requirement is more open; pupils at this level are required to use informal coordinate systems to locate points in a plane (Mullis et al., 2005, p.21). The section on Data Handling in the local syllabus, and likewise the section Data Display: Reading and Interpreting in the TIMSS 2007 content domain are the shortest content categories in the two documents. The local syllabus includes averages, an area not mentioned in the TIMSS content domains. Meanwhile, TIMSS includes the reading of pie-charts which is not covered in the Primary syllabus. Moreover, the TIMSS content domain includes the comparison of data from related data sets; a key component of Data Handling that is not mentioned in the Primary Syllabus. The first section of the Primary syllabus, Problem Solving, has more to do with pedagogy than with content and is exactly the same for the three years, Yearx4 to 284

285 Yearx6. In this brief section, it is pointed out that problem solving should be seen as an ongoing process throughout the year and across all areas of the syllabus. Another aspect in this section, also taken up in the final section of the syllabus Further Guidelines for the Teacher, is the need to emphasise oral and written modes of communication together with recommendations for teachers to include mathematical discussion in their classrooms. These guidelines are consistent with a reform-oriented vision now shared by the international community of mathematics educators away from passive traditional rote learning towards learning through active involvement and reasoning by the pupils (see for example National Council of Teachers of Mathematics (NCTM), 2000; Romberg & Kaput, 1999; Swan, 2005 and van den Heuvel-Panhuizen, 2002). The Examination Papers: Presentation Each of the 2005 and 2006 Junior Lyceum Mathematics papers consist of a written paper to be completed in 1hour 45 minutes. Both papers consist of 20 questions, with the first 10 questions each carrying a total mark of 4 marks and the last 10 questions each carrying a total mark of 6 marks, thus giving an overall total of 100 marks. All the twenty questions are divided into smaller items; in most cases each item within a question can be worked out independently from the other items in the same question. However, a few of the questions are structured so that the appropriate solution to the later items in the same question may depend on whether the previous parts have been suitably answered. Such questions are suitably graded, with successive items in the question becoming harder. For both JL examinations, it is clear that the paper setters were very careful to use language that is clear, succinct and to the point. The diagrams are also clear and unambiguous. The mark allotted to each question is declared in the opening rubric to the examination paper, thus avoiding further clatter to the presentation of the individual questions. Both JL entrance examination papers are write-on and at first sight, the layout appears to be very well organised. On further reflection, when taking into account the candidates possible responses, it is quite clear that in a few cases, not enough working space has been allowed. This is evident for example in JL 2005 for questions 3a, 15c, 18, 19 and in JL 2006 for questions 6a(ii), 6(b), 14b(ii), 19b and 20. Considering also that the papers are printed on both sides and have narrow margins, the pupils do not have the space to carry out rough work as they find 285

286 necessary. Contrary to the vision expressed in the syllabus, the message carried to teachers and pupils is that rough work is not important. Moreover, in cases where different methods requiring a different amount of working space are possible, allotting little working space suggests to teachers and pupils that rather than going for methods to ensure pupil understanding, the focus should be on the teachertaught shorter methods. Question 14b(ii) of JL 2006 exemplifies this situation clearly; this part question could be worked out in a number of ways to include (i) by adjusting the decimal points to remove decimals followed by cancelling and (ii) repeated subtraction of 1.25 from 7.5. Given the space allowed, the message seems to be that method (i) but not method (ii) was a possible option even though according to the syllabus, formal division methods for division by decimals is not expected at Year 6 Primary. Each of the 2005 and 2006 Common Entrance Mathematics papers consist of a written paper to be completed in 1hour 45 minutes. Both papers consist of 20 questions, each carrying a total mark of 10 marks giving an overall total of 200 marks. All the twenty questions are divided into smaller items; and as for the JL papers, a few of these questions are structured so that a positive response to later items is only possible following a correct response to previous part/s. Generally such questions are suitably graded, with successive items in the question becoming harder. A few exceptions are items 2c and 18biii in CE 2005 and item 13f in CE It is pertinent to point out that all these items carry few marks, only one or two marks in each case. In the case of the CE examinations, the language is carefully chosen to be clear and to the point thus avoiding extra unnecessary linguistic demands on the pupils concerned. The diagrams are also clear and unambiguous. Although the papers are write-on and the space for each question is necessarily limited, the pupils have ample space for working. This is because the papers are printed on one side only and the candidates are instructed in the opening rubric of the question paper that they may use the opposite blank page for any additional workings. Analysis of the Examination Papers Analysing the distribution of content In the process of preparing the JL entrance mathematics examinations, a specification grid is used to help manage the distribution of items across the 286

287 different parts of the syllabus and to ensure a good spread of items along the easy to difficult continuum. This is useful in order to make the papers appropriate for a wide range of pupil ability. In distributing the items across the syllabus, the following targets are set: 20±1% for Problem Solving, 40±3% for Number, 15±3% for Measurement, 15±3% for Shape and Space and 10±1% for Data Handling. In ensuring for a good spread of items of different difficulties, the paper setters allot marks for Low Level items, marks for Medium Level items and finally marks to items that are considered to present a High Level of difficulty (Educational Assessment Unit, 2005; Educational Assessment Unit, 2006). While I would certainly agree with this overall strategy, I find difficulty with including Problem Solving with the four content domains: Number, Measurement, Shape and Space and Data Handling. This is because I share the view expressed in the syllabus that problem solving should be seen as an ongoing process spreading across all the four content areas of the syllabus. In considering the distribution of content across each of the papers, for the purposes of this paper, I decided to take each item in turn and list the section and subsection/s of the syllabus that comes into play when working out this item (See Appendix 6a). This process is very similar to the one undertaken by the JL examination paper setters, except that in my case, I was only using the four syllabus content categories, and was not using Problem Solving in addition to the other categories. Sometimes, when coding an item, more than one method of doing this was apparent, and in such instances I chose the content area that appeared to contribute more to the difficulty of the item. This is necessarily a very subjective process which was quite taxing because many items tested more than one content area. In this process, it was also considered useful to get a measure of the cognitive demand of the items. Problem solving would be incorporated in this measure because questions testing recall of mathematical facts are considered to place lower cognitive demands than questions testing application of skills while questions testing non-routine problem solving are considered to make the highest cognitive demands. Analysing the cognitive demand of the items The cognitive demand of mathematics items are mapped out differently in diverse assessment projects. Neidorf, Binkley, Gattis, and Nahara (2006) compare the assessment frameworks in mathematics of three projects; the National Assessment 287

288 for Educational Progress (NAEP) in the United States, the Program for International Student Assessment (PISA) conducted by the Organisation for Economic Cooperation and Development (OECD) and TIMSS. Where it comes to the cognitive aspect, the three assessments structure this dimension differently but there is considerable overlap in the specific competences that are selected for inclusion in each assessment to describe this element. In particular; reasoning, communication and making mathematical connections all feature in mapping out the cognitive dimension in the three assessment frameworks. It was decided to analyse the cognitive demand of the items in the JL and CE examinations under study following the framework that has been prepared for TIMSS 2007, after a revision of the categories that had been used in TIMSS The TIMSS assessments are based on frameworks developed through a consensus process involving mathematics experts, education professionals, and measurement specialists from many countries (Mullis et al., 2005). Moreover, as described previously, the TIMSS assessments for Gradex4 in mathematics are structured around a content domain which is very similar to the content tested in the local 11+ examinations. TIMSS 2007 describes the cognitive domain in terms of three categories: Knowing, Applying and Reasoning. These categories, used in both grade 4 and grade 8 assessments, are described below (Mullis et al., 2005). Knowing: In order to be able to engage in purposeful mathematical enquiry, pupils need to have an adequate knowledge of the mathematics relevant to the situation. Such knowledge includes knowledge of mathematical facts, conventions and notations, mathematical procedures and concepts. As described by Mullis et al. (2005), this cognitive domain is taken to be represented by the following behaviours: 1. Recall Recall definitions; terminology; number properties; geometric properties; and notation (e.g., a b = ab, a + a + a = 3a) 2. Recognize Recognize mathematical objects, shapes, numbers and expressions. Recognize mathematical entities that are mathematically equivalent (e.g. equivalent fractions, decimals and percents; different orientations of simple geometric figures). 288

289 3. Compute Carry out algorithmic procedures for +,,,, or a combination of these with whole numbers, fractions, decimals and integers. Approximate numbers to estimate computations. Carry out routine algebraic procedures. 4. Retrieve Retrieve information from graphs, tables or other sources; read simple scales. 5. Measure Use measuring instruments; use units of measurement appropriately; and estimate measures. 6. Classify/Order Classify/group objects, shapes, numbers and expressions according to common properties; make correct decisions about class membership; and order numbers and objects by attributes. Mullis et al., 2005, pp Applying: This cognitive domain relates to routine problem solving and is used to describe related processes; selecting the relevant information, deciding on a plan and carrying it out to solve a problem. The routine aspect of the problem involved is a distinguishing feature of this cognitive domain; in the case of non-routine problems, the same processes would be classified under the last cognitive domain, Reasoning. Mullis et al. (2005) describe the behaviours associated with this cognitive domain as follows: 1. Select Select an efficient/appropriate operation, method or strategy for solving problems where there is a known algorithm or method of solution. 2. Represent Display mathematical information and data in diagrams, tables, charts or graphs, and generate equivalent representations for a given mathematical entity or relationship. 3. Model Generate an appropriate model, such as an equation or diagram for solving a routine problem. 4. Implement Follow and execute a set of mathematical instructions. Given specifications, draw figures and shapes. 289

290 5. Solve Routine Problems Solve routine problems (i.e., problems similar to those target students are likely to have encountered in class). For example, use geometric properties to solve problems. Compare and match different representations of data (eighth grade) and use data from charts, tables, graphs, and maps to solve routine problems. Mullis et al., 2005, pp Reasoning: This cognitive domain includes processes known to be involved in mathematical reasoning to include intuitive and inductive reasoning as well as deductive reasoning. In justifying their results, pupils also need to be able to communicate their reasoning effectively Mullis et al. (2005) describe the behaviours associated with this cognitive domain as follows: 1. Analyze Determine and describe or use relationships between variables or objects in mathematical situations; use proportional reasoning (fourth grade); decompose geometric figures to simplify solving a problem; draw a net of a given unfamiliar solid; visualize transformations of three-dimensional figures; compare and match different representations of the same data (fourth grade); and make valid inferences from given information. 2. Generalize Extend the domain to which the result of mathematical thinking and problem solving is applicable by restating results in more general and widely applicable terms. 3. Synthesise/ Integrate Combine (various) mathematical procedures to establish results, and combine results to produce a further result. Make connections between different elements of knowledge and related representations, and make linkages between related mathematical ideas. 4. Justify Provide a justification for the truth or falsity of a statement by reference to mathematical results or properties. 290

291 5. Solve Non- Routine Problems Solve problems set in mathematical or real life contexts where target students are unlikely to have encountered closely similar items, and apply mathematical procedures in unfamiliar or complex contexts. Use geometric properties to solve non-routine problems. Mullis et al., 2005, p. 38 As described earlier in this section, for each of the JL 2005, JL 2006, CE 2005 and CE 2006 papers, each item was coded for the syllabus content area to include the section and subsection/s of the syllabus that is tested. A separate spreadsheet was used for each examination. The process was repeated for the cognitive domain, so that the cognitive domain and behaviour associated with each item was included for each item in each spreadsheet. The result of this analysis is presented in Appendix 6a. Some further analysis of this data is provided in the next two sections. The Content Assessed The quantitative data gathered as described in the previous section was used to analyse the content assessed in the four papers. Use was also made of some of the comments I made about individual items. This is also true for the next section where the cognitive demand of the papers is analysed. An inspection of the spreadsheets revealed that all the subsections of the four content domains were tested in each of the examinations under discussion, except that Sectionx4.1, Circumference of a Circle, was neither tested in JL 2005 nor in CE The distribution of marks across the syllabus content categories is given in Table 1. In all cases, most marks were allotted to the Number section; indeed this was as planned since the following targets were set at the planning stage: 20±1% for Problem Solving, 40±3% for Number, 15±3% for Measurement, 15±3% for Shape and Space and 10±1% for Data Handling. The CE examinations allotted an even higher mark for Number than the JL examinations; this was particularly the case for CE The distribution for each examination follows the same pattern with decreasing marks as one moves down the sections Number, Measurement, Shape and Space, and Data Handling. Indeed the Data Handling section has been allotted few marks; this is especially the case for CE

292 Table 1 Distribution of Marks Across the Content Categories JL 2005 JL 2006 CE 2005 CE 2006 % % % % Number Measurement Shape and Space Data Handling Total Table 2 Distribution of Marks Across the Number Sub-categories Number Sub-Categories JL 2005 JL 2006 CE 2005 CE 2006 % % % % 2.1 Whole Numbers The four operations Patterns in number /2.5/2.6 Fractions/ Decimals/ Percentages Proportion Total In all papers, the Number category takes up more than half the total marks allotted. It was therefore considered useful to break down the marks for this category in terms of the subsections for this category (see Table 2). Because Fractions, Decimals and Percentages are intimately related, in some cases, the items seemed to be testing two or more of these subcategories to the same extent. For this reason these three categories are collapsed in Table 2. Although the Number category, in total, is given roughly the same weighting in all papers, the mark distributions across the subcategories are very dissimilar. The low weighting given to Fractions/ Decimals/ Percentages in JL 2005 is particularly noticeable. Working with fractions is known to be much more difficult than working with whole numbers. The cognitive demand of items testing Fractions/ Decimals/ Percentages will be given special consideration in the next section. The Cognitive Demand of the Papers As described earlier, the items belonging to the Reasoning cognitive domain involve pupils in interpreting contexts that are non-routine for the pupils concerned and 292

293 consequently require them to make mathematical connections as necessary in order to sort them out appropriately. Good testing items also need to reflect mathematics that is important for the students to learn; they also need to be accessible to the pupils and need to discriminate well between the more and less mathematically able pupils. Because of their non-routine character, Reasoning items of good quality are generally much more difficult to find or to construct than Knowing or Applying items. Table 3 details the distribution of marks by content category and cognitive domain for the four 11+ examination papers. It is clear that a good proportion of the marks were allotted to items involving Reasoning in all four papers (37%, 31%, 42.5% and 35% for JL 2005, JL 2006, CE 2005 and CE 2006 respectively). The marks allotted to items involving Reasoning were usually coming from items testing the Number content domain. The percentage of the total mark for Reasoning coming from the Number domain is 84%, 71%, 64% and 85% for JL 2005, JL 2006, CE 2005 and CE 2006 respectively. Reasoning items testing Number In the JL papers, most Reasoning items testing the Number domain came from situations involving whole numbers. There were some very good quality items of this type. The Chief Examiner s Report for JL 2005 gives some examples of pupil reasoning in questions 19 and 20, two questions in JL 2005 both involving Reasoning items associated with the Number section of the syllabus and involving operations with whole numbers (Educational Assessment Unit, 2005). Table 3 Distribution of marks by content category and cognitive domain Content Category JL 2005 JL 2006 CE 2005 CE 2006 % % % % Knowing Number Measurement Shape and Space Data Handling Subtotal (knowing) Applying Number Measurement Shape and Space Data Handling Subtotal (Applying)

294 Reasoning Number Measurement Shape and Space Data Handling Subtotal (Reasoning) On the other hand, neither of the JL papers included Reasoning items involving fractions; except for one Reasoning item involving percentages (qn 4c(i) in JL 2005). This question involving percentages (shade 80% of a strip which is divided into fifths) may seem a straightforward question, but pupils at this stage are unlikely to have had experiences where they represent shaded parts of an object as a percentage, particularly when the object is not divided into 100 parts. This may explain why this question, according to the Markers Report proved difficult for the pupils (Educational Assessment Unit, 2005). The JL 2006 paper included a good number of Reasoning items involving decimals (qns 14b(ii), 14b(iii), 18, 20a and 20b in JL 2006). The difficulty associated with qns 18, 20a and 20b, all coming from JL 2006, was more to do with finding a strategy for working out the problems involved than for working with decimals. This was not the case in qn 14b(ii) of JL 2006, where a sensible approach to the problem would have involved a division of 7.5 by It would be interesting to know whether any of the pupils managed this through repeated subtraction, especially because formal division by decimals is not included in the syllabus. In the CE papers, the Reasoning items testing Number were more varied to include a wider representation across the subsections. As compared to the JL papers, there were more Reasoning items testing fractions, decimals and proportion. The CE 2005 paper included one Reasoning item testing fractions (qn 9c). This involved shading 3 / 8 of a shape in the form of a big triangle formed by 4 equilateral triangles. This item requires students to reason about partitioning, an aspect essential to understanding fractions. Question 12b in CE 2006 was an original question on fractions requiring reasoning about fractions. A diagram of a scale was drawn, on one pan there were half a brick and 5kg. On the other pan there was a brick. The question was to work out the size of one brick. Because the fraction involved was the most commonly used fraction, the half, this makes the question accessible to more pupils. 294

295 Reasoning about decimals in an abstract context was required in qns 11a and 11b in CE The first question asked whether or not = 270 given that 27 x 43 = 1161; while in the second question, the pupils were asked to decide whether or not 2.7 x 4.3 = Even though formal multiplication of two decimal numbers is not included in the syllabus, it could be argued that the pupils could reason that 2.7 x 4.3 cannot equal to 1161 basing for example that the product is smaller than 3 x 5 = 15. Similarly, pupils may get a feel as to whether the first statement is correct through approximation. On the other hand, these questions may easily be interpreted as testing multiplication by two decimal numbers and division by decimal numbers; two areas not included in the syllabus. The backwash effect in classrooms is that teachers may feel pressured to teach formal methods for such calculations with decimals. Since working with decimals is an indispensable area, it is important that pupils learn this topic through meaning. This entails giving time for building meaning of related processes and making connections with real life applications. However, at this stage, the vast majority of pupils are still coming to terms with the simpler processes related to decimals that are included in the syllabus. For this reason, classroom time is better spent on consolidating meaning for the simpler processes than for teaching rules for multiplication and division of decimals for examination purposes. The quality of these items is also under question for another reason. These items requiring a True or False response encourage guessing and do not usually discriminate well between the more and less able pupils. There were indeed very few items of this type across the four papers under review. By contrast, the demands of question 10 of CE 2006 are more explicitly in line with the demands of the primary syllabus. Questions 10a and 10b tested pupils knowledge of decimal notation in the context of conversions from litres to millilitres and vice-versa. The problem in question 10c involved reasoning about capacity and also required pupils to know how to multiply the decimal number 2.5 by 10. From Table 2, a slightly heavier weighting is seen to have been allotted to Proportion in the CE papers than in the JL papers. Moreover, the JL items testing proportion; questions 15b(ii), 15(c) in JLx2005 and 19a(i), 19a(ii) in JL 2006 are much easier. All these questions test the use of scales or simple rates. For the CE examinations, Proportion is tested through qns 10b, 18b(ii) and 19(a) in CE 2005 and through qns 11b and 19a, 19b and 19c in CE None of these questions can be solved by simple application of known methods but depend on good use of proportional reasoning. Moreover, I feel that Question 19c in CE 2006 stretches the syllabus too far, given that average speed is not mentioned specifically in the syllabus. 295

296 Reasoning items testing Measurement In the JL papers, only one of the items testing Measurement was classified as involving Reasoning. This was qn 19b of the JL 2006 paper requiring pupils to reason about the areas of different rectangles with the same perimeter; a question that turned out to be of moderate difficulty with a facility of 49% (Educational Assessment Unit, 2006). This question is also interesting because the context involves an authentic application of Area to real life situations. In the CE papers, there were more Reasoning items testing Measurement (qns 9a, 9b, 16a, 16b and 16c in CE 2005 and qns 8b, 18a, 18b, 18c in CE 2006). Question 18 in CE 2006 is another good example of an item involving mathematical reasoning in a real life application of measurement, this time packing of cereal packs so as to leave no gaps. Another positive feature in this question is the demand for a written explanation rather than a numerical answer in question 18b. There were no other items of this type in the four papers reviewed. This type of question sends an important message to both pupils and teachers; that learning mathematics includes as an essential feature the development of pupils verbal and written communication of mathematical reasoning. Reasoning items testing Shape and Space In the JL papers, there were three Reasoning items testing Shape and Space (qn 2a in JL 2005; qns 13b and 17c(i) in JL 2006). Question 2a in JL 2005 was a simple question testing reasoning about symmetry. Question 17c(i) in JL 2006, asking pupils to draw a circle to pass through the vertices of a hexagon, was probably an unfamiliar question to most, but there were many correct solutions and the item scored a facility of 83% (Educational Assessment Unit, 2006). Question 13b in the same paper turned out to be a difficult item, scoring a facility of 33%. This question was about a Captain Gold on a Treasure Island (represented by a map with grid lines along the compass directions, each unit equal to 1.5km). Captain Gold goes from Bay to Treasure on the map by walking 4.5km south and then 4.5 km east. The task was to use compass directions to describe another way Gold could walk from Bay to Treasure. Presumably, the pupils are expected to assume that Gold could take any route he pleases provided he starts at Bay and ends at Treasure. However, this situation is not true to life- usually people cannot walk in any direction. It all depends on the roads available, which were not drawn into the map. In the CE papers, two items were seen to involve reasoning about Shape and Space (qn 2b in CE 2005 and qn 9a in CE 2005). Question 2b involved reasoning about 296

297 symmetry and qn 9a involved reasoning about the perimeter of a shape made from 6 equilateral triangles. Both items involved simple reasoning and were presumably within reach of the majority (data in this respect is not available). Reasoning items testing Data Handling Reasoning items involving Data Handling were pretty scarce. There were only three such items, all to be found in JL Items 12c and 12d were about making inferences from a straight line graph representing the cost of booking a holiday for different number of persons. The Chief Examiner s Report gives a nice example of how one pupil sorted this out (Educational Assessment Unit, 2005). Question 13b is intended to prompt reasoning about averages; certainly a very worthwhile cause from the perspective of mathematics education. On the other hand, the quality of the question suffers from being presented as a True/ False question. Concluding Remarks This section includes the main findings as well as suggestions for improvement of the 11+ examinations in Mathematics. The comparison of the content in the Primary syllabus with the TIMSS 2007 content domain at Grade 4 level suggests that there is no scope for a reduction of the Mathematics Primary syllabus. For all the examinations considered, it is clear that the paper setters were very careful to use graded questions and to choose language that is clear, succinct and to the point. The diagrams were also clear and unambiguous throughout. In write-on examinations, the space allowed for working each item sends a message as to the intended pupil response. When estimating such space, the paper-setters need to take into account the longest rather than the shortest possible methods that the pupils might use. This was not always the case. For complex items, it is also necessary to provide extra space for rough work. The approach taken by the CE examinations to encourage pupils to use the blank pages for rough work is a very sensible approach. There were very few items requiring only a True/False response. Such items are considered to be lacking in quality because they can easily be answered correctly through guessing. There was only one item requiring a simple short written explanation rather than a numerical answer in the four papers reviewed. More such items should be included in this type of examination. This would reflect more importance to 297

298 the need for pupils to learn how to explain and justify their results; an area that is often ignored in the mathematics classroom. In all cases, the papers covered all the sub-sections of the syllabus. Undoubtedly, this reflects a careful use of the specifications grid in the construction of all the examination papers. There is scope for improving the specifications grid. As it is Problem Solving is considered with the other content domains and is allowed 20±1% of the total mark. In this way, no attention is given to ensuring that mathematical reasoning and communication are tested across all the syllabus content. The cognitive demand of the examination items should be considered as a separate dimension in the specifications grid. One example of how this may be done is clearly the method used in TIMSS 2007, described in this paper. All examination papers included a good number of items coded as involving Reasoning. These also reflected a good spread of item difficulty; ranging from very easy to very difficult items in each paper. The Reasoning items were not well spread out between the syllabus content sections. They were mostly concentrated on one subsection of the Number section, namely that named as The four operations involving whole Numbers. This generates a backwash effect in most classrooms; in the other content domains, skills and procedures are stressed at the expense of understanding. Working with fractions is a different matter, and much more complex, than working with whole numbers. In the Primary years, rather than learning a set of unconnected rules and procedures for working with fractions, it is important that pupils start building an understanding of this topic. For this reason, it is important to include Reasoning items testing the simpler ideas about fractions in mathematics examinations at this level. The 11+ examinations need to be aligned to the curriculum. This academic year ( ), an updated version of the syllabus is in place. The rationale of this syllabus exhorts teachers to avoid carrying out only tests which focus on a narrow range of skills such as the correct application of standard algorithms (procedures). (Curriculum Department, 2006, p.11). This will certainly not happen in content areas where the high-stakes examinations themselves, the 11+ examinations, focus on a narrow range of skills. Good quality Reasoning items are much more difficult to construct than those testing skills, procedures and routine problem solving. Moreover, once used on a number of occasions, teachers may find ways to teach their pupils tricks to enable them to answer the items successfully. Once this happens, they are 298

299 no longer Reasoning items. The analysis carried out in this study reflects some expertise, among the paper-setters involved, in constructing good quality Reasoning items in the syllabus area of The four rules applied to whole numbers. Moreover, it suggests that they may need to develop this expertise for the other areas of the syllabus. Questions that may be interpreted as testing division by decimals should be avoided. Otherwise, teachers get the message that they had better teach the algorithm for division by decimals, when they can use time better consolidating their pupils understanding of simpler tasks on decimals. The published annual Chief Examiners Reports for JL entrance examinations in mathematics are a valuable resource for teachers as well as paper setters. As compared to the 2006 such report, the 2005 Mathematics Chief Examiner s Report is much more detailed as to comments about the individual examination questions and gives more insight as to pupils methods for answering a selected number of questions (Educational Assessment Unit, 2005; Educational Assessment Unit, 2006). The publication of the facility of the individual items in the JL entrance examination annual report is likewise a commendable practice. This data is a very useful resource for both Primary teachers and paper setters. 299

300 References Curriculum Department. (2006). Mathematics Syllabus for Primary Schools: Rationale. Retrieved December 20, 2006, from Curriculum Management Department (1997). Syllabus for the Primary School: Years 4, 5 and 6, Malta: Education Division, Ministry of Education and National Culture. Educational Assessment Unit (2005). Entrance Examinations into the Junior Lyceums 2005, Malta: Department for Curriculum Management, Education Division, Ministry of Education, Youth and Employment. Educational Assessment Unit (2006). Entrance Examinations into the Junior Lyceums 2006, Malta: Department for Curriculum Management, Education Division, Ministry of Education, Youth and Employment. Mullis, I.V.S., Martin, M.O., Ruddock, G.J., O Sullivan, C.Y., Arora, A. & Erberber, E. (2005). TIMSS 2007 Assessment Frameworks. Chestnut Hill, MA: Boston College. Retrieved December 20, 2006 from National Council of Teachers of Mathematics (NCTM). (2000). Principles and standards for school mathematics. Reston VA: Author. Neidorf, T.S., Binkley, M., Gattis, K. & Nohara, D. (2006). Comparing Mathematics Content in the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment (PISA) 2003 Assessments (NCES ). U.S. Department of Education. Washington DC: National Center for Education Statistics. Retrieved December 20, 2006 from Romberg, T.A., & Kaput, J.J. (1999). Mathematics worth teaching, mathematics worth understanding. In E. Fennema & T.A. (Eds.), Mathematics Classrooms That Promote Understanding. London: Lawrence Erlbaum Associates, Publishers, pp Schmidt, W.H., Wang, H.C. & Mcknight, C.C. (2005). Curriculum coherence: an examination of US mathematics and science content strands from an international perspective, Journal of Curriculum Studies, Vol. 37, No. 5, Swan, M. (2005). Improving learning in mathematics: challenges and strategies. Department for Education and Skills Standards unit. Retrieved April 30, 2006, from Van den Heuvel-Panhuizen, M. (2002). Realistic Mathematics as work in progress. In F. L. Lin (Ed.), Common Sense in Mathematics Education, Proceedings of 2001, The Netherlands and Taiwan Conference on Mathematics Education (pp. 1 43). Taipei, Taiwan. Retreived December 20, 2006, from 300

301 Appendix 6a Coding of Mathematics Items Junior Lyceum Exams

302 302

303 Junior Lyceum Exams

304 304

305 Common Entrance Exams

306 306

307 Common Entrance Exams

308 308

309 Appendix 7 Report on the Religion Examination Papers for Admission into State Junior Lyceums and Church Secondary Schools Dr. Adrian-Mario Gellel March

310 Background Executive Summary Although Religious Education is taught in almost all European countries, each country has its own model and method. In Malta, Catholic Religious Education is guaranteed by the state through various legislations since it is believed that it contributes to the holistic formation of the person, Religion and Social Studies were included in the Junior Lyceum examination Analysis to ensure that pupils receive a more appropriate education and to reduce the pressure and the exaggerated cramming that used to take place in the other three subjects. Although there is the intention of helping students grow in their affective dimension, neither the textbook nor the examination papers cover this dimension adequately, There is too much emphasis on the cognitive side of content with an accent on the importance of the ability to recall rather than on the ability to understand, analyse and evaluate. This is especially so in the CE papers a below summarized in the table. General description of six papers Paper N o of Pg s. N o of Se ct. N o of Items % Marks allotted to Cognitive Items % Marks allotted to Affective Items T 1 2 T JL CE JL CE JL CE essays short essay short essays Note 1: In the Cognitive Items Column 1 = Remember, 2 = Understand, 3 = Apply and 4 = Analyse. Note 2: In the Affective Items Column 1= Receive and 2 = Respond. Note 3: Where the sum of % of Cognitive and Affective items exceed 100% students answer could be evaluated in more than one way 310

311 Conclusions There is a problem with the language and with the clarity used to describe the learning outcomes of the syllabus, The textbook s content, and not syllabus objectives, determine the content of the papers, The content presented in the textbook is inadequate. There is too much emphasis on one theme (the Eucharistic liturgy) and insufficient material for teachers and students to develop more on the recommended themes and learning outcomes There is too much emphasis on recall type items. It is evident that students are being encouraged to learn by heart. Candidates are not challenged and stimulated to show what they have learnt nor can examiners evaluate what candidates have truly learnt (assimilated). There is not standard format model through which papers are presented. This leads to discrepancies in the amount and type of sections, items. Recommendations It is important that Religion continue to form part of the assessment process of students finishing primary schooling. This assures holistic education. The identity and nature of the teaching of Religion in Schools at Primary level needs to be specified. The needs of the child, those of society and those of the National Minimum Curriculum require that we teach Religious Education and not Religious Knowledge. Objectives need to be clarified and specified. In particular there needs to be more balancing amongst the different categories of objectives, encouraging students to move beyond recall of content and helping them construct meaning, apply it to their daily lives, analyse and be aware of certain values and possibly be committed to them. The content needs to reflect these objectives. At present the content is limited and does not challenge students nor does it help them grow holistically. There is a need to help students understand religious language, and more importantly to contribute to helping them use and construct religious language. Furthermore, there is a need to help students understand the basic values and message of Christianity and help them see their relevance and importance in everyday life. The examination papers should be constructed on the objectives presented in the syllabus and not on the content of the textbook. 311

312 Papers need to have a higher percentage of marks that assess the affective domain and assess whether students are able to apply, analyse and evaluate. The papers should be presented in such a way as to lessen cognitive stress by using titles to describe the content covered by the exercise, use more images, and less text, The exercises should be more balanced, taking into consideration the candidates individual difference in learning and expressing themselves. The similarity between the year six annual examination and 11+ examination papers point to the possibility of having one exam that assesses students learning. This may create some bureaucratic difficulties of how to integrate the CE and JL papers. 312

313 Table of Contents Executive Summary Table of Contents List of Figures List of Tables Context The Religion Syllabus The Examination papers General description of the papers 3.2. JL 2004 paper 3.3. JL 2005 paper 3.4. JL 2006 paper 3.5. CE 2004 paper 3.6. CE 2005 paper 3.7. CE 2006 paper 4. Student Performance Student Performance on JL papers 4.2. Student Performance on CE papers 5. Recommendations List of Figures Figure 1 An alternative image for depicting the Eucharist at the centre of one s activities List of Tables Table 1 Table 2 General Theme 6.1. This is God s Message: He loves us Learning Outcomes and corresponding Chapters in Religion Textbook Table 3 General description of the six papers 313

314 Table 4 Analysis of Junior Lyceum Religion Examination Paper 2004 Table 5 Analysis of Junior Lyceum Religion Examination Paper 2005 Table 6 Analysis of Junior Lyceum Religion Examination Paper 2006 Table 7 Analysis of Common Entrance Religion Examination Paper 2004 Table 8 Analysis of Common Entrance Religion Examination Paper 2005 Table 9 Analysis of Common Entrance Religion Examination Paper 2006 Table 10 Mapping of Knowledge domains covered by items onto examiner s aims Table 11 Percentage of passes per subject and per gender Table 12 Levels of difficulty and discrimination per paper Table 12a Levels of difficulty per paper Table 12b Levels of discrimination per paper Table 13 Percentage of candidates that obtained more than 75% and 50% of the marks Table 14 Percentage of students who finished 314

315 1. Context Although Religious Education (R.E.) is taught in almost every country of the Western World, there is no one single model. For instance, while the U.K. has a long tradition of multi-faith R.E., Italy, Austria and some Landers in Germany, amongst others, have a long tradition of Catholic R.E. Nonetheless, the models, the modality and even the content depend very much on the history, culture, religious tradition/s and politics of the country and, at times, even of a region. 31 Whilst educational models and systems derive their principles from general theory, they are always deeply contextual. This is true both for the curricular subject but also for the assessment methods that are employed. In Malta, Catholic R.E. in State Schools is guaranteed by the Constitution and by the Education Act. 32 Furthermore, the Treaty between the Holy See and the Republic of Malta, apart from reiterating that Catholic R.E. should be provided at all levels of compulsory schooling, also specifies that it is the responsibility of the Maltese Episcopal Conference to establish the teaching methods programmes, and texts for students. 33 It is, however, the Education Officer of Religion within the Division of Education who is to promote the subject and to implement the policies of the Maltese Episcopal Conference and the Division. 34 At the Primary level, the goal of Catholic R.E. is to help students become aware of the love of God for human beings. Consequently it promotes the understanding and practice of natural law and induces students to love God and every creature. 35 The six textbooks covered during primary schooling endeavour to reach this goal by focusing on the following topics: 1. God 2. God loves me 31 See Peter R. HOBSON and John S. EDWARDS, Religious Education in a pluralist society. The key philosophical issues, London: Woburn press pp ; see Flavio PAJER, L istruzione religiosa nei sistemi scolastici europei: verso una funzione etica della religione nella scuola publica, in Seminarium XLII/2 (2002). pp ; See Flavio PAJER, Multifaith education in the Europe of tomorrow: a civic responsibility for universities and schools, in Bert ROEBBEN and Michael WARREN (eds.), Religious Education as Practical Theology, Leuven: Peeters pp ; see Peter SCHREINER, Religious Education in the European Context, in Lynne BROADBENT and Alan BROWN (eds.), Issues in Religious Education, London: Routledge Falmer pp See MALTA. HOUSE OF THE REPRESENTATIVES, Constitution of Malta, art. 2 (3); see MALTA. HOUSE OF THE REPRESENTATIVES, Chapter 327, Education Act, art 20 (2). 33 See Accordo tra la Repubblica di Malta e la Santa Sede per meglio ordinare l istruzione e l educazione Religiosa Cattolica nelle Scuole statali, in Acta Apostolicae Sedis art See Modes of Regulation on Catholic Religious Instruction and Education in State Schools, in Acta Apostolicae Sedis Section III, art. 3c. 35 See Dipartiment tal-kurrikulu, Reliġjon Is Sillabu tal-primarja, (online): docs/ syllab_pr_religionrationale.pdf [15 December 2006] 315

316 3. Jesus 4. The Church: the family of God 5. The Holy Spirit 6. Throughout the year with the Lord It is evident that when drawing up the syllabus for the sixth year and preparing the sixth book, the author/s wanted to synthesize the content of the previous five years and to help students, through liturgy, to find a concrete way of practicing and expressing what they had learnt. Religious Education has always been an examined subject in the last four classes of primary education. However it was not always included as part of the examination for admission in Junior Lyceums or even Church Schools. When on 13 th January 1988, the then Minister of Education announced that Religion, together with Social Studies, would be examined for entry into Junior Lyceum, he explained that the increase in the number of subjects from three to five was to reduce any obsession of coaching pupils at primary level in just three subjects (Maltese language, English language and Mathematics). 36 In an answer to a Parliamentary Question, the Minister clarified that the inclusion of Religion and Social Studies in the Junior Lyceum examination, should ensure that pupils receive a more appropriate education. The introduction of additional subjects was aimed at reducing the pressure and the exaggerated cramming that used to take place in the other three subjects. 37 Since 1988, the Religion examination paper has assessed the syllabus and the content covered during the sixth year of primary. 2. The Religion Syllabus The present year 6 syllabus is divided in 4 major themes i. God s message: He loves us ii. Jesus Christ is God s gift iii. Jesus gives us his Spirit in the Church iv. Building our Christian Character 36 See Junior Lyceum entrance exam ceases to be competitive, in The Times (Malta 18 January 1988). p See Question in Parliament, in The Times (Malta 4 March 1988). p

317 These themes are then divided into 11 learning outcomes each explained further through one or two sentences. The points identified by the document as being Learning Outcomes (miri) cannot be considered objectives; they are more themes that the student needs to cover. The brief further explanation given in the document explains, in rather generic language, the objectives that need to be reached by the end of the theme. While only the fourth major theme, Building a Christian character indicates that the content also needs to tackle the affective dimension of learning, in the further explanation of the Learning Outcomes one can deduce that the syllabus drafters expected pupils not only to achieve cognitive outcomes but also affective ones. Thus, for instance, in the general theme 6.1., Table 1, one notes that that there are both cognitive and affective objectives even though there seems to be an emphasis on the affective dimension. Table 1. General Theme 6.1. This is God s Message: He loves us 38 Learning Outcomes Notes Walk our way to the Father We become conscious of what God did through Jesus and we praise God Forming an idea of how God really is How can we come near to God Through creation we notice that God is great, wise powerful and wanted to make us his children and wanted to remain with us We become aware that God loves us and for this reason we worship him. We go to him through Christ, with Christ and in Christ and with him we worship God the Father. Through this theme, pupils are asked to, i. become conscious (insiru konxji), ii. praise (nagħtu ġieħ) iii. notice (naraw) iv. become aware (nindunaw) v. go to him (nersqu lejh) vi. worship (nagħtu qima) Only the first (which needs to be read in the whole context of the note) and third verbs point to the cognitive dimension of Remember. This alternating between cognitive and affective objectives is also found in the other three themes 38 Free translation 317

318 The syllabus is concretely expounded in the Textbook published by the National Catechetical Commission, Lejn il-missier 6: Mal-Mulej matul is-sena. The fact that there are three different editions might create difficulties to teachers and candidates. While all three editions are practically the same, the minor changes which were done in the text, mainly summaries, and reordering of the chapters, may create. The 2001 and 2004 editions are a cosmetic re-edition of the one that the Commission had published in The textbook practically covers the eleven learning outcomes listed in the syllabus as shown in Table 2. One notes that only Learning Outcome is not covered by the textbooks while the chapters 7 and 12 in the 2001 edition and chapters 13 and 23 in the 2004 edition are not in the present syllabus. With regards to Chapter 6 (2001) and Chapter 22 (2004) one should note that it only covers the Learning Outcome indirectly. Table 2: Learning Outcomes and corresponding Chapters in Religion Textbook Learning Outcomes Chapters 2001 edition edition Walk our way to the Father Forming an idea of how God really is 1,2,3 1,2, How can we come near to God 4, Christmas and Epiphany 8,9,10,11 9,10,11, Mary under the cross St. Paul s Feast During lent we change through 14,15 15, Liturgy changes our lives: The Eucharist (6),18,19,20,21 4,5,6,7,8,23, Sacraments change us Appendix Appendix The great feast: We live what we celebrate During Easter we rise with Christ 17 22,23,24 When comparing the present syllabus with the one published in 1997 one notes that that there is great similarity. The 1997 syllabus is divided in five major themes, i. Who is the God we want to get close to ii. iii. iv. Getting closer to God The great events in the life of Jesus that bring us closer to God throughout the year Our participation in the Eucharist and its relation with our lives v. We live what we celebrate 39 The 2001 edition will be used throughout the text when commenting on the papers. 318

319 The themes are introduced by a short paragraph. Each theme is further explained by another paragraph. The paragraphs point almost directly to the chapters in the textbook. In this case all chapters are represented in the syllabus. Although the language used in the 1997 syllabus is still vague, since it only gives a verbose description of the topics that need to be covered, it is a better guide to teachers and pupils alike. Furthermore, after examining the syllabi and the two versions of the textbooks, it is possible to conclude that the textbooks were written prior to the learning objectives, thus much more weight is given to the content of the textbooks. 3. The Examination Papers For the purpose of the present exercise, the past three Religion papers set for the Junior Lyceum (JL) examination and the past three papers set for the Private School Association (CE) will be analysed General description of the papers Both CE and JL examination papers are in Maltese, although according to the JL examiner s reports and the Form 1 Common Entrance Examination Regulations candidates may choose to be examined in English. Very few candidates, 4 and 5 in 2004, 8 and 6 in and 5 in 2006 have in fact availed themselves of this option. 40 The language used is simple and there is no particular gender bias or incorrect use of the language. None of the papers has a rubric. This leads one to assume that all questions have to be answered. Candidates have one and a half hours to answer all questions. However, there is no rule with regards to the number of sections or number of questions within each section which candidates have to answer. Similarly, there is no fixed pattern in the presentation or type of sections. Table 3 shows that the CE papers are more demanding in the amount of content that candidates need to know and in the amount of writing expected of candidates. While on average, in these past three years, a JL paper had 65 items that covered 60% of the textbook chapters, a CE paper had 150 items and 2 short essays which covered 82.6% of the textbook chapters. The Eucharist and the celebration of the 40 See Education Division Junior Lyceum entrance Examinations Report, Floriana: Education Assessment Unit 2004; Education Division Junior Lyceum entrance Examinations Report, Floriana: Education Assessment Unit 2005; Education Division Junior Lyceum entrance Examinations Report, Floriana: Education Assessment Unit 2006; Charles Mallia, personal communication (17 March 2007). 319

320 Mass were assessed in all papers. Christ s mediating role between God the Father and humanity (learning outcome ) is the least assessed. On the other hand, it should be noted that the veneration of Saints (Chapter 7) and the Church (Chapter 12), which are not mentioned in the new 2005 syllabus, were still assessed in the 2006 papers. Although the syllabus aims at educating pupils in both cognitive and affective dimensions, it is evident that the papers are designed to assess the cognitive content. Table 3. General description of the six papers Paper JL 2004 CE 2004 JL 2005 CE 2005 JL 2006 CE 2006 N o of Pgs. N o of Sect. N o of Items % Marks allotted to Cognitive Items % Marks allotted to Affective Items T 1 2 T essays short essay short essays Chapters in textbook (2001) 3,6,11,13,15-17, 19-22, App 1-4,6,9,12-13,15,17, 19-21, App. 1-3,6-9,11-15, App. 1-4,6-8,12-13,15,19-21, 24, App. 1-15,17-21, 23- App. Note 1: In the Cognitive Items Column 1 = Remember, 2 = Understand, 3 = Apply and 4 = Analyse. Note 2: In the Affective Items Column 1 = Receive and 2 = Respond. Note 3: Where the sum of % of Cognitive and Affective items exceed 100% students answer could be evaluated in more than one way Moreover, when these examination papers are investigated through Bloom s taxonomy of educational objectives, it transpires that they mainly assess the lowest levels of the cognitive process. 41 When compared to each other, it becomes apparent that the JL papers require more ability to transfer knowledge 41 See Lorin W. ANDERSON and David R. KRATHWOHL (Ed.) 2001 A Taxonomy for Learning, Teaching, and Assessing. A revision of Bloom s Taxonomy of Educational Objectives, New York: Kongman. 320

321 than the CE ones. Whilst there is consistency in this regard in JL papers, with 33% of the weight given to items assessing understanding, the CE papers are mainly intended to assess candidate s retrieval capacity. It was only in 2005 that the CE paper assessed recall as much as it assessed the candidates understanding of the material. It should also be noted that it is frequent for paper setters to use the same language, and at times even the same words, used in the Religion textbook. This encourages candidates to learn by heart whole extracts without making any sense out of them. Affective content is only minimally assessed. These items target to assess candidates sensitivity towards certain values. There was only one instance (CE 2005 paper) that the answers of the candidates could have shown that they are somewhat committed to these values. Even in this latter case, candidates answers could have equally shown that they tolerate certain ideas without showing any commitment to them. Indeed, almost all items that could to some degree assess the affective domain could be answered in such a way that shows candidates ability to just remember an idea. This said, it must also be acknowledged that it is difficult to assess candidates affective capabilities. Much also depends on the way students have been educated and on the development of better assessing instruments JL 2004 paper This seven-page paper focuses mainly on content normally covered during the second and third scholastic terms. On the whole the distribution of marks is fair and only a few items are given less weight than they should. Items requiring application and to some lesser extent understanding should have been given more marks than those items requiring only recall. It appears that the paper setters decided to give one mark for every correct answer irrespective of the processing required. Nonetheless my overall evaluation in this regard is positive since there are only very few items which require more than basic cognitive capability. It would have been helpful if the marks allotted to each item were shown on the question paper. The paper setters included a number of images, all of which computer graphics, but these images were not always able to balance the text. Two of the five pages did not have images at all while in another two pages the images used were very small. It is good to note that in most cases the images used were directly related to the questions asked and thus were intended to help the candidates to visualise content through the use of symbols. For instance, the use of images in section D was very well used. The graphical depiction of the sacraments must have surely helped students, especially those inclined to use a visual/imager style, to give a description of what happens in the sacrament. The paper setters initiative to include graphics that is presumably locally designed is to be commended. With regards to section F, it is good to note that whilst the quality of the graphics used is not of a high level, it was equally effective in helping the candidates to remember the narrative that they had to write. 321

322 With the exception of the instruction given in Section D, the language used for the formulation of the instructions at the beginning of each section is clear and to the point. From the instruction given, it is not clear whether Section D is meant to assess the candidates knowledge of what physically happens during the celebration of the sacrament or the spiritual effects of the sacrament. For instance, a candidate could have easily and correctly answered that in the administration of the sacrament of baptism, the baby is immersed in water and that s/he become a member of the Family of God. Such deficiency is also recognised by the paper setters who, in their marking scheme, indicate that both types of answers are correct. A more detailed description and further comments are provided in Table 4. Table 4. Analysis of Junior Lyceum Religion Examination Paper 2004 Secti on & Type of Q. A Fill in the blanks (words Q. Conte nt found in Chapt er 11 Cognitive Remember 1 (x10) Domain Level Mark T. Comment 10 Some of the answers are provided in the same paragraph Short answers Match B 1 17 Cognitive Remember 1 The answer to this item is provided in the instruction introducing the section 2 17 Cognitive Remember Cognitive Understand Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 10 Ċ 2 19 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember

323 Write two sentences Comprehension D Remember 1 3 Cognitive OR 1 Understand 2 11 Cognitive Remember OR 1 Understand Cognitive Remember OR 1 Understand 4 15 Cognitive Remember OR Understand 1 Remember 5 Cognitive OR 1 10 Understand E 1 Cognitive Remember 1 F Short par. Comprehension One sentence 2 Cognitive Remember 1 3 Cognitive Understand 1 4 Cognitive Understand 2 Cognitive Understand 5 1 Affective Receive 6 Cognitive Understand 3 7 Cognitive Affective Apply Receive 13 Cognitive Understand (x10) Ġ 1 Cognitive Remember 1 2 Cognitive Understand 1 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 9 Cognitive Remember 1 10 Cognitive Apply 1 10 G 1 Affective Receive 1 2 Affective Receive 1 3 Affective Receive 1 4 Affective Receive 1 5 Affective Receive 1 6 Affective Receive 1 7 Affective Receive 1 8 Affective Receive 1 10 What is assessed in this section is not present in the learning outcomes of the R.E. syllabus This question is not related to the theme of the section. There is no flexibility or allowance for creativity. The paper setters could have easily avoided giving the ten words and compel students to use them in their short essay. What is assessed in this section is not present in the learning outcomes of the R.E. syllabus 323

324 9 Affective Receive 1 10 Affective Receive 1 10 GĦ 2 Cognitive Apply 2 3 Cognitive Apply 2 4 Cognitive Apply 2 5 Cognitive Apply 2 Write a prayer 6 Cognitive Apply 2 10 H 1 Cognitive Remember 5 2a Cognitive Remember 1 b Cognitive Understand 1 c Our Cognitive Understand 1 d Father Cognitive Understand 1 e Cognitive Understand JL 2005 paper Contrary to the previous year, there is a fair representation of the content covered during the last year of primary school. This paper requires candidates to mainly retrieve knowledge from their long-term memory. On the whole, the distribution of marks is fair and only a few items are given less weight than they should. Compared to the pervious year there are less images, of which only those used in the last section are intended to help candidates recall information. On the other hand, all instructions are clear and to the point. A more detailed description and further comments are provided in Table 5. Secti on & Type of Q. Table 5. Analysis of Junior Lyceum Religion Examination Paper 2005 Q. Conte nt found in Chapt er Domain Level Mark T. Comment A 1 1 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 Multiple choice (candidat 5 2 Cognitive Remember 1 Does not assess Religious knowledge 324

325 Short answers 6 2 Cognitive Remember 1 Does not assess Religious knowledge but the correct use of the Maltese language. Furthermore, the statement is theologically incorrect. It is not only the human soul that is created in the image of God but the whole human person. (CCC 362) 7 2 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 10 B 1 18 Cognitive Remember Cognitive Remember Cognitive Understand Cognitive Remember 3 Not enough space provided 5 21 Cognitive Remember 6 15 All the congregation recite the Our Father and not just the priest Ċ 1 6/8 Cognitive Remember 2 2 6/11 Cognitive Remember Cognitive Remember 2 Complete the sentence 4 6/13 Cognitive Remember 2 5 6/14 Cognitive Remember 2 6 6/18 Cognitive Remember Cognitive Remember 2 8 6/23 Cognitive Remember 2 9 6/7 Cognitive Remember Cognitive Remember 2 20 D 2 18 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 Fill in the blanks with words that are provided 9 18 Cognitive Remember Cognitive Remember Cognitive Remember 1 10 E 1 Cognitive Remember 3 2 Cognitive Remember Cognitive Remember 2 Compr ehension 4 Cognitive Understand Cognitive Understand 3 10 F 1 8 Cognitive Remember Cognitive Remember Cognitive Understand 1 Com preh - 4 Cognitive Remember 1 These statements were taken almost word for word from p.95 of the student textbook 325

326 5 Cognitive Understand 2 6 Cognitive Understand Cognitive Understand 3 15 Ġ 1 21 Cognitive Understand Cognitive Understand Cognitive Understand 1 Fill in the blanks G Brief par Cognitive Understand Cognitive Understand 1 5 Cognitive Remember 15 Whilst the instruction Understand specifies that the Affective Receive questions provided are intended as an extra help, the marking scheme gives the impression to examiners that the candidates had to answer each question. Furthermore the marking scheme does not specify how marks should be allotted JL 2006 paper This is the shortest of the three JL examination papers. The instruction given at the beginning of each section is clear and to the point. Similarly the paper setters were fair in their distribution of marks. It should be noted that with the publication of the new syllabus (2005) the assessment of items which refer to content that is covered by chapters 7 and 12 of the 2001 textbook is an inappropriate oversight. With regards to the layout of the paper, only 5 images were used in this paper and although they were to some extent related to the content assessed they were not intended to help students in their processing. When evaluating section A, one could argue that some items assessed more than just memory as they also require candidates to demonstrate their ability to understand the content by putting the statements given in order of truth. However, one immediately notes that the language used in the formulation of these statements is very similar to the one used in the book and thus candidates were only required to recall the relevant text. Furthermore, it is noted that four of the ten items were also examined in Section A of the previous year. On a positive note, it was good to note that student were left free to choose from amongst three themes about which they had to write 10 sentences of free text. Another positive development over the previous years is the fact that candidates were not given any words or phrases which they had to use and that would have limited them in some way in the development of their comprehension. However, the 326

327 marking scheme does not explain how and on what basis examiners were to allot marks to students. A more detailed description and further comments are provided in Table 6. Secti on & Type of Q. Multiple choice (candidates to choose amongst 4 possibilities) Table 6. Analysis of Junior Lyceum Religion Examination Paper 2006 Q. Content found in Chapter Domain Level Mark T. Comment A a 1 Cognitive Remember 1 Technically both statements iii. and iv are correct. We know statement i. from our parents and the believing community. b 2 Cognitive Remember 1 ċ 2 Cognitive Remember 1 d 2 Cognitive Understand 1 All statements are correct. Furthermore, the second statement is only partially correct since it is the human person as a whole that is created in the image of God not just the human soul e 2 Cognitive Understand 1 All statements are correct and one could even argue that i., ii. and iii. are of equal value f 4 Cognitive Remember 1 All statements are correct. It is only the word truly that qualifies the phrase. ġ 1 Cognitive Remember 1 g 2 Cognitive Understand 1 għ 4 Cognitive Remember 1 h Cognitive Analyse Technically there is no best answer - it all depends on what the students did during their R.E. lessons B a 24 Cognitive Apply 4 b App. Cognitive Understand 4 The questions are only marginally related to ċ App. Cognitive Understand 2 the story d App. Cognitive Remember 2 e App. Cognitive Understand 4 16 Ċ a 8 Cognitive Remember 1 Comprehension 327

328 Comprehension b 15 Cognitive Affective Understand Receive ċ 15 Cognitive Remember 6 d 15 Cognitive Remember 6 e 3 Cognitive Remember 3 6 The content assessed is not present in the learning outcomes f 3 Cognitive Remember 3 25 D a 19 Cognitive Remember 1 b 19 Cognitive Remember 1 ċ 19 Cognitive Remember 1 d 19 Cognitive Remember 1 e 20 Cognitive Remember 2 The word konsagrazzjoni was Fill in the blanks with words that are provided f 20 Cognitive Remember 2 ġ 21 Cognitive Remember 2 10 E a 6 Cognitive Remember 2 b 6 Cognitive Remember 2 Identify the event celebrated F give e.g. Fill in the blanks ċ 6 Cognitive Remember 2 d 6 Cognitive Remember 2 e 6 Cognitive Remember Cognitive Affective Apply Receive 4 4 Ġ a 13 Cognitive Remember 1 b 13 Cognitive Remember 1 ċ 13 Cognitive Remember 1 d 13 Cognitive Remember 1 e 13 Cognitive Remember 1 f 13 Cognitive Remember 1 ġ 13 Cognitive Remember 1 g 13 Cognitive Remember 1 għ 13 Cognitive Remember 1 h 13 Cognitive Remember 1 10 G a 7 Cognitive Understand Brief par. (to chose 1) b 12 Cognitive Understand ċ 19 Cognitive Understand probably too difficult for candidates 3.5. CE 2004 paper Apart from including the time allotted to the paper, the rubric included a statement that candidates can answer in either Maltese or in English. Giving candidates the opportunity to answer in Maltese and/or in English is positive although students still need to know Maltese since the paper is in Maltese. Nonetheless, this feature surely does help candidates who normally speak in English to express themselves better. 328

329 The paper includes a title to each section indicating the topic that is being assessed. This probably helps to reduce candidates stress in that the section title prepares them mentally for the questions on that particular topic. This should result in a better performance stressing the examination. According to research conducted by Riding and his associates, headings enhance recall, especially for males and females with a Wholist Cognitive style. 42 The paper is particularly long. Although there are 11 sections, only one section more than the JL 2004 paper, there are in fact 19 exercises. Even if this paper has more items than other CE papers it was still fair to expect candidates to complete it in the same time since it did not request any essay from candidates. The paper puts a lot of emphasis on recall. 88% of the marks allotted are devoted to items that demand candidates to remember the text. However, the distribution of marks is fair. The low level of difficulty of the exercises compensates for the length of the paper. The language used for giving instructions is clear and to the point. Similarly the language used is simple with no particularly difficult terms. However, it is not always clear what some items are really assessing. Sometimes it is Maltese language, sometimes it is just candidates ability to learn by heart. For instance, in section Ħ, exercise 1, it is not clear what the second sentence is really assessing. No Religious Knowledge can be said to have been demonstrated by the candidate who correctly inserts the word żgħażagħ (youth) in the sentence, There are children, youth and adults, all gathered around the altar. 43 Likewise, there is no sense in a student demonstrating that he has learnt by heart that Jesus is the bridge or ladder between us and the Father. The phrase, that is taken almost word for word from the textbook, is of course only a metaphor. It would have been more appropriate if the paper setters asked the candidates in what way Christ mediates between human beings and God. However, it must be said that there are relatively only a few of such instances. On the other hand, one should also note that these instances occur because there is too much emphasis on the lower category of the cognitive domain. There is a limit to how many recall items one can produce from a specific text. Very few images are used in this paper. Whilst the use of the graphics is related to the topic of the particular section, one should expect that these be chosen more carefully. The graphic on page 9 is inadequate for two main reasons. The tonsure on the priest s head and the celebration of the mass on the high altar and not on the altar table, indicate that the Eucharistic celebration is in fact a pre- Vatican Council liturgical rite. Whilst the meaning of the graphic is self-evident, the images used are not close to the candidates everyday life experience. This is due both to the drawing s referral to an old rite and to the style of the same drawing. Furthermore, the graphic depicts only males present in the celebration The Wholist-Analytic Cognitive Style dimension describes whether the individual tends to organise information in wholes or in parts See Richard J. RIDING and Saleh AL-SANABANI, The effect of cognitive style, age, gender and structure on the recall of prose passages, in International Journal of Educational Research 29 (1998). pp ; see RIDING and WATTS (1997). pp Free translation 329

330 Conversely, Figure 1. is an example of how an image can better convey the same meaning intended by the graphic on page 9 by using photos from children s everyday life (studying, praying, playing, and helping each other), with the celebration of the Eucharist at the centre of their activities. In this case, attention was also given to balance in the use of both males and females. A more detailed description and further comments are provided in Table 7. Figure 1: An alternative image for depicting the Eucharist at the centre of one s activities 330

331 Table 7. Analysis of Common Entrance Religion Examination Paper 2004 Secti on & Type of Q. Q. Conte nt found in Chapt er Domain Level Mark T. Comment A 1 2 Cognitive Remember Cognitive Remember 1 The statement is theologically incorrect 3 Cognitive Remember Cognitive Remember 1 The answer to this item is to be found in the second item 5 2 Cognitive Remember Cognitive Remember 1 All three answers are equally correct 7 3 Cognitive Remember Cognitive Remember Cognitive Remember 1 Is the place where baptism occurs important? 10 3 Cognitive Remember 1 Multiple choice (candidates to choose amongst 3 possibilities) 11 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 18 B 1 1 Cognitive Remember 1 Fill in the blanks with a word that is provided Match 2 1 Cognitive Remember 1 3a 1 Cognitive Remember 1 3b 1 Cognitive Remember 1 4a 1 Cognitive Remember 1 4b 1 Cognitive Remember 1 5a 1 Cognitive Remember 1 5b 1 Cognitive Remember 1 6a 1 Cognitive Remember 1 6b 1 Cognitive Remember 1 10 Ċ 1 4 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 8 Cognitive Remember 1 These items do not test Religious Knowledge It is most probable that candidates found it hard to prioritise unless they had learnt the phrases by heart 331

332 D Fill in the blanks 9 2 Cognitive Remember 1 There is a misprint. The correspondent statement marked as g instead of għ 10 App Cognitive Remember Cognitive Remember 1 (x30) E 1 Cognitive Remember 1 To determine whether the statement is right or 2 Cognitive Remember 1 3 Cognitive Remember 1 4 Cognitive Remember 1 5 Cognitive Remember 1 6 Cognitive Remember Cognitive Remember 1 7 F 1 6 Cognitive Remember 2 To identify the person referred to in the statement 2 9 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 2 20 Ġ 1 Cognitive Remember 1 Comprehension Fill in the blanks 2 Cognitive Understand 1 3 Cognitive Remember 1 4 Cognitive Understand 1 5 Cognitive Remember 1 5 G 1 15 Cognitive Remember 1 GĦ Fill in the blanks 2 15 Cognitive Remember Cognitive Remember 1 30 There is no flexibility. It is evident that the exact word is expected Since the phrase does not identify who is stating it,it is difficult to determine whether it is right or wrong The statement could be both right or wrong 4 15 Cognitive Remember Cognitive Remember Cognitive Remember 1 10 It is not always clear (x 10) what the exercise is assessing GĦ 2 17 Cognitive Remember 1 Comprehensio n through Multiple Choice 3 17 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 332

333 Identify the correct sacrament Multiple Choice 9 17 Cognitive Remember Cognitive Remember Cognitive Remember 1 10 H i App Cognitive Remember 1 Assoc -iate Ħ Fill in the Blanks Re-order ii App Cognitive Remember 1 iii App Cognitive Remember 1 iv App Cognitive Remember 1 Candidates are given a too obvious a hint regarding what they are expected to answer v App Cognitive Remember 1 vi App Cognitive Remember 1 vii App Cognitive Remember 1 Candidates are given a too obvious hint regarding what they are expected to answer viii App Cognitive Remember 1 ix App Cognitive Remember 1 x App Cognitive Remember 1 10 Candidates are given a too obvious hint regarding what they are expected to answer 2i App Cognitive Remember 1 ii App Cognitive Remember 1 The exercise takes iii App Cognitive Remember 1 sometime to finish iv App Cognitive Remember 1 since candidates v App Cognitive Remember 1 would probably have vi App Cognitive Remember 1 to recite the prayer vii App Cognitive Remember 1 continually while writing it in order to viii App Cognitive Remember 1 remember it ix App Cognitive Remember 1 x App Cognitive Remember Cognitive Remember 2 10 (x5) 1 19/ Cognitive Remember / (x5) 21 2i 19/ Cognitive Remember 1 ii 20/ Cognitive Remember 1 21 iii Cognitive Remember 1 iv Cognitive Remember 1 V Cognitive Remember 1 vi Cognitive Remember 1 vii Cognitive Remember 1 viii Cognitive Remember 1 ix Cognitive Remember 1 x Cognitive Remember

334 Fill in the Blanks To identify what helps communion Comprehension Identify whether it is right or wrong 3 19/ 20/ 21 Cognitive Remember 1 (x5) 4i Affective Receive 1 ii Affective Receive 1 iii Affective Receive 1 iv Affective Receive 1 v Affective Receive 1 vi Affective Receive 1 vii Affective Receive 1 viii Affective Receive 1 ix Affective Receive 1 x Affective Receive i Cognitive Remember 1 ii Cognitive Remember 1 iii Cognitive Understand 1 iv Cognitive Apply 1 v Cognitive Remember 1 5 6i Affective Receive 1 ii Affective Receive 1 iii Affective Receive 1 iv Affective Receive 1 v Affective Receive 1 vi Affective Receive 1 vii Affective Receive 1 viii Affective Receive 1 ix Affective Receive 1 x Affective Receive CE 2005 paper This ten-page paper requires candidates to equally show the ability to retrieve information from their long-term memory and to show some degree of understanding. Compared to the other five papers, this paper provided candidates with more possibilities to show their awareness of, or sensitivity towards certain ideas. Moreover, it is also the only paper that gives candidates the possibility to show that they are committed, to a certain degree, to certain attitudes and/or values. Overall, there is a fair distribution of marks. The absence of a marking scheme makes it difficult to determine in what way examiners allotted marks in section IE, where candidates were required to write a short composition. 334

335 While the presentation of the text is less cluttered when compared to that of the pervious year, it is a pity that the section headings were not included. The language used for giving instructions is generally clear and to the point, except for the instruction in section E which could in fact be misleading. Candidates were asked to write whether the statement is right (Tajba) or wrong (Ħażina). Since the statements were written in the first person singular, and keeping in mind that the words tajba and hazina also mean correct and incorrect, candidates could have easily interpreted that the exercise was seeking to verify whether they themselves act in the way described by the statement. For instance, writing tajba next to the statement Immur il-quddies darba fix-xahar biss could mean that the statement is correct because it is true for the candidate. The instruction should have specified that what was wanted from the candidates was for them to state whether it is right or wrong to act in the way described by the statement. Secondly, the statements should never be written in the first person singular as in a questionnaire or survey. There are only four computer-generated graphics on this ten-page paper. These visuals were well-chosen and appropriately placed, particularly the three images put next to the essay section. The visuals of the nativity, messianic entry in Jerusalem and the Passion help candidates, mainly those with a visual style, to use their imagination and be more descriptive. It is a pity that the examiners did not include more graphics which would have been more helpful to the candidates. Asking candidates to write three short essays by picturing a particular event and by taking upon themselves the role of a one of the non-human characters is to be commended. Apart from demonstrating their knowledge of the event, this exercise not only asks candidates to retrieve factual knowledge, but it also gives them the possibility demonstrate that they have understood its importance, that they can interpret elements that constitute the narrative, and that they can re-order information and construct a meaning. Furthermore, it also gives them the possibility to show that they can sympathize with the characters in the narrative, that they are aware of certain values, such as charity and love, and that they are committed to some measure to these values by giving an illustration of what they did for the character. For instance, in the nativity narrative, candidates, imagining themselves to be the donkey or the cow, could have explained that they stayed near baby Jesus as much as possible so that he would not feel cold or that they were happy that he was staying in their manger. 335

336 Table 8 gives a more detailed description and additional comments with regards to the individual items. Table 8. Analysis of Common Entrance Religion Examination Paper 2005 Secti on & Type of Q. Multiple choice (candidates to choose amongst 3 possibilities) Fill in the blanks with a word that is provided Q. Conte nt found in Chapt er Domain Level Mark T. Comment A 1 6/8 Cognitive Remember Cognitive Remember 1 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember /21 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 20 B 1 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 5 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 10 Ċ 1 Cognitive Understand 1 Comprehensio n + brief answers 2 Cognitive Remember 1 3 Cognitive Understand 1 4 Cognitive Understand 2 5 Cognitive Remember 1 6 Cognitive Remember 1 7 Cognitive Remember 2 336

337 Match 8 Cognitive Remember 1 9 Cognitive Remember 1 10 Cognitive Apply (?) 1 12 Any answer should be considered correct (for instance How are you? Or Save me both correct D 1 10 Cognitive Remember Cognitive Remember 1 3 Creed Cognitive Remember 1 4 App Cognitive Remember 1 5 Creed Cognitive Remember 1 6 Creed Cognitive Remember 1 7 Creed Cognitive Remember Cognitive Remember 1 9 Creed Cognitive Remember Cognitive Remember 1 11 Creed Cognitive Remember Cognitive Remember Cognitive Remember 1 13 E 1 Affective Receive 1 To determine whether the statement is right or wrong 2 Affective Receive 1 3 Affective Receive 1 4 Affective Receive 1 5 Affective Receive 1 6 Affective Receive 1 7 Affective Receive 1 8 Affective Receive 1 9 Affective Receive 1 10 Affective Receive 1 10 F 1 6 Cognitive Remember 1 To determine whether the statement is true or false 2 21 Cognitive Remember Cognitive Remember Cognitive Remember 1 5 Affective Receiving 1 6 Cognitive Remember Cognitive Remember Cognitive Remember 1 9 6/14 Cognitive Remember Cognitive Remember 1 10 too theological Ġ 1 9 Cognitive Remember 2 Short answers (Candidat 2 12 Cognitive Remember Cognitive Remember 2 4 Creed Cognitive Remember Cognitive Remember 2 337

338 6 6/18 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 2 14 Cognitive Remember Cognitive Remember Cognitive Remember /11 Cognitive Remember Cognitive Remember 2 36 G 1 Cognitive Understand 2(?) Comprehe nsion + brief GĦ Give 5 e.g. 2 Cognitive Understand 2(?) 3 Cognitive Understand 2(?) Cognitive Understand 4 Or Or 4(?) 10 Affective Receive 1-5 App. Cognitive Affective Understand Receive Respond H 1 App. Cognitive Understand 2 Identify the correct sacrament Ħ Fill in the Blanks Give one word 2 App. Cognitive Understand 2 3 App. Cognitive Understand 2 4 App. Cognitive Understand 2 5 App. Cognitive Understand 2 3x App. Cognitive Understand 2 7 App. Cognitive Understand Cognitive Understand I 1 Cognitive Understand 1 IE Write a brief par. 2 Cognitive Understand Cognitive Understand Cognitive Understand Cognitive Understand Cognitive Understand Cognitive Understand 1 2 (x 15) Cognitive Understand Cognitive Understand Affective Receive 4 2 Cognitive Understand Affective Receive 4 3 Cognitive Understand Affective Receive

339 CE 2006 paper This CE paper gives too much importance to the cognitive dimension of knowledge with no item that assesses the affective part. All items, with the exception of the exercises in section H, assess students memory. However, since section H, which covered content outside the syllabus, was not assessed by the examination board, students were only evaluated for their ability to recall content. No other level of the cognitive dimension was assessed. This may explain why this is the longest paper of all the papers that have been appraised by this review. The paper assesses primarily Religious Knowledge and there is no attempt, with the exception of section H, to go deeper to see whether candidates were able to comprehend the knowledge presented and apply concepts to their everyday life. The lack of relationship between content and everyday life makes content arid and very easily forgotten after the examination process. This kind of paper encourages teachers to coach students to study the text by heart, and students need not attempt to understand or construct and/or internalise meaning. Similar to the JL paper-setters, in designing this paper the examiners have erroneously included the content covered by Chapters 7 and 12 without noticing that this is not required by the 2005 syllabus. On the whole, the language used is simple and the instructions given at the beginning of each exercise are clear. With the exception of section H, there are no images inserted in the paper. Table 9 gives a more detailed description and analysis of each item. Table 9. Analysis of Common Entrance Religion Examination Paper 2006 Secti Q. Conte Domain Level Marks T. Comment on & Type of Q. nt found in Chapt er A 1 8 Cognitive Remember 1 Multiple choice (candidates to choose amongst 2 17 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 7 Cognitive Remember Cognitive Remember 1 339

340 Matching 9 12 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 14 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 20 B 2 20 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 10 Ċ 1 Cognitive Remember 10 Reorder Cognitive Remember 4 D 1 13 Cognitive Remember 1 Fill in the blanks 2 23 Cognitive Remember 1 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember (?) Cognitive Remember Cognitive Remember Cognitive Remember (?) Cognitive Remember 1 13 E 2 5 Cognitive Remember 1 Fill in the blanks (Candidates are restricted to 1 word) 3 11 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 8 Cognitive Remember Cognitive Remember Cognitive Remember 1 340

341 11 12 Cognitive Remember 1 12 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 15 F Fill in the Blank s 1 Cognitive Remember 11 Ġ 1 1 Cognitive Remember 2 Short answers Fill in the blanks G 2 3 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember 1 7 Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember /11 Cognitive Remember 1 14 Cognitive Remember Cognitive Remember 1 14 GĦ 1 2 Cognitive Remember 1 Fill in 2 2 Cognitive Remember 1 the 3 2 Cognitive Remember 1 blank s 4 2 Cognitive Remember 1 4 H 1 Cognitive Analysis 6 Write 2 Cognitive Analysis 6 a 3 Cognitive Analysis 6 brief par. 4 Cognitive Analysis 6 24 Ħ 1 9 Cognitive Remember 3 Short ans. (3 required) words are left missing Exact word is needed. Candidates are given exact space for the number of letters in the word Not assessed since the content is not found in the syllabus. 2 9 Cognitive Remember 3 3 Cognitive Remember 3 Not in textbook 4 22 Cognitive Remember 3 5 Cognitive Remember 3 15 Not in textbook 341

342 I 1 App. Cognitive Remember 3 Give examples (candidates are required to give 3) IE Fill in the blanks 2 App. Cognitive Remember 3 3 Cognitive Remember 3 4 App. Cognitive Remember Cognitive Remember Cognitive Remember Cognitive Remember /20 /21 Cognitive Remember 3 9 Cognitive Remember / Cognitive Remember 1 17 Exact word is 20/ (x17) required. There is no 21 flexibility Student Performance Having analyzed the papers, it is now important to see how students performed in these papers as this will indicate the papers level of difficulty and their quality Student Performance on JL papers Both examiners reports of 2004 and 2005 state that the papers main aim was to see that students are able to apply Religious Knowledge to everyday life. This seems to be the philosophy of the examiners since this aim had already been stated in other past reports, namely those of 2001 and Apart from this aim, the examiners also state that the examination intent is to see whether students i. have learnt (2002, 2004, 2005) ii. have understood (2001, 2002, 2004, 2005) iii. are able to express clearly what they know (2001) iv. can reflect on religious beliefs (2003) Although the examiners ultimate aim is the application of knowledge, translated in the language of Bloom s taxonomy, it seems that examiners are more interested to assess the affective dimension of knowledge. Although examiners use the word apply which could be erroneously interpreted as referring only to application in the cognitive dimension, the examiners qualification of application to everyday life, in the context of religious language can be understood to also refer to commitment to values and ideas, thus to responding. However the other aims clearly refer to the cognitive dimension - remember, understand and evaluate. From the analysis it is clear that the examiners were not able to translate these aims into concrete examination items since the past three J.L. papers assess mainly memory, with an average of 61.33% of the marks dedicated to remembering. These 342

343 three papers then marginally assess Understanding (26.67% of the marks), Application (6.67% of the marks) and Analysis (0.33% of the marks). Only 12.33% of the marks could potentially be allotted to students awareness of ideas/values (receiving). Table 10 lists the examiners aims and maps them against the knowledge categories covered by the items in the papers. The inability to match examiners aims with the items is mainly due to unclear objectives. It appears that the examiners do not keep their objectives in mind when preparing the examination papers. Table 10. Mapping of Knowledge domains covered by items onto examiner s aims. Paper Examiners Aims Remember 46% Percentage of marks reflected in paper items Cognitive Affective Remember Understand Apply Analyse Respond Understand 12% Apply 12% Respond 12% Remember 81% Understand 34% Apply 0% Respond 15% No aims ere specified 0% 0% 57% 34% 8% 1% 10% It is evident that candidates get high passes in Religion. In these past three years the average of those who passed the exam was of 86.17% for males and 88.36% for females. 44 When compared with the other four subjects it transpires that the highest number of passes is in fact obtained in Religion as shown in Table 11. This leads to question whether the exam is in fact too easy and whether it discriminates candidates abilities. Table 12 gathers the information reported by the Educational Assessment Unit regarding levels of difficulty and discrimination over these past three years. It is clear that only a very small number of items, covering 44 The reports show that those who obtained a pass (grade C) in 2006: 89.56% (males), 91.88% (females); 2005: 80.05% (males) 82.62% (females); and 2004: 88.59% (males), 90.58% (females) 343

344 an average 10.67% of the marks, turn out to be correct in the level of difficulty and are able to discriminate well or sufficiently. Table 11. Percentage of passes per subject and by gender Paper Gender Religion Social Studies Maltese Mathematics English Females 90.58% 80.42% 83.73% 74.72% 58.77% 2004 Males 88.59% 81.42% 75.87% 69.97% 54.90% Females 82.62% 82.51% 79.76% % 2005 Males 80.05% 81.58% 75.61% 71.30% 63.82% 2006 Females 91.88% 82.13% % 66.16% Males 89.56% 81.95% % 62.30% Table 12. Levels of Difficulty and Discrimination per paper N o of items % of marks N o of items % of marks N o of items % of marks Correct levels of difficulty and discrimination Correct level of difficulty and discriminated sufficiently On the difficult side but discriminated sufficiently On the easy side but discriminated sufficiently Correct level of difficulty but did not discriminate sufficiently On the difficult side and did not discriminate sufficiently On the easy side and did not discriminated sufficiently 2 2% 9 14% 2 4% 2 2% 3 8% 1 2% 0 0% 9 13% 2 4% 64 71% 35 57% 39 55% 2 2% 2 3% 3 6% 3 3% 4 4% 2 4% 0 0% 1 1% 10 10% TOTAL 73 80% % A further analysis, as illustrated in Tables 12a and 12b, shows that the majority of the items are easy but only a very small number of items, accounting for an average of 11% of the marks, turn out to be non-discriminatory. Table 12a. Levels of difficulty per paper N o of items % of marks N o of items % of marks N o of items % of marks 45 One may note that the figures presented in Tables 12 may not always agree with the figures presented in the official reports. After checking the individual items some corrections were necessary. 344

345 Difficult 3 3% 13 17% 4 8% Correct levels of difficulty 6 6% 14 25% 6 12% Easy 64 71% 36 58% 49 65% TOTAL 73 80% % 59 85% Table12b. Levels of discrimination per paper N o of items % of marks N o of items % of marks N o of items % of marks Discriminated correctly 2 2% 9 14% 2 4% Discriminated sufficiently 66 73% 47 78% 42 61% Did not discriminate 5 5% 7 8% 15 20% TOTAL 73 80% % 59 85% 4.2. Student Performance on CE papers The amount of information available for the analysis of student performace on CE papers is limited. Table 13 shows the percentage of candidates that obtained more than 75% and 50% over the past three years. With the exception of the 2006 paper, it is evident that students find Religion the easiest among the five examinable subjects. The pattern described in the table is very similar to the pattern that resulted from JL examination papers. The exception in the 2006 paper might have resulted from the possible increased anxiety caused to students by a section that did not cover part of the syllabus. With regards to the latter incident, the examiners noted the error in the design of the paper and did not assess students responses to this section. Table 13. Percentage of candidates that obtained more than 75% and 50% of the marks Paper % of mark Religion Social Studies Maltese Mathematics English 75% 82% 39% 16% 22% 10% 50% 99% 96% 91% 76% 74% 75% 80% 70% 17% 39% 14% 50% 98% 97% 83% 83% 66% 75% 67% 78% 11% 33% 22% 50% 96% 98% 80% 80% 79% The likelihood that the Religion paper is the easiest amongst the five subjects is not easily supported by the data in Table 14 which indicates the percentage of students 345

346 that finished before the allocated time (1 hr 30mins). While it is true that in these last two years, two thirds of the student population finished at least 15 minutes (2006: 60.4%, 2005: 69.57%) before the end of the examination, this figure is close to students finishing time in the English examination paper (2006: 55%, 2005: 45.64%). The two examination subjects stand on the extreme ends of the continuum, with students getting the lowest marks in English and the highest marks in Religion. While Table 14 might not support the notion that students find the Religion exam easy, it may indicate that students are not being challenged enough. This might be due to the extremely high amount of items that assess just memory. Table 14. Percentage of students who finished Paper Time Religion Social Studies Maltese Mathematics English * * * * * * * did 2.14 * not 2005 finish * information is not available 5. Recommendations One of the main problems lies with the lack of clarity whether the subject pupils are learning should in fact be Religious Knowledge or Religious Education. While, at times the objectives of the syllabus and those declared by examiners in their papers point to the will to educate pupils affective dimension, in real fact the textbook and the examination papers emphasize knowledge of Religious Content. There is an 346

347 obsession with the knowledge or recall of facts which inevitably leads to rote learning. The identity and nature of the teaching of Religion in schools at primary level needs to be specified. It is my opinion that the needs of the child, those of society and those of the National Minimum Curriculum require that we teach Religious Education and not Religious Knowledge. Through Religious Education, students should be helped to grow in their religious and spiritual dimension. Thus memory should only be the basis for further reflection, critical appraisal and the key to experience. In our Maltese context this means that they are helped to develop through the Catholic language. It is only in this manner that holistic education can be assured. It is primarily for this reason that it is important that Religion continues to form part of the assessment process of students finishing primary schooling. This confusion in the identity of the subject has resulted in an inability to produce clear and focused objectives which can guide the construction of adequate textbooks, lead to the design of examinations that truly assess pupils learning and help teachers prepare lessons that facilitate students holistic education. The content needs to reflect these objectives. The development of new textbooks and Teaching Guides is considered necessary. The textbooks need to be more elaborate, while Teaching Guides need to help teachers in becoming more knowledgeable of the content and give ideas of how to move beyond the remembering level of the cognitive category. At this point it is pertinent to highlight that the content presented is limited and does not challenge students nor does it help them grow holistically. There is a need to help students understand religious language and more importantly contribute to helping them use and construct religious language. Furthermore, there is a need to help students understand the basic values and message of Christianity and help them see their relevance and importance in everyday life. The examination papers should be constructed on the objectives announced by the syllabus and not on the content of the textbook. Papers need to have a higher percentage of marks that assess the affective domain and that assess whether students are able to apply, analyse and evaluate. This could be done through the use of comprehensions using moral dilemma stories, or short compositions which 347

348 encourage students to take on the role of a character present in the gospel or religious narratives. The papers should be presented in such a way as to lessen cognitive stress such as using titles describing the content covered by the exercise, using of more images, and less text. Similarly, the exercises should be more balanced, taking into consideration the candidates individual difference in learning and expressing themselves. Thus it is not impossible to imagine a paper that asks students to build a story from a set of images or to extract the symbols and meaning from some images. After looking at the year six annual examination and 11+ examination papers, one notes the similarity between the three examination papers. In order to reduce student stress and unnecessary repetition, one needs to take into consideration the possibility of having one exam that assesses students learning. This may create some bureaucratic difficulties of how to integrate the CE and JL papers. 348

349 Appendix 8 The Junior Lyceum & Common Entrance Examinations: A Content Analysis of the Social Studies Syllabi and Examination Papers Godfrey Baldacchino Introduction Content analysis (or textual analysis) is a standard methodology used in the social sciences in order to examine the content of communications. Qualitatively, content analysis can involve any kind of analysis where communication content (speech, written text, interviews, images...) is categorized and classified. Babbie (2001: 304) defines it as the study of recorded human communications, such as books, web sites, paintings and laws. Lasswell (1948: 37) formulated the core questions of content analysis as: Who says what, to whom, why, to what extent and with what effect? Holsti (1969: 14) offers a broad definition of content analysis that speaks more to its process by explaining it as any technique for making inferences by objectively and systematically identifying specified characteristics of messages. Content analysis is essentially a method that enables a researcher to discriminate between and within large amounts of textual data, and systematically identify its properties, such as via the frequency of those keywords most used, or by detecting the more important structures of its communication content. Such a trawling of textual information is to be categorized according to and within a specified theoretical framework, which will inform the data analysis, providing at the end a meaningful reading of content under scrutiny. Only a good scientific hypothesis can lead to the development of a methodology that will allow an empirical description, be it dynamic or static. Since the 1980s, content analysis has become an increasingly important tool in the measurement of success in public relations (notably media relations) programs; the assessment of media profiles; and the content of political campaign messages. In these circumstances, content analysis is an element of media evaluation or media 46 Godfrey Baldacchino PhD(Warwick), BA PGCE(Malta), MA(The Hague) is Canada Research Chair in Island Studies at the University of Prince Edward Island, Canada and visiting Associate Professor of Sociology at the University of Malta, Malta. He is author, co-author, editor or co-editor of some two dozen publications, and author or co-author of some 50 peer reviewed journal articles and book chapters. He is also the author of Ninvestigaw is-socjeta and Introducing Social Studies (both Malta, PEG Publications). The contents of this report are only attributable to its author. This report was completed: November 30,

350 analysis. In analyses of this type, data from content analysis is usually combined with media data (circulation, readership, number of viewers and listeners, frequency of publication). Of course, content analysis has its detractors: it has been criticized for being a scientistic, positivist methodology; as well as for the decontextualization of words from the discourse being examined. The criticism leveled by Billig (1989: 206) is typical: This sort of methodology can count words, but it cannot interpret them. Under some circumstances mere counting can lead to misleading conclusions. Yet, here is an example of a methodology that can be used to organize an analysis which is able to capture the logic of the contingent, allowing for powerful insights into processes otherwise difficult to detect or measure. According to Krippendorf (2004), six questions must be addressed in every content analysis: 1. Which data is analyzed? 2. How is it defined? 3. What is the population from which it is drawn? 4. What is the context relative to which the data is analyzed? 5. What are the boundaries of the analysis? 6. What is the target of the inferences? Based on Zipf's Law (Zipf, 1949), the assumption is that distribution of words and phrases is never random but their choice, contextual location and relative frequency most often reflect important concerns in every communication. Therefore, quantitative content analysis starts with word frequencies, space measurements (column centimeters/inches in the case of newspapers), time counts (for radio and TV time) and keyword frequencies. However, content analysis extends far beyond plain word counts, as with keyword in context routines: words can be analyzed in their specific context to be disambiguated. Synonyms and homonyms can be isolated in accordance to linguistic properties of a language. McKeone (1995) has highlighted the difference between prescriptive analysis and open analysis. In prescriptive analysis, the context is a closely-defined set of communication parameters (such as specific messages, subject matter); open analysis identifies the dominant messages and subject matter within the text. A further step in analysis is the distinction between dictionary-based (quantitative)approaches and qualitative approaches. Dictionary-based approaches set up a list of categories derived from the frequency list of words and control the 350

351 distribution of words and their respective categories over the texts. While methods in quantitative content analysis in this way transform observations of found categories into quantitative statistical data, the qualitative content analysis focuses more on the intentionality and its implications. 2. A Reflexive Commentary As an evaluation approach, content analysis is considered to be quasi-evaluation because content analysis judgments need not be based on value statements. Instead, they can be based on knowledge. Such content analyses are not evaluations. On the other hand, when content analysis judgments are based on values, such studies are evaluations (Frisbie, 1986). In this particular case, every attempt has been made to resort to a mechanical, scientific interpretation of the data at hand. In such a case, the outcome of the analysis would be the same, irrespective of who would carry it out. However, I recognize and admit up front that this report is not only an automatic and value-free exercise. As a professional educator, as a teacher at all levels of formal schooling and beyond, and as the author or co-editor of at least three texts that have been used for many years now as textbooks in schools (Baldacchino, 1999; 2000; Sultana & Baldacchino, 1994), I come with a certain amount of baggage and reflexivity to this exercise, and which I do not wish to completely marginalize and keep at bay from my analysis. Thus, I wish to declare that the substantive aspect of content analysis is, as far as is possible, value-free; but, its critical interpretation and the recommendations which subsequent emerge are not. My underlying philosophy as an educator is to provide opportunities for my students to empower themselves via a widening of life-choices and life-chances. Learning has the tremendous potential of fostering a particular vision of the world, in which individuals see themselves as confident, informed, skilled and responsible actors. Social studies is ideally placed in the curriculum, perhaps more than any other subject, to translate this potential into actuality. At the same time, I am deeply aware of the socio-cultural context in Malta in which this liberating learning is meant to take place. There is an enormous sense of individualistic or family-based competition among the Maltese, and credentialing is often understood as being a key vehicle of social mobility. This has never been proven in practice anecdotal evidence suggests that many of the so-called nouveau riche, who ironically send their own kids to private/independent schools, would not typically have done well at school however, the myth is all powerful and 351

352 has been aggressively reproduced and legitimated by parents, teachers, community leaders and other peers. From a young age, boys and girls are drilled into seeking examination success via schooling at school and more schooling via private lessons after school. Even at kindergarten age, the pressure is already so intense that creative expression and play are sacrificed early on for the sake of rushing the acquisition of basic spelling, writing and reading skills. Thus, it is, sadly, often the end a successful examination grade - which justifies the means the prior classroom experience. Most of us who are middle-aged are twice victims of the same system: we remember the high levels of anxiety and stress that we and our parents were subjected to when we were 11 years old and, at that time, most of us would have sat for an exam to seek entry into a particular school (there were no common entrance or Junior Lyceum exams at that time); while as parents, we would have lived through the same nightmare as our sons and daughters reached age 11 and the same critical threshold. The saga is repeated annually, with no hope of reform in sight. Yet, what reform is appropriate? There would be an uproar if the examination system is thoroughly dismantled, removing a mechanism which streams students by aptitude (at least, those are its intentions!) and thus (arguably) better shepherd the young boys and girls into suitable, parallel learning paths. The mechanism actually takes further another well-entrenched streaming mechanism, since the current troika of state, church and private/independent schools is already a streaming structure, defined by social class, aptitude (omitting from this discussion the polemic relationship between the two) and, in most cases, even gender. The mean significant difference in grades achieved by state versus non-state students in the Junior Lyceum entrance exams every year (more below) speaks for itself. The 11+ Exam is also a mechanism that obliges most students to take their schooling seriously; and teachers, in particular, shudder to think of the anarchy and indifference that would prevail were such a power motivator to be removed. Yet, there is certainly scope for discussing alternatives. Ideally, it should be a system that is less stressful to our young boys and girls, still rewards performance and achievement and preferably does a better job at it than the current tools (that is, the JL & CE examination papers). More ideas are proposed later in this report. 352

353 3. The Social Studies Syllabus & the Junior Lyceum/Common Entrance Examinations: A Content Analysis Holsti (1969) places the uses of content analysis within the framework of a basic communication paradigm. He also groups fifteen uses of content analysis into three basic categories: make inferences about the antecedents of a communication describe and make inferences about characteristics of a communication make inferences about the effects of a communication. The following set of six questions - based on Berelson (1952) and adapted by Holsti (1969) - show the various uses of content analysis by purpose, that is, in terms of the general question (culled from Lasswell (1948) - they are intended to answer. The question is directly associated with sub-sets of questions that speak to the particular context being analyzed in this report: specific Social Studies Syllabi and Examinations. Of course, the two are meant to be intimately connected although, ironically, examinations are not usually intended for their own sake but for the sake of higher, nobler objectives: a particular set of knowledge, understanding and skills retention beyond the examination setting proper. a) - Make inferences about the antecedents of communications source: Who? A set of Education Officers from the Curriculum Management Department, comprising administrators from the Education Division (who themselves would have been teachers in earlier phases of their career) are typically those tasked with the compilation and subsequent updating of curricula and syllabi that become the basis of school instruction, exam preparation and exam papersetting. This would be done in the context of a general philosophy of education, and a vision of the school and the classroom in society. Feedback would be received on a regular basis from other teachers, heads of school, as well as parents, and especially in the wake of a completed cycle of examinations. The examination papers per se would be set by Education Officers from the Educational Assessment Unit of the same Department of Curriculum Management. This basically ensures that there is a strong connection between the measuring instrument (the examination paper) and what it is intended to measure (the curriculum). 353

354 Since it is therefore teachers, or former teachers, who have practically full control over this process, we should not be surprised to find that the curriculum is laid out in the systematic format that facilitates its transposition into teaching formats in a classroom setting. While this has its benefits, it reduces space for innovative learning styles. Moreover, some of the set examination questions do not address any of the presumed learning outcomes of social studies, but call for a recollection of what is understood in society as proper a format of response that is typical in a primary school/ early secondary school environment, and one with which teachers would be very familiar and comfortable. b) - Encoding process: Why? The encoding process positions the required learning in terms of four categories: knowledge (relating to factual information); understanding (relating to meaning making); skills (relating to behavioural acquisition), and attitudes (relating to character formation), set in turn within a gradation of eight levels of difficulty. The rationale of all this is expunged in terms of an awareness and appreciation of the geographical environment in all its diversity, a better understanding of Maltese society in a national, regional (Euro-Mediterranean) and global context; and an acquisition of those skills and competencies that will help; students lead lives as responsible citizens as part of a wider national and global citizenry. The knowledge requirements of the social studies curriculum revolve around an understanding of basic natural cycles (weather, wind, seasons, day/night); place identification (locality, Malta & Gozo; Mediterranean Sea; Europe); features of Maltese culture and identity; social and political institutions; and some historical events. The understanding requirements of the social studies curriculum deal with an appreciation of systemic interdependence (such as human-nature interactions; individual-social interactions; Malta within Europe, the Mediterranean and the European Union). Moreover, there are actually three mini-curricula incorporated into the social studies syllabus, since the subject matter is carved up between three, distinct knowledge silos in the actual syllabus: the social/human/civic, the[physical] geographic and the historic environments. 354

355 The understanding requirements apply to each of these three parallel strands, as well as to them holistically and comprehensively. The skills requirements of the social studies curriculum relate to abilities for self-expression, the identification of patterns and interactions, the description of past events, and the ability to collect and reproduce information about such events (that is, basic research and reporting skills). The attitudinal requirements of the social studies curriculum concern respect and care for the natural and social environment via a development of a civic consciousness, involving rights and obligations. Given the examination paper format, it is very difficult, if not impossible, for students to address the skills and attitudinal aspects of their curriculum, not just in social studies but also in other subjects. It is equally problematic for teachers and education officers to come up with suitable instruments that measure such learning targets. Thus, what the examination actually measures is a reported understanding of the required skills and attitudes, rather than the skills and attitudes per se, which can be learnt irrespective and separately from the actual acquisition of the skills and attitudes themselves. In other words, the exam paper is more suitable for testing formalism namely, whether the student has been taught, or studied, these concepts in a sufficiently proper manner for the same student to remember and reproduce what they are about, rather than whether the student has actually learnt (meaning integrated and assimilated into his/her psyche) the curriculum objectives. The difference between teaching formats and learning outcomes has an interesting parallel in studies of the sociology of religion where a clear distinction is made between religiosity (which is formal and manifest, and so can be seen and measured, such as church attendance) and religion (which is a function of values and beliefs). Students, and teachers, would also be familiar with this distinction, since the teaching of religion (another subject tested for by an examination) is again obliged to bend itself backwards to accommodate an exam format. Moreover, various social studies teachers are also, or have been, teachers of religion, so the format is one that they are comfortable with. 355

356 c) - Describe & make inferences about the characteristics of communications channel: How? In our case, the social studies curriculum is the driving instrument, and the examination paper the measuring stick. The obligation to test and measure knowledge and understanding plus (less easily) skills and attitudes on a scale, and ultimately to pass or fail the student on that same scale, is a very powerful tool: one that arguably will determine the future of the student s education, life chances, career and overall human and professional development in Malta and perhaps beyond. There is no stronger basis for persuasion than that. Obviously, in such a context, past examination papers represent the manner in which the curriculum is actually translated or encoded into an examination format. A longitudinal analysis of examination papers in social studies (both Junior Lyceum (JL) and Common Entrance (CE)) over the past 3 years reveals a consistent style of paper setting, question formatting and language use; with similar questions being asked in some instances. Students, parents, as well as teachers would be expected to carefully scrutinize past papers in order to better grasp the tangible manifestation of curriculum objectives, and to test out student skills on that benchmark. d) - Message: What? An indicator of the substance of the message can be derived from a key word in context analysis. The key action words most often used in the latest version (in Maltese) of the rationale for the social studies curriculum (Dipartiment tal-kurrikulu, 2006) are: tagħrif/jagħrfu (26 times), fehma/jifhmu (24 times); juru (11 times); jafu (10 times); ħiliet (9 times); jiddeskrivu (9 times); ikunu (8 times); process/i (8 times); josservaw/osservazzjonijiet (5 times); espressjoni (5 times); trawwim (4 times); relazzjoni/jiet (4 times); jikseb/jiksbu (3 times); jitgħallmu (3 times) and soċjalizzazzjoni (3 times). In contrast, jistaqsi/u appears 3 times; j/ipparteċipa/w and komunikazzjoni appear twice each; j/i/fittex/xu and impenn appear once each. The key word 356

357 exercise with all its limitations - suggests that the emphasis of the curriculum is on the ability to understand, to recollect, to identify, to study. This emerges from a systematic word-counting exercise in spite of the declaration by the same document that the general objective of social studies education is NOT to encourage reproduction, rote and memory work (p.vi). Thus, we have a serious disconnect here: the actual content of the curriculum in terms of repetition of key words - betrays its stated and declared pedagogic intent. Perhaps this obsevation - again - should not come to us as a surprise, given the obligation, at the end of the day, to measure and to grade via an examination. It is as if we have a case of a tail wagging the dog: the format of a written examination with a rigorous marking schedule (which itself allows limited scope for creativity or flexibility) obligates the curriculum to emphasize factual understanding, even if in a subliminal way. In other words, if one should ever wish to change the curriculum, the best and easiest way would be simply to reform the examination. The rest will follow. The chief examiner reports transmit a certain exasperation at how students fail various questions because of their resort to memory work; but most of the questions nevertheless ask for material that can be put to memory, even though that may not be the stated intent of the curriculum setters, the teachers or the examiners. If the current examination practice and format persists, how can one ever discriminate as to whether an answer had been committed to memory or is the outcome of a reflexive experience by the student during the examination session proper? A further point that requires commentary at this stage is that the examination paper is not just an examination in social studies but, concurrently an examination in Maltese (or English, for those who sit for the paper in that language) as well as in Religious Knowledge. A student with a solid grounding in the language will find him/herself facilitated in his/her understanding of the questions and thus more likely to promote their correct choice (in the case of multiple choice or fill in the blanks with words provided type questions) or their otherwise correct answering. Some of the exercises could be lifted as they are and reproduced without any difficulty in an English or Maltese comprehension or general language examination paper. 357

358 The social studies curriculum is a bringing together of three distinct minisyllabi of environmental studies : the social/human/civic, the physical geographic and the historical. The examination papers typically do a good job in fairly representing these diverse strands of the syllabus, and in fairly equal measure. However, the social aspect of the syllabus is the most difficult to organize in terms of factual questions. Some examination questions are crafted well, to the extent that all three environments are tested for in the same question (e.g. CE 2005 Ċ and E; or CE 2006 A). The Private Schools Association exam papers however are better at combining these mini-syllabi within the same question than the Education Division ones. Indeed, no suitable example of such a combination could be found amongst the 2004, 2005 and 2006 JL papers. In other words, the JL exam questions usually expect students to refer to either the social OR the physical geographic OR the historical environment only. A content analysis of the examination papers must include a discriminatory analysis of the examination papers set by the two different examination bodies Education Division and the Private Schools Association, though arguably based on the same identical syllabus. Both sets of papers are set for an exam of 90 minutes duration. They both have a similar number of questions, and set on a similar number of pages using a similarly sized font, and fairly similar exercises, using both text, images and maps to set the questions. There are however some basic, consistent differences between the 2 sets of papers: (a) The CE papers are more wordy, having more words per page than the JL papers in most cases (a word count of the first and last pages of the 2004, 2005 and 2006 papers reveals that the CE version has many more words than the JL version on 5 out of 6 occasions); this comment applies in particular to the length of comprehension-type questions: the one in the 2006 CE paper has no less than 227 words for comparative purposes, the equivalent one in the 2006 JL paper had only 75 words. More wordy, rambling papers require more time, energy and attention to read; 358

359 (b) the CE papers are occasionally (but not always) less graphic; having less questions supported by images and/or with images occupying less space; (c) the CE papers also have narrower margins. They typically have a margin of 1.5 to 2cm, whereas the JL papers have a more relaxed and user friendly margin of 2 to 3cm. The resort to a gutter on the left-hand side is occasional and haphazard. However, there is no consistency in these formats, either between different papers or even within the same paper; (d) only the CE papers have superfluous language, meaning that in fill in the blanks -type exercises, they also include words that are surplus to requirements ( kliem żejjed ), making the exercise more difficult; (e) only the CE papers have clear examples of questions that elicit knowledge and understanding of social studies holistically, while the JL papers are consistently organized in terms of social OR physical-geographical OR historical material, with hardly any interaction between the three. The evaluative summary of these observations is that the two sets of papers are not pitched at the same level of difficulty or challenge. Given that both papers assign students the same time for completion, overall the CE paper is tougher to complete and less student-friendly than the JL paper. Looking next at specific papers, the CE 2004 paper in particular is fairly problematic. The comprehension in Exercise D speaks about the importance of all work and its interdependence; but it also refers to certain types of work/jobs being more important than others which, economically is also true, and is reflected in wage differentials. Exercise G (Vantaggi / Zvantaggi) was in some instances ambiguous in its choice of statements, and the students were not expected to explain their choices. Thus, the poisoning of wells before the Great Siege of 1565 (G9) was both advantageous and disadvantageous to the Maltese; as was the destruction of anything that could be used by the Turks outside the fortifications (G10). Luckily, this type of question format was not used again either in 2005 and 2006, presumably because its ambiguity has been recognized. In Exercise H, the paper reproduces the myth that Faith, Hope and Charity were the names of the 3 aircraft available to defend the island at the start of the 2nd World War. These were actually nicknames. In Exercise J, questions are asked about farmers raising livestock 359

360 (J6; J7), when farming and animal breeding are usually distinct occupations. All in all, the CE 2004 paper has suffered by not having been properly proofread, tested and scrutinized. The CE 2006 paper dedicates its comprehension exercise (Exercise D) to the issue of refugees in Malta and suggests that an appropriate response would be to welcome diversity and difference. Yet question 1v in this exercise asks why should a Muslim student leave the religion class in school because of her faith. The subtle answer is that the religion lesson is actually a lesson in Catholicism, not religion. Had it been a purely religion lesson, students with different faiths would have no problem staying in the class! Moreover, question 2 (same Exercise D) states that religious beliefs remind society of those civic values that bring about greater unity and order in society! This is not an appropriate question to ask, since the examiners are implicitly making a value judgment against religious pluralism. There are various truisms or inexactitudes in the various papers, but more so in the CE papers than the JL ones. Thus, in CE 2005 A8, all three answers are, strictly speaking, correct. Again in CE 2005, this time in Exercise C8, the head of a monarchy is not only a king but a queen. In CE 2006 E10, the statement implies that factories are clustered in industrial zones: this is not necessarily so. In the same exercise (E7), it is suggested that politicians are delivering a major service to the country. According to which criteria? The word politics is defined as organizing a country in terms of order and laws (E1). This evades the key task of politics: to determine policy or take decisions. In JL 2006, B10 suggests that the Santa Marija convoy was the only successful convoy to bring food and fuel to Malta during Wrong: there were others. In JL 2006 F6, the word strait is translated as istratt, apart from fliegu. In JL 2004 G2, the Knights of St John are suggested as friends of the Maltese. They certainly were not. Again in JL 2004 Ħ4, the suggested answer is Fort St Angelo. However, since it is facing the sea-ward side, the fort could not defend Birgu in the case of a land-based attack. And, in JL 2005 Ġ2 & Ġ3, the Suez Canal and the connection between the Mediterranean Sea and the Red Sea do not exist on the map provided! As far as predictability goes, students can be practically guaranteed to have two similar map-recognition exercises in their examination, whether it is JL or 360

361 CE: one relates to a map of the Maltese islands, the other to a map of the Mediterranean sea. Looking at the positive side, there is ONE exercise that does seek to elicit some evidence of student learning, while also appealing to some aspects of creativity otherwise glaringly absent in the examination paper set-up. This is found in the JL papers, where students are asked to imagine that they were participating in a historical episode (like the shipwreck of St Paul in JL 2005), or were themselves some historical figure (like Grandmaster La Vallette during the Great Siege in JL 2006). Students can here combine facts with emotions in their response; they can combine knowledge of history, with knowledge of physical geography and knowledge of basic sociology; they also have the space, and time, to do so. On the other hand, it can be difficult to mark responses to such questions on a completely mechanical scale. Yet, overall, there seems to be a general obsession with order, discipline, control and conformity in the papers, as well as subtle inducement towards the provision of stock replies. By way of example, JL 2005 Ħ1 asks how does an unemployed person feel? The students may have relatives or acquaintances who are, or have been, unemployed. But, rather than commenting about any known experience, the answer is supposed to be a dogmatic and categoric statement, similar to all the sentences in the paragraph above the questions. The syllabus itself needs some updating, a dispelling of myth, as well as better coming to terms with the facts. Just by way of example, students are now expected to study agriculture and manufacturing as economic activities, while the services sector both public and private - which employs almost 80% of all the gainfully employed in Malta today - is not even included in the syllabus. (Most students, and possibly even some of their teachers, have no idea what a raħħal is!). The family is still understood as led by a husband and wife, when this structure is certainly not the only one in modern society. Moreover, still too much emphasis is placed on the history of others, or on the perspective of others on Maltese history, rather than the history of the Maltese. This applies, for instance, to such episodes as the Great Siege (in which the Maltese found themselves in between sworn enemies), in interpretations of the events of the Second World War (when many Maltese harboured Italianate sympathies, and some were deported for that purpose) and the supposed uninterrupted adherence of the locals to the Catholic faith ever since St Paul came to Malta 361

362 (records from the Arab period suggest that the Maltese population was predominantly Muslim during that time). e) - Recipient: To whom? Of course, the ultimate beneficiaries or should one say hapless victims? - of this exercise are the students: primarily, 11-year old boys and girls just finishing primary school. They remain the only key stakeholders that have no input into any part of the operation of this exercise. These students, moreover, are clearly distinguishable between those coming from state primary schools (approx 84% of those sitting for the JL exam) and those from non-state primary schools (the remaining 16%). Many students from both sets of schools actually sit for both the JL and the CE exams organized by the Private Schools Association. Practically the whole Year 6 student population is involved. The number of applicants for the social studies JL Exam reached a peak in the year 2000 with 3,805 students, and has declined since, effects of a demographic shift and a reduced percentage of children within the Maltese population. Meanwhile, the percentage of students who are successful in the examination has steadily increased with time, no doubt partly a response to pressure starting in the early 1980s about Junior Lyceums being too elitist. In fact, the highest ever percentage of successful students in the 24 years that the JL Examination has been in place was achieved in 2005: 60.44%. (Data for 2006 not available.) However, there is a very significant (that is, non-random) difference in the results achieved in the JL Exam in Social Studies by students coming from state primary schools versus those coming from non-state (church and independent/ private) schools. One would hazard to guess that a similar distinction is noticeable in other subjects at the same examination. In Social Studies, the mean success rate of students from state schools over 24 years of testing has been of around 50%; the mean success rate of students from nonstate schools over the same period has been of around 70%. There have only been 3 (out of 24) years when the mean score of state school students exceeded the mean score of non-state school students: 1985, 1986 and Since 1988, the gap has progressively widened, with non-state students faring 362

363 consistently better; the gap between the two cohorts was already 17% in 1992 and 1993, and was as high as 24% in 1998; and 25% in Since these students are sitting for the same exam, the non-state students have higher levels of cultural capital; are being better prepared for the exam (culturally, cognitively and psychologically), are generally better disposed to working under exam conditions, and/or have a stronger grasp of reading, studying, language and memory skills. f) - Make inferences about the consequences of communications decoding process: With what effect? The immediate effect of the examination is to assign a grade A to E to all the students. The Social Studies JL Exam is the one with the highest relative number of A Grades (21.6% in 2004) and English and Maltese the lowest (9.1% and 8.4% respectively in 2004). A Grade C or better in each of the five subjects (the other two being Religion and Mathematics) is required for students to be admitted to the Junior Lyceum (JL). Thus, the examination result determines whether students are admitted to JLs (4 in Malta, 1 in Gozo for boys; 5 in Malta, 1 in Gozo for girls), or otherwise sent to area secondary schools. The long-term implications of these decisions are difficult to pinpoint with precision. However, for those aspiring to a professional career, and/or to the pursuit of a technical diploma or university degree, the probability of making such an achievement from a secondary education in a Junior Lyceum is much higher that from a similar education in a conventional state secondary school, and almost as high as that from those attending a church/private secondary school. A recent Education Division Tracer study has confirmed that, amongst secondary school leavers, over 91% of students from private/church schools continue studying; as do 88% of students from Junior Lyceums, as against just 31% of students from government area secondary schools who continue studying; the national mean was 71%. Since 1997, females are proportionately more likely to continue studying after secondary school than males (Guidance & Counselling Services,2006). A similar, potentially life-changing examination threshold, and its associated drama of syllabus setting, examination writing and script marking, will recur at 363

364 age 15 (SEC / O Level Exams) and yet again at age 17 (MATRIC; A & Intermediate Level Exams). 4. Conclusions The overall objective of social studies education at the primary level is to inculcate a sense of admiration, appreciation and respect for our human, geographic, natural and historical environment. Positive feelings are meant to be nurtured in the students towards themselves and others; values making for hone stand responsible citizenship are meant to be inculcated; a proper moderation between individual and collective responsibility are meant to be instilled; and the skills which allow for an active and creative engagement in social life are meant to be facilitated (CMD, 1997). One wonders to what extent, if at all, are the JL and CE examinations contributing towards achieving these noble and worthwhile goals. In spite of various declarations to the contrary, it is the ability to transmit knowledge and understanding that will ultimately decide whether a student passes or fails his/her JL and/or CE examination as set. The examination format is ill-suited to assess any achievement of competence, attitudes or values. If we really and seriously wish to incorporate a device that would somehow measure student achievement on these fronts, then we need to move beyond the examination paper format, possibly in developing a complimentary profiling dossier for each student, which would include evidence of extra-curricular activity, or of good social conduct and civic responsibility; but this must be done with great care, and with ample prior consultation, otherwise it could very quickly be misconstrued as an additional burden on students. An initiative worth considering would be to create a structure whereby students themselves provide input to this reflexive process on a regular basis. What would students like to see in their social studies syllabus? How can it connect better with the real world? How would they wish to be examined in the subject? They are, after all, the key stakeholders in this exercise. One other outcome is that we must encourage and trust the teacher in the classroom to develop the subject-matter in a more exciting, innovative and engaging manner, bringing history, nature and geography alive and making 364

365 them tangible and relevant to the students, while still preparing them for their examination. Of course, this task becomes more difficult with every passing day. Boys and girls are growing up in environments where TV, computers and videogames compete with each other in making their programmes, games and characters flashier and more dramatic than would have been imaginable, even a few years ago. Teachers must compete with these gizmos for that rare prize -student interest and attention spans with much less resources and information technology disposal. One can imagine how exciting and educational is copying notes from the white/chalkboard. The good news is that social studies probably has the potential to be exciting, engaging and stimulating more than any other primary school subject. Opportunities for nature walks and visits to muse a and fortifications are both plentiful and close at hand, and can be recommended for out-of-school hours. Role play, re-enactments of historical episodes, and encounters with artifacts from the past, or with refugees and immigrants from the present, can be organized in the classroom. All this. However, takes time and preparation on the part of the teacher, and it is so very easy to give up and fall into a more predictable routine. Don t expect routines to be challenging to students, however. The JL and CE examination papers tend to reflect this resignation to routine. They are suited to test what students have been taught, and not what they have actually learnt. They are suited to measure knowledge, facts and reward a shrewd understanding of how one should answer set questions. They do not encourage interaction across the 3 mini-syllabi (physical geography, sociology/civics and history) and lose the rich synergy what such an interaction would provide. They are reliable as far as the accurate measurement of correct answers goes, but they are not valid in so far as they do not measure what they are meant to measure. Acknowledgements My thanks to Manwel Debono (Centre for Labour Studies, University of Malta) and Fr. Charles Mallia (Rector, St Elias College) for comments on an earlier draft, and to Dr Grace Grima (University of Malta) for the overall coordination of this timely review exercise. Responsibility for the contents of this document remains howevermy own. 365

366 References Babbie, Earl (2001) The Practice of Social Research, 9th edition, Wadsworth, Thomson Learning Inc. Baldacchino, Godfrey (1999) Ninvestigaw is-socjeta, 2nd edition, Malta, PEG Publications. Baldacchino, Godfrey (2000) Introducing Social Studies, Malta, PEG Publications. Berelson, Bernard (1952/1971) Content Analysis in Communication Research, Glencoe IL, Free Press. Billig, Michael (1988) Methodology and Scholarship in Understanding Ideological Explanation in C. Antaki (ed.) Analyzing Everyday Explanation: A Casebook of Methods, London, Sage. CMD (Curriculum Management Department (1997) Syllabus for the Primary School Years 4, 5 & 6, Malta, Education Division, Ministry of Education and National Culture. Dipartiment tal-kurrikulu (2006) Studji Socjali: Sillabu ta l-iskejjel Primarji, Furjana, Educational Assessment Unit (2004) Entrance Examinations into the Junior Lyceums 2004, Malta, Department for Curriculum Management, Education Division, Ministry of Education, Youth and Employment. Educational Assessment Unit (2005) Entrance Examinations into the Junior Lyceums2005, Malta, Department for Curriculum Management, Education Division, Ministry of Education, Youth and Employment. Frisbie, Richard (1986) The Use of Microcomputer Programs to improve the Reliability and Validity of Content Analysis in Evaluation. Paper presented at Annual Meeting of the American Educational Research Association, San Francisco CA, April. Guidance and Counselling Services, Education Divison (2006).Tracer Study Malta:Education Divison. Website: Holsti, Ole, R. (1969) Content Analysis for the Social Sciences and Humanities. Reading MA, Addison-Wesley. Krippendorf, Klaus (1980/2004) Content Analysis: An Introduction to Its Methodology. 2nd edition, Thousand Oaks CA, Sage. Lasswell, Harold (1948) The Analysis of Political Behaviour: An Empirical Approach, London, Routledge & Kegan Paul. McKeone, Dermot (1995) Measuring Your Media Profile, London, Gower Press. Sultana, Ronald G. & Baldacchino, Godfrey (eds.) (1994) Maltese Society: A Sociological Inquiry, Malta, Mireva. Zipf, George K. (1949) Human Behaviour and the Principle of Least-Effort, Cambridge MA, Addison-Wesley. 366

367 Appendix 9 ENGLISH Evaluation of the Junior Lyceum and Common Entrance papers 2006 Evaluation of the 2005 syllabus Joy Richardson Cambridge International Examinations 24 November

368 Evaluation of the Junior Lyceum and Common Entrance English examination papers 2006 (based on the 2005 syllabus) Junior Lyceum Entrance Examination 2006 Around 90% of the Year 6 (age 10) cohort of 5000 students sit the Junior Lyceum examination, and in different years between 50% and 60% of students are reported as qualifying. The threshold requirement is achievement of Grade C (50%) in all five subjects examined and no compensation is allowed between subjects. The Grade C threshold is 50%, Grade B 68% and Grade A 78%. The specification grid categorises questions as low, 'middle' or 'high' in level of difficulty. In English in 2006, low level questions provided 24% of the marks, middle level questions 51% and higher level questions 25%. Grades may be used for setting pupils subsequently, but marks are not passed on. The grades, and levels of difficulty in the questions, do not relate directly to any differentiation explicit in the syllabus. Language Usage This is a substantial section of the examination, providing 40 of the total 100 marks. These 40 marks are awarded principally for identifying grammatically correct usage, half in the context of a single sentence, half in a sequence of sentences within a short passage. All these questions are multiple choice or 'fill the gap' and (with a few exceptions in Section F) allow only one answer. The specification grid shows the primary objective in most questions as parts of speech (17 marks) or correct use of tenses (12 marks). Although the primary objective for Section F and the secondary objective for Sections B, C and D is the deducing of meaning from context, the context is largely that of sentence structure rather than any broader interpretation of meaning. The questions do not probe richness of vocabulary or understanding of word meanings, despite the focus on vocabulary in the 2005 syllabus, within sections on spoken language, reading and writing. The weight of questions about correct grammatical usage of parts of speech and verb tenses within discrete sentences is greater than might be expected from the 2005 syllabus. Under the heading of writing, this syllabus includes knowledge of the 368

369 function of words within a sentence and the use of grammatically correct sentences. However, the main emphasis is on using grammar to extend meaning within writing. Sections A and B test usage that would have been assimilated early by those speaking English as a first language. It may be that the weight given to correct grammatical usage reflects the needs of second language learners, but this is not explicit in the syllabus, or in the structure of the examination. Items in this section also test logic and reasoning, as in a verbal reasoning test. In this sense they are likely to be a good indicator of general ability, rather than showing specific skills in reading and writing. Comprehension Thirty marks are awarded for reading comprehension. The questions are varied in presentation, moving from multiple choice questions to those requiring written answers of a word, phrase or sentence. The reading passage is broken into three half-page sections, and this is a helpful device. The first three questions all relate to the first paragraph. The questions broadly follow a chronological sequence, or flag the section to be drawn on. One question (a sequence of true/false statements) requires some scanning across the sections, although the statements to be tested appear in order in the text. The questions show progression from literal to inferential comprehension, as detailed in the 2005 syllabus. Few of the questions require the simple lifting of words from the text. Many require the matching of word meanings, using different words in the question to those used in the text. Later questions require inferences about motivations and feelings and an element of 'reading between the lines' about what is not explicitly stated. Some aspects of reading comprehension noted in the 2005 syllabus receive light attention. Although understanding of vocabulary is required, few questions probe how words enhance meaning, or the use of figurative language. There is little testing of children's ability to draw out themes and ideas from the story as a whole, or to link information from different parts of the text. Question 42, asking 'Why was the ancient cave a secret place?' (a 'high' level question) was one of very few that required children to think back over the whole of what they had read. 369

370 By comparison with language usage and comprehension, the comprehension questions are bunched at the 'middle' level of difficulty, with only 6% marks from the test as a whole given for higher level comprehension. Each question is worth one mark, with the exception of the true/false statements which carry ½ mark each. Marks are not weighted according to the difficulty involved in finding or formulating the answer. Penalties in the mark scheme for inaccuracies in writing maintain the focus on correct use of language. Composition The examination gives a choice of five topics for writing. The genres offered are narrative, descriptive and letter writing, all of which are included in the syllabus. The requirement is for 'not less that 150 words'. This reflects the expectation in the syllabus that pupils in Year 6 produce pieces of writing about 150 words according to ability. This appears to be a fairly modest requirement, perhaps reflecting time constraints. Shorter writing is pegged to a lower band in the mark scheme, but the examiner's report (2005) makes no overall comment about length in relation to quality. The layout of the composition page of the examination paper is unappealing and somewhat confusing. The guidance given varies between titles. While two offer no additional instructions, the letter option has a dense paragraph setting the context and suggesting what to include. The story in pictures would attract pupils because of the visual prompts. (The 2005 examiner's report commented that ' the majority of candidates, as usual, opted for the picture composition' but that it elicited 'the worst and most humdrum responses'.) The 2005 syllabus for Year 6 does not refer specifically to the planning of writing, although the editing and revision of work is mentioned. The examination rubric makes no reference to planning, and no framework is provided to help with the process. However, demonstration of the ability to plan work is included in the specification grid for writing. The syllabus refers to writing legibly and using correct language forms, spelling and punctuation. The mark scheme focuses on grammar, spelling and punctuation, vocabulary, and coherent and appropriate organisation, but does not include handwriting. It provides descriptors for each of the elements for each band of marks 370

371 but does not show the weighting of each element or how marks are to be allocated. The specification grid does not relate directly to these descriptors but shows an allocation of 10 marks each for planning work and writing logically, for writing 'relevantly and functionally' and for writing 'accurately and creatively. These attributes are respectively described as 'low', 'medium' and 'high' in level of difficulty. This categorisation lacks precision and does not fit easily with the descriptors in the mark bands. By contrast, guidance in the mark scheme is very specific about the deduction of marks for irrelevance, or an incorrect format for the letter. Common Entrance Examination 2006 The Common Entrance Examination is taken by around 1300 students, and around 450 pass. Pupils are ranked by totalling the marks from all five subject papers. There is no 'cut-off' threshold mark, and compensation between subjects would be possible. In English, questions are not categorised or ordered according to level of difficulty, and questions are more consistently demanding than in the Junior Lyceum exam. Language usage The Common Entrance Examination gives 100 marks (50%) to language usage. This is 10% more than in the Junior Lyceum examination, but the scope of questions is broader. There is more emphasis on vocabulary, word structure and punctuation, as well as on grammar. Most of the questions require choosing the right word to complete a sentence. Some are more open-ended, as in suggesting items found in different locations, or writing questions to match given answers. The language used (for example in sentences about feelings) is rather less formal than in the Junior Lyceum examination. Comprehension Forty marks (20%, compared with 30% in the Junior Lyceum examination) are awarded for reading comprehension. This seems a low percentage, given the importance of the skills being tested. The passage is fairly demanding and written from an adult viewpoint. The layout does little to engage pupils. The questions provide a good level of challenge for able pupils, often requiring the drawing together of information from different parts of the passage. Pupils have to think about what they have read in order to make inferences, and to back up their answers from the text. Word meanings and usage are explored in context. The level of challenge reflects the fact that this is an exam 371

372 that ranks pupils, and is more selective in purpose than the Junior Lyceum entrance exam. It reflects the more demanding aspects of the Year 6 reading syllabus in requiring pupils to justify inferences, to make evaluative responses and to understand the passage as a whole as well as in parts. Composition The composition is allocated 60 marks (30%), the same percentage as in the Junior Lyceum exam. As in the Junior Lyceum exam, and in the syllabus, the requirement is for a composition of not less than 150 words. Five choices of topic are offered, although only one of these gives a title to work from. The types of writing offered include stories, the writing of a dialogue, a letter and a sports report. This range reflects the variety of types of writing for an audience and with a purpose included in the syllabus. All the topics give prompts to help pupils think about what to write, but it is explained that these do not have to be used. A blank page is provided for use in planning the composition. The mark scheme is straightforward, giving 15 marks for content, 15 for vocabulary and expression, 10 for spelling and 20 for grammatical accuracy. There is no specific reference to organisation which is coherent, or appropriate for different types of writing. The focus on vocabulary and expression, also seen in the mark scheme for the Junior Lyceum exam, accords with the attention to vocabulary evident in the 2005 syllabus. Evaluation of the Primary English Syllabus 2005 This evaluation considers the structure and content of Malta s new Primary English Syllabus, in relation to the National Curriculum for English in England. It then compares more precisely the expectations in English for the end of primary education in Malta with expectations at the equivalent stage in England. (Children finish primary at age 10 in Malta and 11 in England, so Year 6 in Malta is equivalent to Year 5 in England.) An overview of syllabus structure and content The new English syllabus for Malta, published in 2005, replaced the 1997 syllabus for English. The 2005 syllabus is structured in terms of learning outcomes for each year, with notes that exemplify content to aid teachers planning. It is clearly organised in years from Year 1 to Year 6, with headings that run through all years. 372

373 The National Curriculum for England is not broken into years but provides a programme of study in English for each Key Stage: Key Stage 1 for Years 1 and 2, ages 5-7; Key Stage 2 for Years 3-6, ages Outcomes are separately described in terms of attainment targets for speaking and listening, reading and writing. In each of these areas there are descriptors for attainment across Key Stages 1 and 2, from Level 1 to Level 5. These levels are not age specific, but over the years the assessment regime has crystallised age-related expectations. The national benchmark is that children should reach Level 4 in literacy by the age of 11, as shown in nationally set and marked reading and writing tests, taken by all children. Around four in five 11 year olds succeed in reaching this level in literacy, while around a third achieve the higher level 5. The Primary Literacy Framework, introduced in 1998, expanded on the National Curriculum for English. It detailed the content of teaching for each year and guided teaching methodology by dividing literacy teaching into word level, sentence level and text level work. The renewed Primary Literacy Framework, which is being introduced to schools this year, is organised in 12 strands that run through all the years. It is less prescriptive about teaching, gives more emphasis to speaking and listening, to building word recognition in the early stages of reading, and to the inter-relatedness of speaking, reading and writing. Learning objectives show progression within each strand and are related to year groups. The Primary Literacy Framework continues to sit within the National Curriculum for English and its level descriptors. The table below shows the structure of the new English Primary Syllabus for Malta, in comparison with England s National Curriculum for English and the Primary Literacy Framework. Malta: Primary English Syllabus (2005) Oracy listening and speaking stories, poems and songs language interactive conversations vocabulary England: National Curriculum English Speaking and listening speaking listening group discussion and interaction drama activities Reading England: Primary Framework for Literacy (2006) 12 Literacy strands (Speak and listen for a wide range of purposes in different contexts) speaking listening and responding 373

374 attitude Reading reading strategies reading vocabulary attitude Writing handwriting presentation skills writing strategies writing vocabulary attitude reading strategies understanding texts reading for information literature non-fiction and non-literary texts language structure and variation Writing composition planning and drafting punctuation spelling handwriting and presentation standard English language structure group discussion, interaction drama (Read and write for a range of purposes on paper and on screen) word recognition understand and interpret texts engage with, respond to texts creating and shaping texts text structure and organisation sentence structure, punctuation word structure, spelling presentation Comparison of the primary English curriculum in Malta and in England shows much that is similar. The following appear to be differences of emphasis: There is greater emphasis in Malta on listening to stories and poems, and on building up vocabulary. In England there is more focus on speaking and listening in different contexts and when working in groups, in Malta on clarity and correctness in conversation. Both include role play but drama has a higher profile in England (although this is a new development). More is expected in England about the use of information and communication technology across the curriculum. Outcomes related to attitude are explicit in Malta, and implicit in England. There is a strong recent shift in England towards phonics for word recognition in the early stages of reading rather than the use of a range of strategies. Focus on word recognition reduces beyond age 7, assuming early mastery and a later emphasis on language comprehension. The reading strategies in Malta 374

375 remain broad throughout (as in the original Primary Literacy Framework in England). In England, more emphasis is given to response to texts, and appreciation of writers use of language, making stronger connections between reading and writing. Malta focuses more on achieving grammatical knowledge and correctness. In England there is more explicit emphasis than in Malta on non-fiction, perhaps anticipating more cross-curricular linkage. Comparative expectations of achievement in English at the end of primary schooling. The expectation in England is that children will achieve Level 4 in English by the age of 11 when they complete primary school. Children are expected to achieve Level 3 by the age of 9. At the age of 10 (when their peers leave primary school in Malta) they would be expected to be at the boundary of Level 3 and Level 4 (ie 3A/4C: each level has three subdivisions for the purposes of annual assessment). Above average pupils at age 10 would be working at the high end of Level 4, in anticipation of reaching Level 5 by the age of 11. Results in reading and writing are brought together to give a level in English. However, when separated out, results at age 11 show only 67% reaching Level 4 in writing, compared with 83% in reading. There has been much work in recent years to try and close this gap which is particularly marked for boys. Malta Primary Syllabus Learning Outcomes for Year 6 SPEAKING AND LISTENING Enjoyment, interest and attentiveness during listening and viewing activities. Confidence in being able to understand spoken language without reference to the written form. Recite poems with correct intonation and pronunciation. Use appropriate language when participating in discussions on read aloud text. Initiate discussion and contribute to England National Curriculum Level Descriptors for end KS2 Level 3 Pupils talk and listen confidently in different contexts, exploring and communicating ideas. In discussion, they show understanding of the main points. Through relevant comments and questions, they show they have listened carefully. They begin to adapt what they say to the needs of the listener, varying the use of vocabulary and the level of detail. They are 375

376 conversation. Speak confidently with accuracy, clear diction and correct intonation in front of an audience. Appropriate and grammatically correct language to communicate meaningfully and with a purpose. Vivid descriptions of characters, scenes, objects and pictures using grammatically correct language. Explain the different steps of a simple procedure. Acting out a simple event. Develop and participate in dialogues using appropriate language. Understand and practice the conventions of social discourse. Use a range of vocabulary. Participate enthusiastically in oral activities. Demonstrate a positive view of their own opinions and appreciate the views of others. READING Knowledge of text organisation and book conventions. Ability and efficiency in accessing information from a range of sources. Integrate and adapt their reading strategies when reading difficult or unfamiliar text. Apply knowledge and use word identification strategies appropriately and automatically when encountering beginning to be aware of standard English and when it is used. Level 4 Pupils talk and listen with confidence in an increasing range of contexts. Their talk is adapted to the purpose: developing ideas thoughtfully, describing events and conveying their opinions clearly. In discussion, they listen carefully, making contributions and asking questions that are responsive to others' ideas and views. They use appropriately some of the features of standard English vocabulary and grammar. Level 5 Pupils talk and listen confidently in a wide range of contexts, including some that are of a formal nature. Their talk engages the interest of the listener as they begin to vary their expression and vocabulary. In discussion, they pay close attention to what others say, ask questions to develop ideas and make contributions that take account of others' views. They begin to use standard English in formal situations Level 3 Pupils read a range of texts fluently and accurately. They read independently, using strategies appropriately to establish meaning. In responding to fiction and non-fiction they show understanding of the main points and express preferences. They use their knowledge of the alphabet to locate books and find information. 376

377 an unknown word. Select and use prior knowledge suitable for each text. Participate in shared reading experiences. Read confidently, with fluency, expression and clear diction. Read and understand unfamiliar text. Use a wide range of monitoring and adjusting strategies to aid comprehension (eg skim and scan) Sustain an increasing bank of words that are recognised in different contexts. Discuss and recommend books and share opinions. Express preferences for a range of stories and non-fiction. WRITING Develop an individual handwriting style. Use a range of presentational features. Competence and efficiency in the use of various strategies to spell words correctly. Plan writing through brainstorming, classifying and organising information prior to writing. Write in a coherent and cohesive manner. Proof-read and edit own work; with support revise own work Use drawing and writing to complement and support each other. Participate in shared teacher/pupil(s) writing. Write for an audience and with a purpose. Level 4 In responding to a range of texts, pupils show understanding of significant ideas, themes, events and characters, beginning to use inference and deduction. They refer to the text when explaining their views. They locate and use ideas and information. Level 5 Pupils show understanding of a range of texts, selecting essential points and using inference and deduction where appropriate. In their responses, they identify key features, themes and characters and select sentences, phrases and relevant information to support their views. They retrieve and collate information from a range of sources. Level 3 Pupils' writing is often organised, imaginative and clear. The main features of different forms of writing are used appropriately, beginning to be adapted to different readers. Sequences of sentences extend ideas logically and words are chosen for variety and interest. The basic grammatical structure of sentences is usually correct. Spelling is usually accurate, including that of common, polysyllabic words. Punctuation to mark sentences full stops, capital letters and question marks is used accurately. Handwriting is joined and legible. Level 4 Pupils' writing in a range of forms is 377

378 Write expanded sentences according to ability. Create and write own book(s) and experiment with different genres. Understanding of the functions of sentence structure eg punctuation, parts of speech. Understanding of the function and relation of the words in context. Grammatically correct sentences. Write answers to questions on picture, topic, story and poem. Use of interesting, subject specific and evocative words to achieve specific effects. Enjoyment and motivation to participate in writing activities. lively and thoughtful. Ideas are often sustained and developed in interesting ways and organised appropriately for the purpose of the reader. Vocabulary choices are often adventurous and words are used for effect. Pupils are beginning to use grammatically complex sentences, extending meaning. Spelling, including that of polysyllabic words that conform to regular patterns, is generally accurate. Full stops, capital letters and question marks are used correctly, and pupils are beginning to use punctuation within the sentence. Handwriting style is fluent, joined and legible. Level 5 Pupils' writing is varied and interesting, conveying meaning clearly in a range of forms for different readers, using a more formal style where appropriate. Vocabulary choices are imaginative and words are used precisely. Simple and complex sentences are organised into paragraphs. Words with complex regular patterns are usually spelt correctly. A range of punctuation, including commas, apostrophes and inverted commas, is usually used accurately. Handwriting is joined, clear and fluent and, where appropriate, is adapted to a range of tasks. Despite the importance it is given in the curriculum in both countries, there is little formal assessment of speaking and listening against expected outcomes. The syllabus for Malta does not indicate any differences in approach stemming from the learning of English as a second language by most pupils. It may be that this is 378

379 reflected in a greater emphasis on building confidence in speaking and ensuring correctness in grammar and pronunciation. The English curriculum makes more assumptions about pupils mastery of spoken English, though requiring that pupils develop awareness of standard English and how it may differ from their own use of language. In reading, there seems to be a greater emphasis in England on the development of inference in comprehension (reading between the lines), and of exploring themes requiring an overview of a text or texts. In Malta there seems a rather stronger focus on the mechanics of reading, including the building of vocabulary. Again, this may be an appropriate emphasis for English second language learners. In England and in Malta, there are clear expectations about progress in writing and these are broadly similar at the age of 10. The strong linkage between speaking and listening, reading and writing in England leads to an emphasis on contexts for writing. There are also more explicit expectations in England about handwriting, emphasising fluency rather than individual style. Tests at the end of primary school in England National Curriculum tests in reading and writing are taken by all children at the age of 11, and are also available for the end of each primary year, for children aged 7, 8, 9 and 10. These tests include reading comprehension, a separate spelling test, and a writing test (including a shorter and a longer writing task, in narrative or nonnarrative form, reflecting the wide variety of types of writing that children are expected to have experienced). Language usage is not separated out from other questions. Tests are attractively presented, in colour, in keeping with pupils' normal work in literacy, and usually have a content theme that runs through them. The questions and the mark scheme draw closely on the National Curriculum level descriptors. Thus in England most children at the age of 11, and higher attaining pupils at the age of 10 would be expected in their reading comprehension to show understanding of significant ideas, themes, events and characters, beginning to use inference and deduction. to refer to the text when explaining their views. to locate and use ideas and information. 379

380 In testing writing, marks are allocated for sentence structure and punctuation, text structure and organisation, composition and effect, and also handwriting. In the writing test, most children at the age of 11, and higher attaining pupils at the age of 10 would be expected to demonstrate writing in a range of forms that is lively and thoughtful. ideas that are often sustained and developed in interesting ways and organised appropriately for the purpose of the reader. vocabulary choices that are often adventurous and words that are used for effect some use of grammatically complex sentences, extending meaning. accurate spelling, including polysyllabic words that conform to regular patterns correct use of full stops, capital letters and question marks, and some use of punctuation within the sentence. handwriting that is fluent, joined and legible The italicised characteristics above indicate Level 4 achievement in reading and writing (the requirement for 11 year olds). Tests for 10 year olds indicate that just over half reach this level in reading by the age of 10, which is comparable with the percentage passing the Junior Lyceum entrance exam at this age. However, only a third of pupils in England reach Level 4 in writing by age 10. Selective entrance examinations in England A variety of entrance tests is in use for the relatively small proportion of pupils (around one in ten) who attend selective state grammar schools or independent schools in England. The National Foundation for Educational Research provides tests for use in 11+ selection for grammar schools. The Independent Schools Examination Board produces syllabuses and sets Common Entrance exams for 11 and 13-yearold entry to independent schools, and many independent schools set their own tests. These exams take close account of the National Curriculum. English exams include reading and writing but separate testing of language usage is not normal practice. Broadly, the expectation would be that pupils would be achieving at least Level 4 and usually Level 5 at the age of 11, as described above, but these examinations would produce a ranking rather than assessment of attainment against a set of descriptors. 380

381 International assessment A Primary English Curriculum Framework, broadly based on the National Curriculum and Primary Literacy Framework, is produced by Cambridge International Examinations for international use as part of the Cambridge International Primary Programme. This is accompanied by a Primary Achievement Test for the end of primary education (in use since 2005) and Progression Tests for interim stages. The tests in English include reading, writing and language usage. 381

382 Appendix 10 MATHEMATICS Evaluation of the Junior Lyceum and Common Entrance papers 2006 and Junior Lyceum specimen paper 2007 Evaluation of the Primary Mathematics Syllabus (published 2005) Nicky Rushton Cambridge International Examinations 24 November

383 Evaluation of Junior Lyceum and Common Entrance Exams - Mathematics In this evaluation the Junior Lyceum exam and Common Entrance exam are assessed with regard to the 1997 syllabus for mathematics. This syllabus was used within schools from 1997 until 2006, when it was replaced with an updated syllabus. Its use was compulsory in schools in Malta. The analysis also considers the balance of the questions across the curriculum strands; the difficulty of the questions; and the construction of the paper. Junior Lyceum Entrance Examination 2006 The examination covers the 1997 syllabus through 20 questions. The initial 10 questions are short questions worth 4 marks in total, the final 10 questions are more complex and are worth 6 marks each. The specification grid produced for the examinations shows that the questions start at a low level, and gradually more difficult questions are introduced. Overall 27% of the questions are at a low level, 41% are at a medium level, with the final 32% available for the high level questions. The levels of difficulty allocated to the questions do not relate directly to any differentiation explicit in the syllabus. The balance of the examination is summarised in the specification grid. 39% of the marks were available for Number, 18% for Measurement, 14% for Shape and 9% for Data. A further 20% of the marks were available for problem solving. These percentages were ascertained through a process of matching the questions to specific learning outcomes in the 1997 syllabus. Analysis of the questions has shown that there are a number of errors within this process. Some questions were allocated to incorrect strands of the syllabus. Further questions were allocated to strands that didn t reflect their main assessment purpose. Further analysis showed that the proportions are closer to 42% for Number, 19% for Measurement, 7% for Data and 18% for problem solving. The allocation for Shape remained unchanged. These differences are small in terms of percentage points, but mean that in some cases the proportions are no longer within those recommended. This is most marked in the data handling questions. Therefore the marks available for data handling don t accurately reflect the importance and coverage that is given to this strand in the 1997 syllabus for mathematics. The individual questions are well matched to the syllabus, covering the majority of the year 6 syllabus. A wide range of question styles are used within the 383

384 examination, which adds interest into the paper. The use of quick questions that test the quick recall of number facts at the beginning of the paper helps to make it accessible to the lower ability pupils. The questions appeared to be pitched at an appropriate level for the majority of pupils upon reaching their final year at primary school. The specification grids showed the levels of individual marks awarded in the questions. The analysis of the questions showed that the levels given to these marks were appropriate. On the paper overall, the majority of the marks available are at the medium level, with slightly more high level questions than low level. This is appropriate within the context of this exam. However the breakdown within the strands is not always matched to this overall allocation. Within the number strand there are approximately equal numbers of questions at each of the levels, as would be expected. The measures questions have an emphasis at the medium level, with fewer low level questions than high level ones. Whilst not achieving the balance shown in number, this allocation is appropriate when the questions are compared to the syllabus. Both the Shape and Data questions are disproportionately biased towards the lower end of the assessment, with no higher level data question. This was connected to the earlier finding concerning the lack of data questions. The lack of higher level data questions is linked to the findings regarding the balance of the questions across the strands. The syllabus for Data Handling shows that opportunities exist for setting more difficult questions. The questions are generally clearly laid out. However the layout doesn t make clear the importance of showing working. The mark scheme allocates marks for choosing correct methods, and gives partial credit to pupils who are able to show some understanding of the question. For example in question 10, three marks are available to pupils. The first mark is available for attempting the correct division, the second mark for calculating the division, and the final mark for rounding up the final answer. Whilst space is given to the pupils in which to work out their answer, no mention is given on the paper to the use of spaces for this purpose, nor are the number of marks allocated to this question made explicit. Both of these would encourage the use of space for working. 384

385 Common Entrance The Common Entrance Examination is taken by around 1300 students, and around 450 pass. Pupils are ranked by totalling the marks from all five subject papers. There is no 'cut-off' threshold mark, and compensation between subjects would be possible. In Mathematics questions are not categorised or ordered according to level of difficulty, and the level of questions on some parts of the syllabus is more demanding than the Junior Lyceum. The specification for the examination states that the weightings of the strands and the levels of the questions are in line with those of the Junior Lyceum Examination. The summary statistics for these state that 26% of the questions were at low level, 39% at Medium level and 35% were at High level. This allocation confirms that the questions were slightly more difficult than the Junior Lyceum examination for the same year. Analysis of the syllabus shows that the Common Entrance Examination questions cover majority of the 1997 syllabus. However the analysis suggests that in common with the Junior Lyceum Examination there is too little emphasis on data handling in the questions. The level of the questions is appropriate, and they have been set to cover a range of levels. Generally the questions are slightly more difficult than the equivalent calculations on the Junior Lyceum Examination. Calculations are more complex, and often involve more than one step to solve them. A few of the questions extend pupils beyond the syllabus. An example is the speed question, which requires pupils to calculate a distance covered using two different speeds. The syllabus states that speed calculations should be done Through simple proportion using the unitary method. Exclude the formula for speed. The calculations within the examination question are not simple. The final part requires knowledge of how average speed is calculated and would probably be calculated using the formula. The marks for the Common Entrance are given on the paper. This gives pupils an indication of where multiple marks are available for one calculation. However the mark scheme is less generous than that of the Junior Lyceum. It still gives partial credit to pupils, but this is done on a basis of removing marks for each error, rather 385

386 than awarding marks for attempting a question and selecting the right method. This makes it harder for pupils to gain good marks on this particular examination. Evaluation of the 2007 Junior Lyceum Entrance Examination - Mathematics (specimen paper) The examination covers the 2006 syllabus through 28 questions. The initial 10 questions are short questions worth 1 mark each. The next 10 questions are worth 4 marks each and the final 8 questions are more complex and are worth 6 marks each. There is no specification grid available for this examination, and the questions haven t been allocated a difficulty level. This is the first examination that has been written using the new syllabus. The appearance of the examination remains largely unchanged, despite the change in the syllabus. The layout and order of the questions is very similar. In this examination the early questions have been allocated separate question numbers, rather than being parts of a larger question, but the actual tasks remain similar to the 2006 paper. Several questions from 2006 have been used within the 2007 examination, but with slight changes to the calculations required of the pupils. These common questions should allow performance to be compared between the two examinations. The questions generally use content from the year 6 syllabus, with a few questions referring back to year 5 syllabus. The level of the questions and the complexity increases throughout the examination. All the questions set are at a suitable level for pupils at the end of their primary education. As a detailed specification grid wasn t available, an analysis was carried out on the examination, matching questions to the syllabus. Many of the questions couldn t easily be matched to the learning outcomes, but did match onto the more detailed information contained within the programme. The questions cover all the strands of the syllabus, but certain sections of the syllabus are covered in more detail than others. Within the examination, questions on Number tended to emphasize the work done on fractions, decimals and percentages. Several questions test children s ability to recognise fractions and percentages, and calculate quantities using these fractions. The more difficult aspects of the fractions work covered in the syllabus (ordering 386

387 fractions and converting fractions to their simplest form) weren t covered in this particular examination. Calculation questions focused initially on mental methods, as is appropriate for short answer questions. Some later questions tested more difficult calculations which would have required pupils to use written methods, but generally these weren t as difficult as the examples given in the syllabus. For example the syllabus requires pupils to be able to divide Lm. c by a 2 digit number. The examination requires them to divide Lm6 by 27c which is a simpler calculation than is required in the syllabus. The question which requires use of written methods for addition also requires conversion of measures. It is the only question that covers decimal addition, but only if they initially convert all the measurements to decimals. The measures section of the syllabus is covered in several questions, but there don t seem to be many questions on Shape and Space. Two of the main year 6 Measures, Shape and Space objectives aren t covered in the questions, and most of the extra content covered in the programme but not the learning outcomes is not included in the examination. There are questions covering problem solving in the examination, but in general these are lower and medium level questions. There aren t any long problem solving questions for pupils to work through, unlike the 2006 examination. The balance of questions between high, medium and low level questions is difficult to comment on without a specification grid. The curriculum content in Malta isn t always covered at the same age as it would be in England. This means that making judgments about difficulty based on experience of the English curriculum isn t appropriate. The Junior Lyceum examination questions are spread across a variety of levels of difficulty. These levels seem to be spread out across the strands of the curriculum as well as throughout the paper. It appears to match previous requirements for the balance of question levels within the examination. The balance of questions across the 4 strands of the curriculum appears to match the balance set in Without the specification grid it is difficult to comment on the exact spread of questions, but a rough approximation shows that the balance is approximately right. Problems with the data questions targeting the easier parts of the syllabus haven t been replicated in this examination. 387

388 In general analysis of the Junior Lyceum examination for 2007 has shown it to be appropriate for use. It covers the year 6 syllabus, but the content does not go beyond it. The level of the questions should allow most children to access the paper. Evaluation of the Primary Mathematics Syllabus (published 2005) This evaluation considers the structure and content of Malta s Primary Mathematics Syllabus, in comparison with the curriculum in England. It then compares more precisely the expectations in Mathematics for the end of primary education in Malta with expectations at the equivalent stage in England. (Children finish primary at age 10 in Malta and 11 in England, so Year 6 in Malta is equivalent to Year 5 in England.) An overview of syllabus structure and content The new Mathematics syllabus for Malta is structured in terms of learning outcomes for each year, with notes that exemplify content to aid teachers planning. It is clearly organised in years from Year 1 to Year 6, with headings that run through all years. The National Curriculum for England is not broken into years but provides a programme of study in mathematics for each Key Stage: Key Stage 1 for Years 1 and 2, ages 5-7; Key Stage 2 for Years 3-6, ages Outcomes are separately described in terms of attainment targets for using and applying mathematics; number and algebra; shape, space and measures and data handling. In each of these areas there are descriptors for attainment across Key Stages 1 and 2, from Level 1 to Level 5. These levels are not age specific, but over the years the assessment regime has crystallised age-related expectations. The national benchmark is that children should reach Level 4 in mathematics by the age of 11, as shown in nationally set and marked mathematics tests, taken by all children. Around three in four 11 year olds succeed in reaching this level in Numeracy, while around a third achieve the higher level 5. The Primary Numeracy Framework for Teaching Mathematics, introduced in 1999, expanded on the National Curriculum for Mathematics. It detailed the content of teaching for each year, and provided a supplement of examples to aid teachers planning. The renewed Primary Numeracy Framework, which is being introduced into schools this year, is organised into 7 strands that run across all the years. Its content is largely unchanged, although the learning objectives have been updated to reflect the important aspects of mathematics to be included in teaching. These 388

389 learning objectives show the progression within each strand and are related to year groups. The Primary Numeracy Framework continues to sit within the National Curriculum for Mathematics and its level descriptors. The table below shows the structure of the new Mathematics Primary Syllabus for Malta, in comparison with England s National Curriculum for Mathematics and the Primary Numeracy Framework. Malta: Primary Mathematics Syllabus (2005) England: National Curriculum Mathematics England: Primary Framework for Mathematics (2006) Number and Algebra Measures, Shape and Space Data Handling Problem Solving Number Using and applying number Numbers and the Number system Calculations Solving numerical problems Shape Space and Measures Using and applying shape, space and measures Understanding properties of shapes Understanding properties of position and movement Understanding Measures Handling Data Using and applying handling 7 Numeracy strands Using and applying Mathematics Counting and understanding number Knowing and using number facts Calculating Understanding shape Measuring Handling Data 389

390 data Processing, representing and interpreting data Comparison of the primary mathematics curriculum in Malta (M) and England (E) shows that much is similar. There seems to be a similar emphasis given to the different sections of the curriculum in both E and M. The following are the key differences between the two: Pupils in M are more advanced in their written calculations, especially those involving decimals. These calculations are usually taught a year earlier in M than they would be in E. Mental calculation work is largely identical in M and E. Some of the exemplifications of the quick derivations contained within the curriculum in M would be taught a year later in E. Fraction work and the equivalence between fractions, decimals and percentages is also covered in M is also approximately 1 year ahead of the equivalent work in E. The curriculum in E would cover negative numbers through sequences and ordering. This isn t explicitly mentioned in the curriculum in M. The use of a calculator would be taught in year 4 in E, with a specific calculator paper in the tests taken in year 6. The curriculum for M doesn t contain any references to calculators. The new Primary Framework for Mathematics in E has caused a shift in some of the Shape content. This means that pupils in E would be covering perimeters at an earlier age than they would in M, with calendars covered later. Some of the properties work in shape would be covered a year later in E than it is in M. This includes knowledge of the angle sum of a triangle, and the sum of angles around a point. The data handling strand is introduced much later in M than it is in E. The strand is introduced in year 5 in M. In E children start data handling in Reception, and start constructing simple bar graphs and tables in year 1 (equivalent to year 2 in M). The data handling curriculum in M doesn t contain any references to probability. Pupils in E would be referring to the likelihood of events happening at this stage. 390

391 Finally, the curriculum in E covers mode in year 5, leaving mean until year 6. The curriculum in M starts with mean in year 6, making no mention of mode. Comparison of expectations of achievement in Mathematics at the end of primary school. The expectation in England is that children will achieve Level 4 in Mathematics by the age of 11 when they complete primary school. Children are expected to achieve level 3 by the age of 9. At the age of 10 (when their peers leave primary school in Malta) they would be expected to be at the boundary of Level 3 and Level 4 (3A/4C: each level has three subdivisions for the purposes of annual assessment). Above average pupils at age 10 would be working at the high end of Level 4, in anticipation of reaching Level 5 by the age of 11. The results are reported across Mathematics and are not broken down into Attainment Targets. Approximately 76% of pupils reach Level 4 at the age of 11, with no large differences in results between the genders. Malta Primary Syllabus Learning Outcomes for Year 6 England National Curriculum Level Descriptors for end KS2 NUMBER AND ALGEBRA Multiply and divide decimals mentally by 10 or 100, and integers by 1000, and explain the effect. Reduce a fraction to its simplest form by cancelling common factors. Use a fraction as an operator to find fractions of numbers or quantities. Solve simple problems involving proportion. Order a mixed set of numbers with up to three decimal places. Understand percentage as the number of parts in every 100. Find simple percentages of small whole-number quantities. Level 3 Pupils show understanding of place value in numbers up to 1000 and use this to make approximations. They begin to use decimal notation and to recognise negative numbers, in contexts such as money and temperature. Pupils use mental recall of addition and subtraction facts to 20 in solving problems involving larger numbers. They add and subtract numbers with two digits mentally and numbers with three digits using written methods. They use mental recall of the 2, 3, 4, 5 and 10 multiplication tables and derive the associated division facts. They solve whole-number problems involving multiplication or division, including those that give rise to remainders. They use simple fractions that are several parts of a whole and recognise when two simple fractions are equivalent. Level 4 Pupils use their understanding of place value to multiply and divide whole numbers by 10 or 100. In solving number problems, pupils use a range of mental methods 391

392 Extend written methods to column addition and subtraction of numbers involving decimals. Derive quickly: division facts corresponding to tables up to Extend written methods to: short multiplication of numbers involving decimals; long multiplication of a 3-digit by a 2-digit integer; Short division of numbers involving decimals. of computation with the four operations, including mental recall of multiplication facts up to 10 x 10 and quick derivation of corresponding division facts. They use efficient written methods of addition and subtraction and of short multiplication and division. They add and subtract decimals to two places and order decimals to three places. In solving problems with or without a calculator, pupils check the reasonableness of their results by reference to their knowledge of the context or to the size of the numbers. They recognise approximate proportions of a whole and use simple fractions and percentages to describe these. Pupils recognise and describe number patterns, and relationships including multiple, factor and square. They begin to use simple formulae expressed in words. Pupils use and interpret coordinates in the first quadrant. Level 5 Pupils use their understanding of place value to multiply and divide whole numbers and decimals by 10, 100 and They order, add and subtract negative numbers in context. They use all four operations with decimals to two places. They reduce a fraction to its simplest form by cancelling common factors and solve simple problems involving ratio and direct proportion. They calculate fractional or percentage parts of quantities and measurements, using a calculator where appropriate. Pupils understand and use an appropriate non-calculator method for solving problems that involve multiplying and dividing any three-digit number by any two-digit number. They check their solutions by applying inverse operations or estimating using approximations. They construct, express in symbolic form, and use simple formulae 392

393 involving one or two operations. They use brackets appropriately. Pupils use and interpret coordinates in all four quadrants. MEASURES, SHAPE AND SPACE Understand area measured in square centimetres. Understand and use the formula in words, length breadth for the area of a rectangle. Calculate the perimeter and area of simple compound shapes that can be split into rectangles. Use a protractor to draw acute and obtuse angles to the nearest 5º. Level 3 Pupils classify 3-D and 2-D shapes in various ways using mathematical properties such as reflective symmetry for 2-D shapes. They use non-standard units, standard metric units of length, capacity and mass, and standard units of time, in a range of contexts. Level 4 Pupils make 3-D mathematical models by linking given faces or edges, draw common 2-D shapes in different orientations on grids. They reflect simple shapes in a mirror line. They choose and use appropriate units and instruments, interpreting, with appropriate accuracy, numbers on a range of measuring instruments. They find perimeters of simple shapes and find areas by counting squares. Level 5 When constructing models and when drawing or using shapes, pupils measure and draw angles to the nearest degree, and use language associated with angle. Pupils know the angle sum of a triangle and that of angles at a point. They identify all the symmetries of 2-D shapes. They know the rough metric equivalents of imperial units still in daily use and convert one metric unit to another. They make sensible estimates of a range of measures in relation to everyday situations. Pupils understand and use the formula for the area of a rectangle. DATA HANDLING Solve a problem by extracting and interpreting data in tables, Level 3 Pupils extract and interpret information presented in 393

394 graphs and charts. simple tables and lists. They construct bar charts and pictograms, where the symbol represents a group of units, to communicate information they have gathered, and they interpret information presented to them in these forms. Level 4 Pupils collect discrete data and record them using a frequency table. They understand and use the mode and range to describe sets of data. They group data, where appropriate, in equal class intervals, represent collected data in frequency diagrams and interpret such diagrams. They construct and interpret simple line graphs. Level 5 Pupils understand and use the mean of discrete data. They compare two simple distributions, using the range and one of the mode, median or mean. They interpret graphs and diagrams, including pie charts, and draw conclusions. They understand and use the probability scale from 0 to 1. Pupils find and justify probabilities, and approximations to these, by selecting and using methods based on equally likely outcomes and experimental evidence, as appropriate. They understand that different outcomes may result from repeating an experiment. PROBLEM SOLVING Identify and use appropriate operations (including combinations of operations) to solve word problems involving numbers and quantities. Level 3 Pupils try different approaches and find ways of overcoming difficulties that arise when they are solving problems. They are beginning to organise their work and check results. Pupils discuss their mathematical work and are beginning to explain their thinking. They use and interpret mathematical symbols and diagrams. Pupils show that they understand a general statement by finding particular examples that match it. Level 4 394

395 Pupils are developing their own strategies for solving problems and are using these strategies both in working within mathematics and in applying mathematics to practical contexts. They present information and results in a clear and organised way. They search for a solution by trying out ideas of their own. Level 5 In order to carry through tasks and solve mathematical problems, pupils identify and obtain necessary information. They check their results, considering whether these are sensible. Pupils show understanding of situations by describing them mathematically using symbols, words and diagrams. They draw simple conclusions of their own and give an explanation of their reasoning. Within Number and Algebra the advanced teaching in calculations in M is reflected within the learning outcomes, many of which map across to level 5 of the English National Curriculum. From the learning outcomes it would appear that no formal assessment is made of mental calculation skills in addition and subtraction in M. The knowledge of number facts is assessed in both countries, but the English National Curriculum appears to be the only one that assesses the use of these skills within calculations. The curriculum in Malta also appears to have less emphasis on beginning of Algebra. In England this is assessed through number patterns and relationships. Whilst this is covered within the programme of learning in Malta, it isn t covered within any of the learning outcomes. The outcomes for Measures, Shape and Space contain less detail that their equivalents within the English National Curriculum. Work on measures only has one specific outcome relating to the ability to draw angles to the nearest 5 degrees. No mention is made of the accuracy required in using other measuring instruments. The programme contains detailed learning outcomes that refer to the units of measurement, conversions between them and practical measurement. None of these skills are represented in the learning outcomes. 395

396 In England the National Curriculum contains very explicit requirements in the Data Handling strand. These detail the methods of collecting, representing and interpreting data that are appropriate at each level. In contrast the syllabus in Malta merely requires children to extract and interpret data. The programme contains some examples of graphs and charts, but doesn t make the requirements for the interpretation of these more explicit. The problem solving outcome in the syllabus for Malta reflects a strong emphasis in the skill of solving word problems with no specific mention of strategies and recording. The emphasis in England is on pupils developing strategies and recording these. Analysis of the learning outcomes in M reveals that there are several areas of the curriculum that are included in the programme, but don t have a related learning outcome. This is particularly noticeable in Measures, Shape and Space, but occurs in the other strands too. If the outcomes are used as a measure of assessing the children at the end of primary education, this could mean that large parts of the curriculum remain unassessed. Tests at the end of primary school in England National Curriculum tests in mathematics are taken by all children at the age of 11, and are also available for the end of each primary year, for children aged 7, 8, 9 and 10. These tests include a mental mathematics test and two written mathematics tests. Children are allowed to use calculators in one of the written tests. The questions and the mark scheme draw closely on the National Curriculum level descriptors. Selective entrance examinations in England A variety of entrance tests are in use for the relatively small proportion of pupils (around one in ten) who attend selective state grammar schools or independent schools in England. The National Foundation for Educational Research provides tests for use in 11+ selection for grammar schools. The Independent Schools Examination Board produces syllabuses and sets Common Entrance exams for 11 and 13-yearold entry to independent schools, and many independent schools set their own tests. These exams take close account of the National Curriculum. International assessment A Primary Maths Curriculum Framework, broadly based on the National Curriculum and Primary Numeracy Framework, is produced by Cambridge International 396

397 Examinations for international use as part of the Cambridge International Primary Programme. This is accompanied by a Primary Achievement Test for the end of primary education (in use since 2005) and Progression Tests for interim stages. More information on this is available at: (qualifications/cipp). 397

398 Appendix 11 FES Programmes The Hilti Programme is a a community-based primary prevention after school family literacy initiative. After-school family literacy clubs called Hilti Clubs are aimed at pupils in the early primary years and their parents. Hilti Clubs are based in state primary schools and run from 2.30pm to to 4.15pm. At these clubs, trained FES personnel use differentiated teaching methodologies within a mixed ability and family literacy context. Hilti Clubs currently operate in 23 community-based primary schools and reach around 400 children and their parents per scholastic term. A team of FES teachers trained in family literacy approaches and Activity Teachers employed by the Education Division run each Hilti Club; teachers who co-ordinate these clubs participate in a 112 hour training programme. Club activities are specifically designed with an in-built literacy and/or numeracy component that the children immerse themselves in as an integral part of the fun activity. The NWAR Programme is a secondary prevention family literacy initiative. The Nwar Service is currently operational in six regional centres: Birgu, B Kara, Hamrun, Tarxien, Rabat and Gozo. Through NWAR, parents and their children with severe reading and writing difficulties go through an assessment process and subsequently participate in an intensive one-to-one/two literacy support programme over a period of one semester, with the possibility of extended service, if necessary. The service is national in scope and accepts referrals mainly from the Statementing Moderating Panel, the Ministry of Education, and various psycho-social services within the Education Division. Individual learning plans are prepared, implemented and reviewed with the parents of each referred child. Programm Id f Id is a parent capacity building and empowerment initiative. Parents of children participating in a Hilti family literacy Club are encouraged to learn and practice, in a paired work setting, practical tools that stimulate their children to learn more effectively. Parents have the opportunity to meet and discuss educational strategies with teachers twice weekly through the after-school family literacy centres. They also join their children during the club to practice some of the tools. Following participation in this process, many parents decide to engage in other non-formal learning opportunities offered by FES and other organisations. FES provides in-school technical assistance to those primary and secondary schools that plan to develop in-house literacy strategies and tools. Initiated in January 2003, 398

399 the programme now provides technical assistance to a number of schools including the Boys and Girls schools for low-achieving secondary level students where interschool collaborative measures have been set in motion. Monthly inter-team meetings are held in the latter schools on a rotating basis. Peripatetic literacy teachers attached to the Literacy Unit at the University of Malta also attend such meetings in order to actively contribute to inter-agency collaboration in this key area. Learning targets have been collaboratively produced by Form 1 and Form 2 teachers from the Boys and Girls Schools for Maltese, Maths, and English. Resource materials are being published for use by these schools. FES shall focus on working hand-in-hand with schools in the Gozo island and Cottonera inner-harbour regions to develop and implement basic skills action plans. FES has also focused on the development and provision of demand driven specialist teacher training opportunities in such areas as differentiated teaching methodologies, parental participation in the school community, the teaching of creative writing, the use of phonics especially with children grappling with literacy skills, and individualised learning plans for students with literacy difficulties. 399

400 Appendix 12 Records of Achievement: Reporting Pupils Progress using Level Descriptors The plan of action for assessment recommends the reporting of the pupils progress in terms of levels of achievement. This presumes that Level Descriptors for the various areas of the primary and secondary curricula are available and that teachers are trained to use them. Copies of the Level Descriptors have already been circulated by the Department of Curriculum Management of the Education Division in 2003 and the Department is in the process of developing them further to facilitate their use by class teachers. Teachers can assess their pupils level of achievement by judging how far the knowledge and skills that the pupils demonstrate in their work in a particular curriculum area during the year, in the examination or in any other formal or informal assessment agrees with the description of performance at a particular level. The teacher s judgment is then recorded in the pupils record of achievement. This can take the form of a booklet with one page for each curriculum area. It can also form part of a computerised database in which all data about the pupils are entered into the database, which can be programmed to produce individualised reports. The record of achievement will include a summary of the knowledge, skills or competences that the pupil demonstrates during the year. The summary could take the form of a Table, which shows the level and sub-levels reached by the pupil in a curriculum area, and a description of the level and sub-levels. At primary level, the achievement in the ten curriculum areas will be recorded, that is, English, Maltese, Mathematics, Religion, Social Studies, Science, Expressive Arts, PE, PSD, ICT skills. At secondary level, records of achievement will be compiled in Forms 1 and 2 only and cover the core subjects (English, Maltese, Mathematics, Religion, Integrated Science, PSD, History/Geography/Social Studies PE, ICT and the five optional subjects. The record of achievement should be informative to learners and parents so that from the level descriptors they can judge what progress has been achieved and what the next targets are. The teacher may add comments on the pupil s effort and other comments on the aspects which need improvement. Examples 1 and 2 below are given as an illustration of the possible format of one page from the record of achievement of a Year 4 primary school pupil and a Form 1 secondary school pupil respectively. The first example is completed to illustrate the record in English of a Year 4 pupil. 400

401 Raħal X Primary School Record of Achievement end of Year 4 Scholastic Year Name of Pupil: John Vella Class: Year 4 Yellow Date of Birth: 10 July 1999 Expected Average: 2c Curriculum Area: English Level 2 Level 3 a b c a b c Speaking Listening Reading Writing a b c a b c a b c a b c a b c a b c Comments: John is progressing well in English though he must make greater effort in writing. He needs to read more widely to be able to generate ideas for writing and to produce sentences on various themes. He needs to pay more attention to his homework since he seems to produce it in a rush and makes many silly mistakes. Levels: During their Primary education most learners are expected to progress from Level 1 to Level 4. During Year 4 they are expected to conclude Level 2. However learners progress at different speeds and some might be a little more advanced and some a little behind. Level 2: Learners listen and respond correctly to questions on simple stories, songs, rhymes and other text related to themes. They memorise rhymes, songs and chants. They participate actively in speaking tasks such as role-play and retell familiar stories with support. They express preferences and engage in simple dialogues. They show knowledge of all the letters and sounds of the alphabet and read words, phrases and sentences in context. With support, they generate ideas and start to produce sentences on various themes according to ability. Level 3: Learners listen with increasing span of concentration, to longer stories and texts and make appropriate oral responses. They can understand and us a variety of sentence structures showing an ability to interact with various texts. Learners actively participate in shared and guided reading and exhibit greater confidence and fluency in reading more extended texts, including poetry. They begin to show the ability to recognise different writings for different purposes. They begin to construct coherent narrative and descriptive writing in shared and guided tasks. Sub-levels: a: The learner has just reached the level. b: The learner shows competent and consistent use of skills at this level. c: The learner shows highly competent use of skills at this level and is moving towards the acquisition of skills of the next level. Signature of Class Teacher Date 401

402 San Xmun Secondary School Record of Achievement end of Form 1 Scholastic Year Name of Pupil: Joseph Borg Class: Form 1 Green Date of Birth: 11 July 1997 Expected Average: 5b Curriculum Area: Mathematics Level 4 Level 5 a b c a b c Number work Shapes and space Algebra and Graphs Problem solving a b c a b c a b c a b c a b c a b c Comments: Joseph must make greater effort especially in Algebra and Graphs. He needs to spend more time on these topics and work out more examples starting with the simplest and progressing to harder ones. He is good in number work but over-confident in the other topics. He needs to pay more attention to homework and to rely less on his fellow students during class work. Levels: During their Secondary education most learners are expected to progress from Level 5 to Level 8. During Form 1 they are expected to conclude Level 5. However learners progress at different speeds and some might be a little more advanced and some a little behind. Level 4: Learners use the number system confidently, and recognise and describe number patterns and relationships. They calculate fluently with all four number operations, including fractions, decimals and percentages in multiples of 10 and 25. They understand, estimate, measure and calculate simple features of shape and space such as perimeter, area, volume and size of angle. They present information and results in a clear and organised way while searching for solutions to routine and non-routine problems by trying out their own ideas. Level 5: Learners extend their calculating skills to involve larger numbers, fractions, percentages, decimals, and show understanding of the importance of proportional reasoning. They start to make use of algebraic symbols and understand linear functions with their corresponding graphs and show a deeper understanding of the features of shape and space using definitions and reasoning to understand geometrical objects. They begin to understand reasoned arguments, communicating mathematics orally and in written form (in English) to explain their reasoning to others. They carry out simple analyses of data, make simple inferences and understand and apply the probability scale. They start using ICT to explore mathematical ideas. Sub-levels: a: The learner has just reached the level. b: The learner shows competent and consistent use of skills at this level. c: The learner shows highly competent use of skills at this level and is moving towards the acquisition of skills of the next level. Signature Teacher of Mathematics Date 402

403 Appendix 13 Measures to Improve Students Learning in Maltese Secondary Schools (when reform takes place) Introduction Leonard Grech Low-achieving students span the full range of intellectual abilities and gifts. Some are highly intelligent in multiple areas but become bored with the traditional academic experiences encountered in schools. For very bright low-achievers, the classroom instructional experience lacks sufficient challenge and does not further their intellectual endeavors. For students who are not as intellectually precocious, the classroom does not support their learning needs. These experiences fail to stimulate these low achievers' interests in learning. Relevance for learning, the appropriate cognitive challenge, and the personal needs of the learner are too often not addressed in the design and delivery of instruction. Low achievement in schools manifests itself as refusing to participate in class activities, failing to complete homework, displaying hostility and disruptive behavior in class, day-dreaming or exhibiting poor attention spans, making little or no eye contact with the instructor, constantly offering excuses for incomplete assignments, and having excessive unexplained absences. Low achievers simply do not see the need to participate in the school experience (Stiff-Williams, 2002, p. 19). A. Curriculum Development There is no reason why the only, or even the best, ways of furthering [educational] aims should be traditional academic subjects. There are other vehicles projects, topics, practical enterprises, interdisciplinary work, and, not least, whole school processes (White, p.10). It could be that in Malta we are giving too much importance to academic subjects which have become compartmentalised, we are trying to specialise too early and we are missing out on giving a general education during the secondary phase. We should therefore examine the curriculum and reflect on whether there are areas of syllabuses which should be revised either in content or in the way that they are being presented to students. Considering the wide range of abilities among students, it is only fair that we differentiate between them and we do not adopt the maxim that one fits all. Perhaps students referred to as lower ability students, offer the greatest challenge. We should ask ourselves whether the study of another language apart from Maltese and English is recommendable for all secondary school 403

404 students, whether more emphasis should be given to the spoken rather than to the written language, whether the choice of certain texts in Literature (for example Shakespeare) is recommendable, whether Physics should be substituted by Co- Ordinated Science, or whether in Social Studies there should be greater emphasis on the environment and on citizenship, just to mention a few examples. Could it be that our concept of secondary education is still that of a Grammar school? 1. making the curriculum sufficiently interesting and fascinating so students are motivated by the nature of the tasks, is central to developing students interest in learning (Ollerton, 2004, p.77). It would be very fruitful for us to introduce new topics which are closer to students everyday experiences. Vocational topics such as Home Maintenance and Craft could help us bridge the gap between school and work expectations. Also very appropriate would be Home Economics and various other areas of interest to students. (refer to suggested timetable). Such a system of study would be best catered for if we introduce modules. Students would be assessed on each module and presented with a certificate so as to increase their motivation. It is necessary to understand that our secondary schools must provide students with an appropriate education that will prepare them for future courses at the Sixth Form / Junior College (academic) or at MCAST (Vocational). 2. Design and Technology should be made available to all students following secondary education. We should also study the possibility of teaching and assessing the subject through the medium of the Maltese Language for those groups of students who opt for this choice. 3. Topics forming the syllabuses should be relevant to everyday life and presented as such. For example in History we should give preference to events taking place after World War 2 rather than go back to times with no direct impact on the present situation. 4. Language skills, namely listening, speaking, reading and writing, as well as numeracy, Science and ICT skills should be emphasised. Moreover, being the medium of many other areas of learning, English should also be given 404

405 particular importance. Perhaps more time should be dedicated to English than at present, particularly to the oral and aural aspect. 5. There are also other skills that are fundamental to learning. It would perhaps be a good idea to dedicate a whole afternoon every week to the teaching and learning of these skills, a list of which is to be found at the end of this document. The acquisition of such skills will train our students to become critical thinkers and independent learners. It will greatly contribute to a more holistic education for our students. 6. Importance also needs to be given to basic concepts. Nowadays we are using the term concept attainment. Students are asked to organise and re-organise what they know, and to add to what they know. (One has to realise that this is an important skill in itself.) Concepts such as healthy living, sustainable development of the environment, and respect for diversity are but a few of these concepts that might interest students. (Here one has to emphasise not only the wise choice of content but also the method of teaching that should be applied. Refer to section B). 7. Teaching topics where history, geography, design and technology, and art, as well as literacy skills and sometimes mathematics come together is a desirable and achievable challenge (Ollerton, 2004, p.37). It would be recommendable to introduce a new area of study on the timetable - General Knowledge - which lends itself to an Integrated Curriculum and involves teaching in a crosscurricular way. It also involves practical sessions and hence greater motivation on the part of students. One must identify areas of study which are of interest to students. For each area of study they follow, students are presented with a certificate. Examples of areas of study may include: Making a video Buying a digital camera Surfing on the web Keeping a hobby Basic electricity Basic Carpentry Basic Printing 405

406 Farming / Gardening Inside the Kitchen Public Speaking The Climate Eating Healthily The Value of Drama Different kinds of Music Young People s Poetry Art Appreciation Drugs: their use and abuse 8 Focusing on Competencies: Building Stones (we have to introduce new and unfamiliar terminology which appears to be fresh and to belong to the 21 st century). Students themselves choose a topic which involves the learning of various skills, and they work out a project in a practical and realistic manner as if it were the real thing. Examples of titles which may be chosen: Organising a Seminar, Holding a Marathon, Preparing for a One Week Camping, and Collecting from Six Different Sources Information about the Introduction of the Euro in Malta. In each case the success criteria is developed and made available to all concerned. During the activity the teacher may be called The Coach. An afternoon every week may be dedicated to these activities which are very much student centred. The learners themselves would list the skills that are to be learnt, and assess their own work under the supervision of the coach. At the end of the activity they are given a certificate that is graded from 1 to 4. 1 = Beginner 4 = Expert B. Teaching and Learning Motivation plays a crucial role in the teaching and learning process. But as Dylan Wiliam points out (2006, p. 5) we have to think of motivation not as a cause of achievement, but as an outcome. Motivation takes place when the level of challenge is just at the limit of your competence. When the level of competence is high, and the level of challenge is low, the result is boredom, and when the level of competence is low, and the level of challenge is high, one faces alienation. So we have to be realistic and examine what is happening in the classroom. 406

407 This kind of teaching [formative assessment] is designed to meet the needs of each pupil personally, but in the context of whole classroom teaching rather than individual tuition (Brooks & Tough, 2006, p.16). Characteristics of Formative Assessment Having clear learning objectives (step by step learning) Sharing learning objectives with learners Establishing Success Criteria (shared by teacher and learners) Questioning technique Students active participation (engaging students, ownership of their own learning) Concept mapping (What do I know? What can I add to what I know?) Brainstorming. Feedback, which includes marking learners work. Scaffolds: These strategies include modelling, coaching, inviting verbal articulation of newly learned material, and engaging students in reflection. Stiff-Williams (2002, p. 20) says that When given opportunities to massage newly learned material, low achievers extend their skills in learning. Scaffolds support the learning process by helping students to assimilate new content and extend their skills. Scaffolds work like the training wheels on a child s bicycle by providing support until the learner can advance independently. Peer Assessment Self Assessment Plenary Session The above has to be reinforced by Continuous Professional Staff Development which, in this area of formative assessment is still in its infancy. Evidence of surveys of teacher practice shows that formative assessment is not at present a strong feature of classroom work. It follows that to establish good formative assessment practices in classrooms requires that most teachers make significant changes (Black et al., 2003, p.2). There is close correlation between teaching and students achievement. 407

408 Increased learning and achievement among low-achieving students depends heavily on the quality of instruction they receive (Stiff-Williams, 2002, p. 19). It is necessary to invest in teachers. if you re serious about raising student achievement, then you have to be serious about improving teacher quality (William, 2006, p. 3). A great deal of support to teachers and learners is also required. It is far better to invest financially in schools in preventive measures than to invest greater sums of money later on in remedial action. There are other factors that play a significant part: Management and especially the leadership of the school Ethos Resources (Human and Physical) All the above will hopefully result in + better student performance and hence the raising of standards + an improvement in classroom management + an increase in teacher job satisfaction and morale + better public image of schools. References Black, Paul, et al., (2003) Assessment for Learning: Putting it into Practice, Open University Press, Maidenhead Philadelphia. Brooks, Richard and Tough, Sarah, (2006) Pupil Attainment: Time for a three Rs guarantee, Institute for Public Policy Research, London. [Online], Available: %5FATTAINMENT%2Epdf [30 May 2007]. Ollerton, Mike, (2004) Creating Positive Classrooms, Continuum, London. Stiff-Williams, Helen R., (November 2002) Get Their Attention, Principal Leadership (pp ), National Association of Secondary School Principals, Reston Virginia. [Online], Available: [4 June 2007]. White, John, (N.D.) The Aims of School Education, Institute for Public Research Policy, London. [Online], Available: %20of%20School%20Ed%20FINAL.pdf [5 June 2007]. 408

409 Wiliam, Dylan, (2006) Assessment for Learning: why, what and how. (Edited transcript of a talk at the Cambridge Assessment Network Conference on 15 September 2006 at the Faculty of Education, University of Cambridge.) [Online], Available: Search dylan wiliam s web site and go to publications, [1 June 2007]. 409

410 The Curriculum (focusing on Primary and Secondary Education) We need to ask the following questions: 1. What sort of person do we want a child to become? 2. So what are the aims of education? 3. How best to achieve these aims? 4. Are traditional subjects satisfying the aims of education? 5. Which aims are not being catered for? A list of skills/competencies apart from Literacy, Numeracy, ICT and Physical skills. A Mental skills 1. observing 2. developing an eye for detail 3. describing 4. sorting, grouping, categorising 5. matching 6. finding similarities and differences 7. comparing and contrasting 8. remembering 9. reproducing 10. making a list 11. compiling 12. sequencing 13. following instructions 14. asking oneself relevant questions 15. prioritising 16. focusing 17. following a line of thought 18. recognising logical fallacies in reasoning 19. selecting 20. breaking a task into smaller parts 21. deconstructing 22. finding identical situations 23. finding patterns 24. following patterns 25. being able to generalise 26. outlining 27. recognising specific facts 28. eliciting information (recalling) 29. detecting contradictory information in a text or speech 30. detecting bias 31. assessing validity 32. assessing reliability 33. asking for information 34. analysing 35. inferring 36. differentiating between facts, opinions and inferences 37. interpolating 38. extrapolating 39. reading between the lines 40. interpreting 41. estimating 410

411 42. predicting 43. imagining 44. concentrating 45. evaluating 46. finding advantages and disadvantages 47. organising oneself 48. forming a criteria 49. assessing oneself and others against a criteria 50. identifying areas for improvement 51. taking remedial action to affect improvement 52. investigating 53. recognising hidden meanings 54. coming to conclusions 55. being original 56. being inquisitive 57. synthesizing 58. outlining the relationship among components 59. recognising the relationship between cause and effect 60. putting parts together to make a whole 61. recognising the bigger picture 62. identifying strengths, weaknesses, threats and opportunities 63. identifying problems 64. finding solutions 65. translating 66. applying 67. checking 68. revising 69. modifying 70. finding new ways to improve outcome 71. working according to plan 72. being well organised B Mental skills with social implications 1. expressing oneself 2. exposing an argument 3. being concise 4. explaining to others 5. giving instructions to others 6. being well organised and organising others 7. taking the initiative 8. narrating 9. describing 10. arguing 11. writing a message 12. repeating a message 13. writing a report 14. managing time 15. asking for relevant information 16. giving the relevant information 17. finding the relevant information 18. handling of information 19. organising data 20. investigating and designing 21. evaluating and reflecting 22. deciding 411

412 23. being entrepreneurial 24. understanding the environment 25. understanding one s place in society C Personal Skills 1. knowing safety rules 2. adopting healthy life style 3. understanding one s emotions 4. controlling emotions 5. understanding one s needs 6. understanding one s ambitions 7. coming to terms with one s situation 8. understanding life s blessings and setbacks 9. knowing when and how to ask for help 10. preparing oneself for the unpredictable 11. facing change 12. being proactive 13. respecting oneself 14. sharing 15. co-operating 16. exploring alternatives 17. deciding 18. forming a strategy (plan of action) 19. implementing decisions 20. facing consequences 21. developing confidence and self-esteem D Affective Skills 1. communicating 2. putting oneself in someone else s shoes 3. playing a role 4. empathising 5. appreciating others 6. being sensitive to the needs of others 7. responding to people s needs 8. understanding the feelings of others 9. respecting values of other people 10. treating persons as individuals (for example remembering their names) 11. recognising the need for balance between personal freedom and responsible behaviour 12. accepting responsibility for one s behaviour 13. accepting professional ethical standards 14. accepting one s abilities and limitations 15. leading a life which is in harmony with one s abilities, interests, and beliefs 16. prioritising time effectively to meet the needs of the organisation, family, and self. 17. defending the rights of others 18. co-operating in group activities 19. displaying a professional commitment to ethical practice on a daily basis 20. revising judgements and changing behaviour in light of new evidence 21. valuing people for what they are, not how they look 412

413 E Social Skills 1. speaking in public 2. debating 3. listening to others 4. being assertive 5. working in a team 6. understanding norms 7. managing 8. leading 9. communicating with others 10. understanding others point of view 11. appreciating the value of ideas 12. respecting cultural diversity 13. analysing a situation 14. detecting hidden agendas 15. detecting propaganda 16. detecting peer pressure 17. valuing democracy 18. valuing standards of behaviour 19. appreciating rights and duties 20. understanding one s role in society Reference: Bloom s Taxonomy 413

414 Suggested Timetable for Secondary School Students Forms 1 & 2 Number of Lessons Subject/s or Area of Study TOTAL: 40 Lessons 3 Maltese (including Literature) 6 English (including Literature) 5 Mathematics 2 The Environment & General Science 2 Religion 3 PSD & Citizenship (this includes Entrepreneurship & Health Educ.) 2 Home Economics 2 Physical Education & Sports 2 History & Geography 2 Design & Technology 2 ICT 2 Art, Music, & Drama 3 Foreign Language (optional) 2 Building Stones (this focuses on skills and competencies) 2 Modules (Choice from a wide variety of topics from various areas of study, such as: (a) Tourism Studies (b) Agriculture & Animal Husbandry (c) Media Studies (d) Textiles & Design (e) Art, Design & Craft (f) Health, Hair & Beauty (g) Care & Community Studies (h) Performing Arts (i) Mechanical, Electrical & Technical Studies 414

415 415

416 416