Trends in effective enrolment Measuring access and basic-quality improvements in education for nine African countries,

Transcription

1 RESEARCH PAPER Trends in effective enrolment Measuring access and basic-quality improvements in education for nine African countries, Nicholas Spaull 1 & Stephen Taylor, September Introduction In the modern discourse surrounding economic development in Africa, few ideas have reached such a level of consensus as the unimpeachable notion that educational opportunity should be extended to all children. As a result, few would challenge the validity of the goal to universalize access to primary schooling as exemplified in the Education For All (EFA) drives of the 1960s and 70s, and more recently as codified in the Millennium Development Goals. This is primarily due to widespread recognition that economic and social development is not possible without increased access to education, and an improvement in the quality thereof. Lewin (2007, p. 2) summarizes this view as follows: Fairly universally poverty reduction is seen as unlikely unless knowledge, skill and capabilities are extended to those who are marginalised from value-added economic activity by illiteracy, lack of numeracy, and higher level reasoning that links causes and effects rationally. In most societies, and especially those that are developing rapidly, households and individuals value participation in education and invest substantially in pursuing the benefits it can confer. The rich have few doubts that the investments pay off; the poor generally share the belief and recognise that increasingly mobility out of poverty is education-related, albeit that their aspirations and expectations are less frequently realized (Lewin, 2007, p. 2). The only caveat to this consensus is the widely held belief that there exists some trade-off between expanding access on the one hand, and declining educational quality on the other. However, scrutiny of the most recent cross-national datasets of educational access and educational quality shows that there is less empirical support for this belief than was traditionally thought to be the case (Taylor & Spaull, 2013), as will be shown in this policy research note. Despite large increases in access to education over the 2000 to 2007 period, all countries in the sample also managed to increase the number of children acquiring basic numeracy and literacy skills. This research note uses the methodology developed in Spaull & Taylor (2012) to combine measures of educational access (enrolment) with educational quality (numeracy and literacy). The new composite statistic - called effective enrolment - is the proportion of the entire age-appropriate population that has 1 This paper was commissioned by Save the Children as an input paper to the Getting to Zero report ( Any comments or queries should be directed to W.Paxton@savethechildren.org.uk and the corresponding author, NicholasSpaull@gmail.com. 1

2 reached some basic threshold of numeracy and literacy proficiency whether in school or out. Put simply, it is enrolment that produces learning. This method is used to compare effective numeracy and literacy enrolment between 2000 and 2007 for nine African countries (Kenya, Lesotho, Malawi, Namibia, South Africa, Swaziland, Tanzania, Uganda and Zambia) as well as providing sub-national estimates of effective enrolment for the five wealth quintiles within each country. After calculating rates of improvement between 2000 and 2007, the paper also discusses what rates of progress are in fact possible and calculates how long it would take all nine countries to reach 95% effective enrolment under varying assumptions. 2. Background A sequential analysis of the access-to-education literature, and subsequent policy dialogues, shows an important development in the thinking of educational researchers. What started out as an almost singleminded focus on access, Education For All, has slowly developed into a more nuanced concept of quality education for all (UNESCO, 2005; Lewin, 2007). It is now widely accepted that the ability of a country to educate its youth cannot be measured by access to schooling or enrolment rates alone, but rather by its ability to impart to students the knowledge and skills necessary to function as literate and numerate members of the broader society. While access is a necessary condition for this type of education, it is by no means a sufficient one. Notwithstanding the above, the extant literature is almost entirely bifurcated on this issue: reports either focus on the quality of education (for example the PIRLS, TIMSS and PISA reports) or on access to education (for example the Education for All reports), but not both. This is problematic for two reasons: 1) Observing access to education without regard for the quality of that education clouds the analysis, primarily because labour-market prospects and social mobility are driven by cognitive skills acquired and not only years of education attained (Hanushek & Woessmann, 2008), and 2) Analysing the quality of education without taking cognizance of the enrolment and dropout profiles of the countries under review is likely to bias the results. Countries with lower enrolments and higher dropout rates perform better on average, than otherwise similar countries that have higher enrolments and fewer dropouts (UNESCO, 2005, p. 48). This is largely due to the selection effects involved where the strongest (i.e. wealthiest, most advantaged, and most able) students enrol and then remain in the schooling system. This distinction between access and quality is important not only for theoretical reasons, but also for policy targets. For example, the Millennium Development Goals (MDGs) as they stand at the moment are phrased entirely in terms of access and progression, not quality. The second MDG states that By 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling. It is implicitly assumed that children who progress through school learn as they go, something that may not in fact be true. Indeed, passing grades in the absence of quality-assured standardised assessments is a very poor indication of learning. As Pritchett (2004, p. 11) notes, The completion of primary schooling or higher in itself does not guarantee that a child has mastered the needed skills and competencies. In fact, all of the available evidence suggests that in nearly all developing countries the levels of learning achievement are strikingly, abysmally low. Even in the unlikely event that we do manage to meet the MDG concerned with education by 2015 (i.e. all children complete primary school), without a reference to the quality of that primary schooling, this will be 2

3 a hollow achievement at odds with the United Nations Millennium Declaration. Children may very well have completed primary school, but if they remain illiterate and innumerate we cannot say that they have been empowered to pursue a life of their own choosing. In light of the approaching expiration of the MDGs, and the on-going talks surrounding the form of their replacement, many have proposed that learning outcomes be made an explicit criteria of any post-2015 educational goals. The effective enrolment statistic reported in this research note may provide one such measure. 3. Method Given that educational quality, as measured by student test scores, is a continuous variable and educational access is binary (enrolled or not), some transformation is necessary in order to create a single indicator of education system performance. This is possible by making certain assumptions about the numeracy and literacy competency of grade 6 aged children who are not in school. Since we have data on both the educational competencies of the school-going population (from SACMEQ 2 ), and also the age-specific attendance rates of the population (from DHS 3 ), we calculate what proportion of school-aged children (whether in school or out) have acquired basic numeracy and literacy skills. Given that there have been two rounds of SACMEQ where each of the nine countries have participated, SACMEQ II (2000) and SACMEQ III (2007), it is possible to calculate changes in effective enrolment over time. Based on the results of the numeracy and literacy tests, SACMEQ classifies school-going children into one of eight categories for reading, ranging from pre-reading (level 1) to critical reading (level 8), and similarly for mathematics, where the levels range from pre-numeracy (level 1) to abstract problem solving (level 8). Detailed descriptions of the eight competency levels can be found in Hungi, et al. (2010, p. 6). According to this classification system, if children have not reached level three in either reading ( basic reading ) or mathematics ( basic numeracy ) they are deemed functionally illiterate 4 and functionally innumerate respectively. As Ross et al. (2005, p. 262) explain, It is only at Level 3 that pupils can be said to read [otherwise they] could be said to be illiterate. By this definition, if students are functionally illiterate they cannot read a short and simple text and extract meaning; and if students are functionally innumerate they cannot translate graphical information into fractions or interpret common everyday units of measurement. This threshold of competency has been used in the literature before, for example, Shabalala (2005, p. 222) also uses the bottom two SACMEQ levels and deems students below this threshold as non-readers and non-numerate. 2 SACMEQ (Southern and East African Consortium for Monitoring Educational Quality) is a consortium of education ministries, policy-makers and researchers who, in conjunction with UNESCO s International Institute for Educational Planning (IIEP), aims to improve the research capacity and technical skills of educational planners (Moloi & Strauss, 2005: 12) and to provide policy-relevant information on the quality of education in participating countries. 3 Demographic and Health Survey 4 The terms illiterate and innumerate have a number of possible meanings ranging from the inability to write a sentence or complete a one-step arithmetic sum, to more demanding definitions which include reading for meaning or using numerical skills in everyday life. We take the latter approach and use the terms functionally illiterate and illiterate interchangeably in the paper. It is of little use if children can write down and read a memorised paragraph if they do not understand what they are reading or writing. Similarly, if children cannot relate basic arithmetic skills into real world situations, these skills are only of limited value. 3

4 To provide a tangible example, suppose a country has one million 14 year old (grade six aged) children and that only are enrolled in school (enrolment rate of 80% which we can tell from DHS). Further suppose that of the children enrolled in school, only children are functionally literate and are functionally numerate (which we can tell from SACMEQ). The effective literacy enrolment rate in that country would be 60% ( / ) and effective numeracy enrolment would be 50% ( / ). Thus, while the enrolment rate is 80% the effective literacy enrolment would only be 60% and the effective numeracy enrolment would be 50%. This method assumes that all grade 6 aged children that are not enrolled in school are illiterate and innumerate. Whether these children never enrolled in the first place, or enrolled but dropped out prior to grade 6, is an important question, but not relevant for the current purposes. For those children that never enrol, it is highly unlikely that they would learn to read, write and compute at a sufficient level to be able to pass competency level one and two on the SACMEQ tests5. For those children that do enrol but drop out before grade 6, it is also improbable that they would have acquired these skills prior to dropping out. Many students that drop out do so because they failed previous grades or repeated grades multiple times. Those that drop out due to income constraints or remoteness are also statistically less likely to be in the better performing part of the distribution prior to dropout. Lastly, given that many of the students that remain in school do not reach level 3 by grade 6 (our literacy threshold), it is unlikely that those that have dropped out would have already reached level 3 prior to dropping out. See Spaull & Taylor (2012) for further discussion. Throughout this paper, all figures are reported only for grade-six aged children. Thus, any net attendance rates or effective enrolment rates refer to the rates for grade-six aged children, not for the entire primary school aged cohort. This is because SACMEQ tests grade six pupils only and thus all effective enrolment rates can only be calculated for grade six aged pupils. 4. National changes in enrolment and effective enrolment Table 1 below reports the enrolment and effective enrolment rates (for both numeracy and literacy) for each country for 2000 and Looking at only the nominal enrolment 6 rate, which is taken from DHS household surveys 7 or equivalents, one can see that the three countries with the fastest growth in nominal enrolment are Zambia (16 percentage point increase in enrolment), Tanzania (11 percentage point increase in enrolment), and Lesotho (10 percentage point increase in enrolment). Looking specifically at effective literacy enrolment i.e. where children are enrolled and have acquired basic literacy skills it is surprising to note that the country which improved the most over the 2000 to 2007 period was Namibia who managed to improve their effective literacy enrolment rate from 51% in 2000 to 5 While there will obviously be exceptions to the rule where educated parents may teach their children informally at home, this is so small as to be negligible on a national scale. 6 The terms enrolment rate and net attendance rate are used interchangeably in this paper. This is because the statistics used for enrolment are Filmer s (2010) net attendance rates. 7 The DHS household survey (or equivalent) that was closest to 2000 (SACMEQ II) and 2007 (SACMEQ III) were selected to represent the state of enrolment at the time of the SACMEQ surveys. In each instance, the DHS household survey (or equivalent) is not more than two years before or after 2000 or For a full list of DHS (or equivalent) household surveys see Filmer (2010). 4

5 80% in 2007, an impressive 29 percentage point increase (see Figure 1). The reason why this is surprising is that Namibia had one of the smallest increases in nominal enrolment (from 90% to 93%) over the period. Thus, the vast majority of the improvement over this period must have been from improvements in the quality of education in Namibia rather than simply expanded access. Although there weren t many more children in Namibian schools in 2007 than in 2000, far more of those that were in school in 2007 were acquiring basic literacy skills than those in school in For further discussion on the case of Namibia between 2000 and 2007 see Van der Berg & Moses (2011). The other countries which recorded large improvements in effective literacy enrolment were Tanzania (14 percentage point improvement), Lesotho (14 percentage point improvement) and Zambia (12 percentage point improvement). The first two of these countries expanded access to primary schooling by roughly 10 percentage points over this period, with Zambia improving by 16 percentage points. Given that effective literacy enrolment combines both enrolment and the acquisition of basic skills, these countries not only expanded access to schooling, but to schooling where children acquire basic literacy skills. Figure 1: Improvement in enrolment and effective literacy enrolment between 2000 and 2007 (percentage point change; using SACMEQ and DHS) A similar picture emerges if one observes the changes in effective numeracy enrolment (Figure 2 below). Again Namibia stands out as the country with the largest improvement with Lesotho and Tanzania following in second and third positions. One notable exception is that Zambia is now third to last on the list of improvers. Although Zambia increased enrolment by 16 percentage points, effective numeracy enrolment increased by only eight percentage points. Thus, many more Zambian 14 year olds had access to 5

6 schooling, and many of these children acquired basic literacy skills (Figure 1), but the improvement in numeracy was more modest (Figure 2). This is in stark contrast to countries such as Lesotho and Tanzania where expansions in enrolment were matched with even greater expansions in effective enrolment for both numeracy and literacy. Uganda is the only country where effective numeracy enrolment did not increase between 2000 and 2007, that is to say that the same number of grade-six aged children in Uganda acquired basic numeracy skills in 2007 than did so in 2000, despite a marginal increase in the number of children enrolled in school. Figure 2 Improvement in enrolment and effective numeracy enrolment between 2000 and 2007 (percentage point change; using SACMEQ and DHS) 5. Scenario projections for effective enrolment Figures 1 and 2 above show that there has been varied progress across the nine countries. While some countries managed to improve their effective literacy and numeracy enrolment rates substantially, others did not improve. Figure 3 and Table 1 below both show that some countries started with very low effective literacy enrolments in 2000 (Malawi, Zambia, Lesotho and Namibia), while others started with relatively high effective literacy enrolment (Swaziland, Kenya). By 2007 Namibia, Zambia, Tanzania and Lesotho had all registered substantial improvements in effective literacy enrolment and, in addition, Namibia, Tanzania and Lesotho also improved their effective numeracy enrolment rates significantly. Although Malawi and Zambia had effective literacy enrolment rates of less than 55% in 2007, other countries such as Swaziland, 6

7 Kenya, Tanzania and Lesotho now all had effective literacy enrolment rates in excess of 80%. South Africa and Uganda made only modest progress in effective literacy enrolment over the period. As we approach the expiration date (2015) for the Millennium Development Goals, a number of international organizations have devoted time and energy into researching what the post-mdg targets should look like. There is now widespread consensus that there should be a quality element to any post- MDG education goal, rather than only an access requirement. While creating such a statistic is certainly possible 8 (see Spaull & Taylor, 2012), it is less clear when using a combined statistic what rates of progress are actually possible. This can lead to aspirational planning where targets are far removed from what is actually possible. As Lewin (2007, p. 7), commenting on the process of setting goals for developing countries, explains: Generally targets and indicators are not contextualised or related to different starting points, realistic assessments of capacity, and recent rates of progress. This can result in increasingly unrealistic goals. Notwithstanding the above, goals will (and should) always be set, and thus it is important to find some empirical evidence to substantiate what in fact is a plausible or possible rate of progress. In order to provide some empirical evidence on this issue, we calculate the change in effective literacy and numeracy enrolment for each country over the period. These figures are reported in Table 1 below. Using these figures it is possible to calculate how long it would take each country to reach 95% effective literacy enrolment and how long it will take to reach 95% effective numeracy enrolment. The reason why we choose 95% rather than 100% is that there are always likely to be a small number of children that do not acquire basic numeracy and literacy skills and thus 100% effective enrolment may be an unrealistic target. Two possible solutions present themselves when estimating the projected timelines for each country to reach 95% effective enrolment: 1. We can assume that each country will achieve the same growth in effective numeracy and literacy enrolment after 2007 as it achieved between 2000 and 2007, i.e. it will continue along the same improvement trajectory as the one it achieved between 2000 and One can then extrapolate into the future and calculate how long it will take, given current rates of progress, to achieve 95% effective numeracy and literacy enrolment rates. These figures are reported in Table 1 below. Depending on the 2007 baseline effective enrolment rate, and the rate of progress that country achieved, one can expect vastly differing projected timelines to reach 95% effective enrolment. If Namibia continues to achieve an impressive 4.2 percentage point annual improvement in effective literacy enrolment, they will reach the 95% target in three years. By contrast, South Africa s modest annual improvement of 0.73 percentage point annual increase in effective literacy enrolment means that they will only reach the 95% target in 32 years. 2. Another possible solution is to use the improvement rates of the fastest three improving countries as a guideline for what is potentially achievable for all countries. If one looks at only the fastest three improving countries (Namibia, Tanzania and Lesotho), the average annual improvement in effective literacy enrolment was 2.8 percentage points per year and for effective numeracy enrolment it was 3.3 percentage points per year. Looking at these three countries, one can see that they started from very different positions but all managed to achieve large increases in effective 8 The only caveat to this is that countries need to participate in a cross-national survey of educational achievement and have comparable household survey for roughly the same period. The worst-off countries in Africa may not have either and thus calculating effective enrolment for these countries is problematic. The importance of these statistics warrants international organizations to encourage non-participating countries to take part in surveys like SACMEQ. 7

8 enrolment rates. Namibia started off with relatively high enrolment in 2000 (90%), but relatively low effective literacy enrolment (51%) and very low effective numeracy enrolment (22%). By contrast, Tanzania started with low enrolment in 2000 (74%) but the third highest effective literacy enrolment rate (68%) and the third highest effective numeracy enrolment (59%). Lesotho had a combination of low enrolment and low effective enrolment rates in Yet all three countries managed to achieve large improvements in both effective literacy enrolment and effective numeracy enrolment by 2007 despite starting from very different positions and with very different enrolment profiles. This seems to suggest that countries can improve their effective enrolment rates in a number of different ways, whether by expanding access to schooling of sufficient quality as to impart basic skills (Tanzania and Lesotho), or by improving the quality of schooling in the country (Namibia). Figure 3 below shows the projected timeline for each country to reach 95% effective literacy enrolment assuming all countries improve as quickly post-2007 as the three fastest improving countries did between 2000 and Figure 3: Projected timeline for each country to reach 95% effective literacy enrolment assuming each country improves as quickly post-2007 as the fastest three improving countries did between 2000 and

9 Enrolment rate in 2000 (DHS) Enrolment rate in 2007 (DHS) Effective literacy enrolment rate 2000 Effective literacy enrolment rate 2007 Change in effective literacy enrolment Annualised change in effective literacy enrolment between 2000 and 2007 Number of years it would take each country to reach 95% effective literacy enrolment (assuming the same growth rate in effective enrolment between 2000 and 2007) Number of years it would take each country to reach 95% literacy effective enrolment (assuming the growth rate of fastest three improving countries between 2000 and 2007) Effective numeracy enrolment rate 2000 Effective numeracy enrolment rate 2007 Change in effective numeracy enrolment Annualised change in effective numeracy enrolment between 2000 and 2007 Number of years it would take each country to reach 95% effective numeracy enrolment (assuming the same growth rate in effective enrolment between 2000 and 2007) Number of years it would take each country to reach 95% effective numeracy enrolment (assuming the growth rate of fastest three improving countries between 2000 and 2007) KEN LES MAL NAM SOU SWA TAN UGA ZAM Table 1: Enrolment, effective enrolment and projections for literacy and numeracy (SACMEQ grade 6 and DHS) 9

10 Country Year of DHS Net Attendance Rates '2000' Effective literacy enrolment '2000' Effective numeracy enrolment '2000' Wealth quintiles '2000' Wealth quintiles '2000' Wealth quintiles '2000' Q1 Q2 Q3 Q4 Q5 Q1 Q2 Q3 Q4 Q5 Q1 Q2 Q3 Q4 Q5 Kenya MICS % 89% 88% 91% 86% 83% 85% 83% 86% 84% 81% 78% 77% 81% 81% Lesotho MICS % 73% 75% 86% 89% 48% 52% 56% 59% 63% 24% 25% 28% 27% 31% Malawi DHS % 76% 79% 79% 82% 39% 39% 39% 45% 56% 18% 19% 18% 21% 27% Namibia DHS % 89% 91% 94% 94% 39% 43% 45% 54% 77% 11% 13% 12% 22% 53% South Africa OHS % 96% 97% 98% 96% 51% 55% 62% 77% 89% 33% 34% 39% 51% 77% Swaziland MICS % 87% 78% 86% 88% 75% 85% 76% 84% 86% 56% 67% 62% 65% 73% Tanzania IHS-WDR % 66% 69% 88% 77% 65% 59% 63% 82% 75% 51% 47% 49% 67% 64% Uganda DHS % 88% 87% 91% 87% 60% 62% 67% 68% 68% 48% 50% 53% 59% 59% Zambia MICS % 63% 57% 84% 83% 22% 29% 24% 43% 58% 12% 14% 12% 25% 36% Country Year of DHS Net Attendance Rates '2007' Effective literacy enrolment '2007' Effective numeracy enrolment '2007' Wealth quintile '2007' Wealth quintiles '2007' Wealth quintiles '2007' Q1 Q2 Q3 Q4 Q5 Q1 Q2 Q3 Q4 Q5 Q1 Q2 Q3 Q4 Q5 Kenya DHS 2008/9 91% 95% 96% 97% 95% 80% 86% 88% 92% 91% 77% 82% 85% 90% 89% Lesotho DHS % 85% 92% 94% 94% 63% 64% 70% 74% 80% 41% 47% 50% 58% 64% Malawi HIS-WDR % 80% 90% 90% 93% 44% 50% 57% 56% 64% 30% 32% 35% 35% 41% Namibia DHS 2006/7 87% 93% 93% 96% 98% 70% 77% 78% 86% 91% 33% 40% 45% 56% 71% South Africa GHS % 97% 98% 99% 100% 56% 63% 70% 77% 91% 42% 48% 55% 64% 85% Swaziland DHS % 86% 87% 93% 94% 88% 84% 86% 91% 93% 80% 77% 79% 85% 89% Tanzania DHS 2007/8 80% 83% 89% 86% 89% 75% 80% 86% 84% 87% 65% 71% 79% 75% 81% Uganda DHS % 91% 90% 96% 89% 58% 68% 70% 79% 78% 41% 53% 52% 63% 64% Zambia DHS % 84% 84% 94% 96% 37% 45% 44% 56% 63% 22% 21% 24% 35% 42% Table 2: Net attendance rates, effective literacy enrolment and effective numeracy enrolment by wealth quintiles for 2000 and

11 6. Trends and projections for the rich and the poor In addition to calculating national trends and projections for effective enrolment, it is also possible to calculate these for different groups within a country. The available evidence suggests that certain groups of children benefit disproportionately when there are improvements in access or quality. For example, expanding access to education is likely to benefit the poor and those living in rural areas since these are the children who were previously excluded. Table 2 above reports the net attendance rates and effective literacy and numeracy enrolment rates for each wealth quintile within each country. Appendix A shows this same information graphically. From these sources one can see that in some countries, for example Malawi, the net attendance rate of the wealthiest 60% of 14 year old children increased by far more than that of the poorest 40% of 14 year old children. In some countries the increase in enrolment is mostly constant across the five wealth quintiles, as in Lesotho, while in others there was little change between 2000 and 2007, as in South Africa, Uganda and Namibia. In Zambia the enrolment rate of the poorest 20% of fourteen year olds increased by 29 percentage points with the figure for the wealthiest 20% of fourteen year olds increasing by 13 percentage points. Increases in enrolment may or may not be accompanied by an increase in effective enrolment, depending on how countries choose to expand access and whether or not they can provide education of a sufficient quality to impart basic numeracy and literacy skills. The effective numeracy and literacy enrolment rates for the five wealth quintiles of each country for 2000 and 2007 are also reported in Table 2 above. Appendix B shows this same information graphically for each country separately and includes the 95% confidence interval using only the SACMEQ standard errors 9. Figure 4 and Figure 5 below summarise this information in a single graph for literacy and a single graph for numeracy. Observations in the North-West quadrant of the graph (notably the rich and poor in Zambia), had large increases in access (net attendance rates) and smaller increases in effective literacy enrolment. Countries in the South-East quadrant of the graph (notably the poor in Namibia) had large increases in effective literacy enrolment but only marginal increases in access. Figure 5 shows that there were larger improvements in effective numeracy enrolment than in effective literacy enrolment for almost all countries. It is perhaps helpful to contrast two countries; Namibia and Uganda. From Figures 4 10 and 5 one can see that the poor in Namibia benefited far more than the wealthy over the 2000 to 2007 period, particularly so for effective literacy enrolment. By contrast, the rich in Uganda benefited more than the poor for both effective literacy enrolment and effective numeracy enrolment. It is important to remember that the y-axis in these graphs represents access, while the x-axis represents both access and quality, i.e. effective enrolment. Thus, any observation with a positive increase in effective literacy or numeracy enrolment means that there are more children (as a percentage) acquiring basic numeracy and literacy skills in 2007 than there were in This is the case for all the countries except 9 Given that the effective enrolment rate is a composite statistic created by combining a statistic from DHS and a statistic from SACMEQ, each of which has a confidence interval around it, the confidence interval around the composite statistic should be a combination of the standard errors from each original statistic. Creating and reporting such a composite standard-error is beyond the scope of this research note. However, to provide some indication of the variation of the statistic, the standard error from the SACMEQ estimate of those who are functionally literate and numerate has been used. This is likely to be an underestimate of the true standard error of the composite statistic but serves to show that these statistics do have uncertainty around them, given that they are calculated from samples, not full populations. 10 Incidentally, the graphs also show that there were larger improvements in access to education for the rich in Kenya than the poor in Kenya. These results are somewhat peculiar and counterintuitive, since, according to the DHS data, there were larger increases in enrolment for the richest 20% of Kenyan grade six aged students than for the poorest 40% of students 11

12 the poor in Kenya for effective literacy enrolment and the poor in Uganda for effective numeracy enrolment. Table 3 below summarises Figures 4 and 5 and shows whether the rich or poor benefitted more than the other for effective numeracy and literacy enrolment, as well as for nominal enrolment (net attendance rates). Table 4 below report the average annualised percentage point changes in effective numeracy and literacy enrolment for the rich and the poor for each country for 2000 and Table 5 uses this data and calculates the number of years it would take the rich to reach 95% effective literacy and numeracy enrolment under two scenarios: 1) the rich in each country continue along the trajectory of improvement that they managed to achieve over the 2000 to 2007 period, and 2) the rich in each country improve each year by the same amount as the average of the rich in the three fastest improving countries between 2000 and The table also calculates and reports these statistics for the poor. The three countries used in the calculation for the fastest possible improvement rate for the rich and the poor need not be the same. For example the three countries used to calculate the fastest possible improvement rate for effective literacy enrolment for the poor were Namibia, Tanzania and Zambia, but for the rich they were Lesotho, Namibia and Tanzania. ZAM ZAM KEN MAL TAN TAN LES KEN SOU SWA SWA UGA SOU MAL NAM LES NAM UGA Improvement in effective literacy enrolment ( ) Poorest 40% Richest 20% Figure 4: Improvement in enrolment (net attendance rate) and effective literacy enrolment for poorest 40% of children and richest 20% of children between 2000 and 2007 (all improvements are percentage point increases between 2000 and 2007) 12

13 ZAM ZAM KEN MALTAN TAN LES SWA SWA SOU NAM LES KEN UGA MAL SOU NAM UGA Improvement in effective numeracy enrolment ( ) Poorest 40% Richest 20% Figure 5: Improvement in enrolment (net attendance rate) and effective numeracy enrolment for the poorest 40% of children and the richest 20% of children between 2000 and 2007 (all improvements are percentage point increases between 2000 and 2007) Table 3: Summary table of changes in access and effective enrolment (numeracy and literacy) between 2000 and 2007 ACCESS (Changes in net attendance rate ) ACCESS & BASIC NUMERACY LEARNING (Changes in effective numeracy enrolment ) ACCESS & BASIC LITERACY LEARNING (Changes in effective literacy enrolment ) No large differences between richest 20% of students and poorest 40% Namibia, South Africa, Swaziland, Tanzania, Uganda. Malawi, South Africa, Swaziland, Tanzania, Zambia Lesotho, Malawi, Swaziland, Tanzania Rich benefitted more than the poor by at least five percentage points Poor benefitted more than the rich by at least five percentage points Kenya, Malawi Kenya, Lesotho Kenya, Uganda Lesotho, Zambia Namibia Namibia, Zambia 13

14 Table 4: Percentage point change in net attendance rates and effective literacy and numeracy enrolment rates for the richest 20% of grade six aged children and the poorest 40% of grade six aged children between 2000 and 2007 Country Percentage point change in Net Attendance Rate (2000 to Poorest 40% (Q1 & Q2) 2007) Richest 20% (Q5) Percentage point change in effective literacy enrolment Poorest 40% (Q1 & Q2) (2000 to 2007) Richest 20% (Q5) Annualised percentage point change in effective literacy effective literacy enrolment for poorest 40% Annualised percentage point change in enrolment for richest 20% Percentage point change in effective numeracy enrolment Poorest 40% (Q1 & Q2) (2000 to 2007) Richest 20% Annualised percentage point change in effective numeracy enrolment for poorest 40% Annualised percentage point change in Kenya 2% 10% -1% 7% -0,15 1,05 0% 7% -0,05 1,07 Lesotho 11% 4% 14% 17% 1,93 2,36 20% 33% 2,79 4,76 Malawi 1% 12% 8% 8% 1,15 1,10 12% 14% 1,72 2,04 Namibia 2% 4% 33% 14% 4,69 2,05 24% 19% 3,49 2,67 South Africa 1% 4% 7% 3% 0,97 0,42 12% 8% 1,65 1,19 Swaziland 6% 6% 6% 7% 0,86 0,96 17% 16% 2,44 2,31 Tanzania 12% 12% 16% 12% 2,22 1,76 19% 16% 2,71 2,35 Uganda -2% 2% 2% 10% 0,25 1,42-3% 5% -0,36 0,73 (Q5) effective numeracy enrolment for Zambia 25% 14% 15% 5% 2,20 0,66 9% 5% 1,25 0,77 richest 20% 14

15 Table 5: Projected timelines for each country to reach 95% effective numeracy and literacy enrolment under varying assumptions Literacy projections Numeracy projections Country Number of years it would take each country to reach 95% effective literacy enrolment (assuming the same growth rate in EE between 2000 and 2007) Number of years it would take each country to reach 95% effective literacy enrolment (assuming the growth rates of fastest three improving countries between 2000 and 2007) Number of years it would take each country to reach 95% effective numeracy enrolment (assuming the same growth rate in EE between 2000 and 2007) Number of years it would take each country to reach 95% effective numeracy enrolment (assuming the growth rates of fastest three improving countries between 2000 and 2007) Poorest 40% (Q1 & Q2) Richest 20% (Q5) Poorest 40% (Q1 & Q2) Richest 20% (Q5) Poorest 40% (Q1 & Q2) Richest 20% (Q5) Poorest 40% (Q1 & Q2) Richest 20% (Q5) Kenya (NA) (NA) Lesotho Malawi Namibia South Africa Swaziland Tanzania Uganda (NA) Zambia Negative values mean that there was a decline in effective enrolment between 2000 and If these countries continue on this trend, they will never reach 95% effective enrolment. 15

16 From Table 5 one can see that the rich are in a far better position to reach 95% effective literacy and numeracy enrolment in the near future. The rich in six of the nine countries will reach 95% effective numeracy and literacy enrolment by 2017 if they continue to improve along the trajectories they achieved over the 2000 to 2007 period. For the poor, only three countries (Namibia, Swaziland and Tanzania) will achieve 95% effective literacy enrolment within a decade if they continue along their respective trajectories. Looking at effective numeracy enrolment for the poor, there are only two countries who will reach the 95% target by 2017 if they continue along their trajectories; Swaziland and Tanzania. In the case of Zambia for effective literacy enrolment, Table 5 predicts that the poor will reach 95% effective literacy enrolment 24 years before the rich which is counter-intuitive and implausible. The reason for this is that extrapolating the trend for the poor in Zambia assumes they will be able to continue to increase as quickly post-2007 as they did over the 2000 to 2007 period. This is unlikely. Given that the poor may recently have caught up with the rich in a country it could well be argued that they would not be able to sustain such high levels of progress indefinitely. It is for this reason that the second method of using the average of the fastest three improving countries is preferable in the case of Zambia. If the poor in all countries managed to improve at the same annual rate as the average increase of the poor in Namibia, Tanzania and Zambia for effective literacy enrolment (three percentage points per year) and Namibia, Tanzania and Lesotho for effective numeracy enrolment (three percentage points per year), seven of the nine countries would reach the effective literacy enrolment target (95%) within 12 years of 2007 with a slightly longer time horizon for Zambia (18 years) and Malawi (16 years). For effective numeracy enrolment the timelines for the poor are considerably longer with all nine countries reaching 95% effective numeracy enrolment within two and a half decades (i.e. by 2032), and this is assuming the growth rate of the fastest three improving countries between 2000 and While some countries will reach the 95% targets within five to ten years of 2007 (notably Kenya, Swaziland and Tanzania), others will take a lot longer (Zambia, Malawi, Uganda and the poor in South Africa). 7. Conclusion This research note has aimed to provide some empirical evidence of the actual growth rates in effective numeracy and literacy enrolment for grade six aged children between 2000 and 2007 for countries as a whole, and for the richest 20% and poorest 40% of students separately. The seven major findings and recommendations are summarised below: 1. Large improvements are possible. Some countries have managed to improve effective numeracy and literacy enrolment substantially over the 2000 to 2007 period, particularly Namibia, Lesotho and Tanzania. Given that many of these countries also experienced large increases in enrolment over the period, this challenges the notion that there is a strong access-quality tradeoff, at least as far as the acquisition of basic numeracy and literacy skills. The situation of the average grade-six aged child improved in every country for literacy and numeracy between 2000 and There is still scope to increase access for the poor. Despite large increases in enrolment and effective enrolment, the poor continue to lag behind the rich in almost all countries. While the rich enjoy near universal access to education, many of the poor are not enrolled and thus their opportunity to acquire foundational numeracy and literacy skills are severely limited. Further expansions in enrolment are needed if the poor are to benefit from schooling. This can be regarded as the low-hanging fruit of effective enrolment since improving access to schooling where this is not already near universal is a key step to improving effective enrolment. Furthermore, this strategy should not be only reluctantly pursued for fear that the average country test scores may suffer somewhat, since a change in the distribution of achievement can 16

17 lower the average test score while at the same time, because of increased access there are more children achieving at each level than before the access expansion. 3. Different countries improve using different strategies. Looking at the starting points and the trajectories of each of the fastest improving countries, one can conclude that there is no one method to increase effective enrolment. Some countries increased access to education substantially over the period (Lesotho, Tanzania), while others had stable enrolments but nevertheless managed to drastically increase effective enrolment (Namibia). 4. Some improvements were more pro-poor, others more pro-rich. The improvements in effective numeracy and literacy enrolments were pro-poor in some countries (Namibia and Zambia for literacy), while they were pro-rich in others (Kenya and to some extent Uganda and Lesotho). 5. Zambia is a special case. Zambia was the only country where enrolments expanded rapidly and effective enrolment did not keep pace. Although enrolment expanded by 25% for the poor over the 2000 to 2007 period, effective literacy enrolment for this group increased by only 15 percentage points and only 9 percentage points for effective numeracy enrolment. This being said, the fact that there was a positive increase in effective enrolment over the period means that the average grade-six aged child in Zambia was better off in 2007 than in 2000 as far as acquiring basic numeracy and literacy skills. 6. For most countries, improving the quality of schooling for those enrolled is likely to be the primary challenge in the next phase of increasing effective enrolment. This may not be as easy as increasing access to schooling. 7. Projections are dependent on assumptions. Under conservative assumptions about improvement, only five of the nine countries will reach 95% effective numeracy and literacy enrolment within 20 years of Under more ambitious assumptions about improvement i.e. assuming other countries can improve as fast as Namibia, Tanzania and Lesotho all nine countries will be able to reach 95% effective enrolment within 20 years of The above discussion on effective enrolment is based on data from SACMEQ II (2000) and SACMEQ III (2007), as well as DHS data that were closest to 2000 and The next round of SACMEQ (SACMEQ IV) is scheduled to be conducted in This will provide a third data point in the series and allow one to assess whether the projections put forward in this research note were too ambitious or too conservative, at least for the six years following On the release of the SACMEQ IV data (perhaps 2015) one can recalculate these trends in effective numeracy and literacy enrolment and recalibrate the projections using the new information. Whenever one uses past trends and assumptions to inform future projections it is necessary to explain that these projections are in no way certain, and in some instances are unlikely to occur. They are based on survey data, both household data (DHS) and school-system data (SACMEQ), both of which have some level of uncertainty around them by virtue of them being samples. Furthermore, extrapolating from one set of countries to another is problematic since it ignores the complexities of context. What worked in Namibia may not be possible in Malawi, for example. Notwithstanding the above, there is still value in calculating past trends in effective enrolment and using these empirical findings to inform ideal trajectories of improvement and subsequently policy goals. In the absence of any empirical information to inform the post Millennium Development Goals, these goals are likely to be set entirely on ambitious political whims, or incorrect data. Using trends on nominal enrolment or completion to inform goals which combine access and learning (like effective enrolment) are likely to yield overly optimistic goals which will be almost impossible to reach. 17

18 In summary, despite drastic increases in enrolment for many countries, there were still improvements in effective numeracy and literacy enrolments for almost all countries. That is to say that the situation of the average grade six aged child in all nine countries improved between 2000 and 2007 as far as the acquisition of basic numeracy and literacy skills, with the exception of Uganda for numeracy where children were no better. This evidence suggests that the belief that there exists a strong access-quality trade-off has far less empirical support than was previously believed to be the case. Finally, with respect to goal-setting for the post-mdg period, it is important to remember that goals should be contextualised and related to different starting points, recent rates of progress, and should use appropriate composite statistics as measures of progress. Doing so will be a substantial improvement on the theoretically naïve and empirically uninformed goals of an earlier era. 18

19 Bibliography Filmer, D. (2010). Educational Attainment and Enrollment around the World. The World Bank. econ.worldbank.org/projects/edattain: Development Research Group. Hanushek, E., & Woessmann, L. (2008). The Role of Cognitive Skills in Economic Development. Journal of Economic Literature, 46 (3), Hungi, N., Makuwa, D., Ross, K., Saito, M., Dolata, S., van Capelle, F., et al. (2010). SACMEQ III Project Results: Pupil Achievement Levels in Reading and Mathematics. Paris: Southern and Eastern Africa Consortium for Monitoring Educational Quality. Lewin, K. (2007). Improving Access, Equity and Transitions in Education: Creating a Research Agenda. Co. Sussex: Consortium for research on Educational Access, Transitions and Equity (CREATE). Pritchett, L. (2004). Towards a New Consensus for Addressing the Global Challenge of the Lack of Education. Copenhagen: Copenhagen Consensus. Ross, K., Saito, M., Dolata, S., Ikeda, M., Zuze, L., Murimba, S., et al. (2005). The Conduct of the SACMEQ III Project. In E. Onsomu, J. Nzomo, & C. Obiero, The SACMEQ II Project in Kenya: A Study of the Conditions of Schooling and the Quality of Education. Harare: SACMEQ. Shabalala, J. (2005). The SACMEQ II Project in Swaziland: A Study of the Conditions of Schooling and the Quality of Education. Retrieved 2009, from Southern and Eastern Africa Consortium for Monitoring Educational Quality: Spaull, N., & Taylor, S. (2012). Effective enrolment - creating a composite measure of educational access and educational quality to accurately describe education system performance in sub-saharan Africa. Stellenbosch Economic Working Papers WP21/2012 [Online] Available: Taylor, S., & Spaull, N. (2013). The effects of rapidly expanding primary school access on effective learning: The case of Southern and Eastern Africa since Stellenbosch Economic Working Paper Series WP01/2013 [Online] Available: UNESCO. (2005). Education For All Global Monitoring Report Paris: UNESCO Publishing. Van der Berg, S., & Moses, E. (2011). The remarkable improvement in functional literacy and numeracy in Namibia: A comparison between SACMEQ II and SACMEQ III. SACMEQ Policy Brief. Available: %20Final%20Oct2011.pdf [30 June 2012] 19

20 Appendix A *All figures in this appendix are derived from Filmer (2010). 20

21 21

22 Appendix B Given that the effective enrolment rate is a composite statistic created by combining a statistic from DHS and a statistic from SACMEQ, each of which has a confidence interval around it, the confidence interval around the composite statistic should be a combination of the standard errors from each original statistic. Creating and reporting such a composite standard-error is beyond the scope of this research note. However, to provide some indication of the variation of the statistic, the standard error from the SACMEQ estimate of those who are functionally literate and numerate has been used. This is likely to be an underestimate of the true standard error of the composite statistic but serves to show that these statistics do have uncertainty around them, given that they are calculated from samples, not full populations. 22

23 23

24 24

25 25

26 26

27 27