Incentivizing schooling for learning: Evidence on the impact of alternative targeting approaches 1

Incentivizing schooling for learning: Evidence on the impact of alternative targeting approaches 1 Felipe Barrera-Osorio Harvard Graduate School of Education and Deon Filmer World Bank This version: October, 2012 Preliminary version. Please do not circulate. Abstract. Demand-side incentive programs such as scholarships or Conditional Cash Transfer programs have been shown to increase measures of school participation in a number of countries, although impacts on learning outcomes have been harder to identify. We evaluate the impact of a large scale primary school scholarship pilot program in Cambodia and show that the program increases school participation (enrollment and attendance) despite being targeted to some of the poorest and most remote areas. The program was designed to evaluate the equity and effectiveness implications of two alternative targeting approaches: in some randomly selected schools recipients were targeted on the basis of poverty, in others recipients were targeted on the basis of merit. While we show positive impacts on enrollment and school progression emerging from both targeting approaches, learning impacts are only detectable among meritbased recipients. We present evidence on student effort and household education investment compatible with the asymmetry in learning impacts. While there are some equity implications of a merit-based approach to targeting (the poverty-based approach unsurprisingly identifies a poorer group of recipients), the tradeoff is not particularly stark. Scaling up an approach that targets students with high academic potential while ensuring that the poorest student are among that set is likely to be the approach that maximizes both equity and effectiveness objectives. JEL classification codes: I21; I24; I28; O10 Keywords: education; Cambodia; randomization; scholarships; merit-based targeting; poverty-based targeting. 1 We thank Luis Benveniste, Norbert Schady, Beng Simeth, and Tsuyoshi Fukoaka and the members of Primary School Scholarship Team of the Royal Government of Cambodia s Ministry of Education for valuable input and assistance in carrying out this work. Adela Soliz provided able research assistance. The paper has also benefitted from comments by Muna Meky, Halsey Rogers, and Shwetlena Sabarwal. The authors are, of course, responsible for any errors. This work benefited from funding from the World Bank as well as through the EPDF Trust Fund (TF095245). The findings, interpretations, and conclusions expressed in this paper are those of the authors and do not necessarily represent the views of the World Bank, its Executive Directors, or the governments they represent. 1

1. Introduction There is a steadily growing international evidence-base on the impact of targeted cash transfers on a range of outcomes. Conditional Cash Transfer (CCT) programs have gained popularity in much of the developing world and in several countries are now the largest form of social assistance (see the review in Fiszbein and Schady 2009). These programs, in which cash is transferred to families on the condition that they comply with a set of conditions (typically that children enroll in and regularly attend school, pregnant women make regular prenatal visits, young children are taken for regular health monitoring and checkups visits), have been rigorously evaluated in many countries. Scholarship programs, such as the one we evaluate here, can be thought of as single-child (i.e. individual recipient), single-sector (i.e. education) CCTs. Much of the rigorous evidence on the impact of CCTs have been middle-income and in the Latin America region; the Cambodian evidence joins a much smaller set of evidence focused on low-income countries, such as Bangladesh and Malawi. 2 Moreover, the setting is one in which the primary education survival rate is low due to high a dropout rate. Indeed, much of the evidence of the impact of CCTs at the primary level is from countries where baseline enrollments are high, and impacts subsequently small (in part because there is little room for increase). Establishing the extent to which cash transfers are effective at this school level in this type of setting is an important contribution of this evaluation. Despite this large body of evidence of the impact of CCTs on enrollment and attendance, there has been limited evidence to date on the impact of these programs on test scores, and the few available studies show mixed results: in the case of Cambodia (Filmer and Schady 2009) and Mexico (Behrman, Parker and Todd 2005; Behrman, Sengupta, and Todd 2000), no results on achievement tests; in the case of Kenya (Kremer, Miguel and Thornton 2009) and Malawi (Baird, McIntosh and Özler, 2011), positive effects. The question of how to turn incentives for schooling into learning remains an open one. An important feature of the program we evaluate here is that students were selected on the basis of two alternative approaches. In some schools recipients were selected purely on the basis of poverty (actually, a proxy thereof), but in other schools recipients were applicants who scored well on an assessment test i.e. a merit-based targeting approach. Kremer, Miguel and Thornton (2009) describe a merit-based approach to targeting scholarships in a secondary school program in Kenya and show substantial impacts on both attendance and learning. Targeting students with high academic potential, through the merit-based selection approach, might be a good approach if it maximizes the probability of 2 See Chaudhury and Parajuli (2008) and Baird, McIntosh and Ozler (2009). 2

having an impact on learning as well as enrollment incentivizing schooling for learning. But, targeting high performing students may come at the cost of reaching the poorest, since there is likely to be a positive relationship between academic success and household economic status. Moreover, some authors will argue strongly against merit scholarships based on equity considerations (see Orfield, 2002). The question of whether the efficiency gain (defined narrowly as getting more learning and enrollment per dollar transferred ) comes at too great a cost in terms of not reaching the poor is one that this evaluation is able to investigate. This evaluation aims at addressing two key questions associated with this pilot program: first, what is the measurable impact of these primary school scholarships on measures of school participation and learning? And second, given a choice between targeting recipients based on poverty versus merit, what are the potential tradeoffs in terms of impacts versus reaching the poorest? The evaluation shows two main results. First, both targeting approaches cause higher enrollment and attendance rates, but only the merit-based scholarship shows positive impact on learning, measured as result in test scores. Second, while there are some equity implications of a merit-based approach to targeting (the poverty-based approach unsurprisingly identifies a poorer group of recipients), the tradeoff is not particularly stark. Before addressing the evaluation it is important to emphasize that we focus on a narrow set of objectives, schooling and learning as measured by test scores. The program aimed to transfer cash to poor households, which in itself is potentially welfare enhancing and we do not address that directly. Moreover, learning is but one objective of schooling albeit an important one. There are additional social as well as personal (for example, better health, delayed marriage) impacts of more schooling that we are not addressing or evaluating here. 3 In the next section we describe the setting, program and the evaluation. In section 3 we present the empirical strategy and results. Finally, in section 4 we discuss the results and we present main conclusions. 2. Country setting, program design, evaluation design, and data Country setting 3 As discussed below, we focus below on the fact that the merit scholarship showed impacts on learning, while the poverty scholarships did not. It is possible that poverty scholarships nevertheless have salutary impacts on other outcomes through their impact on schooling. 3

Cambodia has a tradition of demand-side incentives intended to raise school enrollment and attendance rates. While some of these operate at the primary level such as school feeding programs, or small scale programs that incentivize attendance for primary school children the bulk of the programs are targeted at the lower secondary school level. The largest of these programs have been the so call PB Scholarships. 4 The programs do not operate as simple fee-waivers ; rather, the families of children selected for a scholarship receive a small cash transfer, conditional on school enrollment, regular attendance, and satisfactory grade progress. Two rigorous evaluations of the impact of these programs have shown substantial increase in school enrollment and attendance as a direct consequence of the programs. 5 Recipients are on the order of 20 to 30 percentage points more likely to be enrolled and attending school as a result of the scholarships. The evaluation of scholarships offered through the CESSP also showed that the scholarships targeted to lower secondary school students led to more expenditures on education, and to less work for pay among recipients. There were no negative spillover effects either to non-recipients in schools or to ineligible siblings in households. Impacts on learning outcomes were limited, pointing to issues of quality and the match between students skill levels and the instruction they are receiving. One important finding from previous programs, however, was that their targeting was only mildly pro-poor. For example for CESSP scholarships, despite the fact that the program was able to reach the poorest children who applied for the scholarships, the poorest of the poor have already dropped out of school before grade 6 the point at which they would apply for secondary school scholarships. Figure 1 shows the proportion of children 15 to 19 who have completed each grade, based on nation-wide data: clearly children from the poorest quintiles are much less likely to make it to 6 th grade. This suggests that a program that targets children at the end of grade 6 is not likely to be pro-poor and that a program targeted at poor students, earlier in the schooling cycle is needed if the goal is to reach the poorest of the poor. Based in part on these findings, and on a desire to assess the viability, effectiveness and optimal design of such a program, the Royal Government of Cambodia included a pilot primary school 4 This program, formerly called PAP12, is operated from the government s Program Budget, the Japan Fund for Poverty Reduction (JFPR) Scholarships funded by the Asian Development Bank and UNICEF, the Belgian Education and Training Trust (BETT) Scholarships, and the Cambodia Education Sector Support Project (CESSP) Scholarship Program which is funded through a World Bank project. Students who were receiving JFPR and BETT scholarships, but who were threatened by a cessation of these scholarships because of lack of funds in the projects, were ultimately covered by the CESSP program. 5 For the JFPR evaluation see Filmer and Schady (2008); for the CESSP evaluation see Filmer and Schady (2009) and Ferreira, Filmer and Schady (2009). 4

scholarship program as a component of the activities funded by the Fast Track Initiative-Catalytic Fund (FTI-CF) Grant that it received. The stated goal of the program was to increase schooling by offsetting the direct and opportunity costs. 6 Implicitly, the goal was also to improve learning outcomes through that additional schooling. This paper reports the results of the impact evaluation of that pilot program. Program and evaluation design The basic design of the primary scholarship pilot was to select participating schools; and then within schools identify scholarship recipients according to a clear and transparent criterion. Once selected, recipients needed to stay enrolled, attend school regularly, and maintain passing grades in order to keep the scholarship until they graduate from primary school. 7 The program targeted students in selected schools entering the upper-primary level (Grades 4, 5 and 6). The scholarship amount was set at US$20 per student, per year. 8 The scholarships were intended to be disbursed in two tranches of US$10 over the school year: once towards the beginning of the year, and once towards the middle. In the first year of the program, scholarships were distributed in one lump sum due to delays in implementation. 9 The pilot program was targeted to the three Provinces where average dropout rates between grades 3 and 6 were highest, as determined by an analysis of Cambodia s Education Information Management System (EMIS). These Provinces were Mondulkiri, Ratanakiri and Preah Vihear. In order to narrow the geographic scope of the program, 7 (of 9) districts in Ratanakiri with the highest dropout rates were selected for participation, and all districts in the other Provinces were included. Within these selected districts, all primary schools which offered classes through to grade 6 participated in the program. In order to evaluate program impact, 209 schools were randomly assigned to join the program in its first year, 2008-09 (referred to as Phase 1 schools 104 schools), or in 2009-10, its second year (Phase 2 105 schools), as depicted in Figure 2. The identification of impact is based on the fact that among the cohort of students studied, Grade 4 at baseline students in randomly selected Phase 2 schools were 6 Primary schools are officially non-fee based. Opportunity costs include various forms of child labor which are relatively common in the areas under study although typically labor is combined with schooling at the primary school ages. 7 There is moderate enforcement of the conditionality. Students absent for many days are followed up by school officials and if they return to school would remain eligible for the scholarship. After a student is absent for too many days they would be classified as having dropped out and no longer be eligible for the scholarship. 8 CESSP lower-secondary scholarships were in the amounts of $45 and $60 however the evaluation found little impact on enrollment and attendance of $60 over and above $45 (Filmer and Schady 2011) 9 Scholarships distributions for the cohort of recipients in Phase 1 schools analyzed here took place in July 2009 (US$20), November 2009 (US$10); April 2010 (US$10); November 2010 (US$10); and April 2011 (US$10). 5

not eligible for scholarships. These students therefore serve as a valid counterfactual group a group that differs, on average, from the treatment group only in that it did not receive the scholarships. Since these students in control schools were never exposed to the program (even after the subsequent cohort became eligible when scholarships were implemented in Phase 2 schools), the two groups of students can be tracked over time and enrollment, attendance, and other outcomes compared. 10 Schools were further randomly allocated to one of two groups in order to evaluate the effectiveness of the alternative ways of targeting: poverty-based and merit-based targeting. The first group of schools ( poverty-based targeting) used a score, similar to that currently in use in the secondary school programs (52 Phase 1 and 53 Phase 2 schools) see Figure 2. All targeted students filled out a simple form with questions relating to their household and family socio-economic characteristics. 11 These forms were scored according to a strict formula based on weights derived from an analysis of household survey data. 12 Scoring of the actual application forms was carried out centrally by a firm contracted specifically for this purpose, thereby reducing the ability to manipulate the program. Within each school, the applicants with the highest scores (i.e. the highest poverty ) were selected to be offered a scholarship. In a second group of schools ( merit-based targeting), applicants were ranked based on scores on a test of learning achievement (52 Phase 1 and 52 Phase 2 schools). The test was adapted from the Grade 3 National Learning Assessment which was developed under the Cambodia Education Sector Support Project (CESSP) project. 13 All eligible students took the test and, within each school, the applicants with the highest test scores were selected to be offered a scholarship. Again, scoring the tests was done centrally in order to minimize the risk of program manipulation. The number of students in each school type was fixed exogenously, and set to half the number of registered students in the year prior to the program (as determined by an analysis of EMIS data). 14 10 Because scholarship offers are made according to a strict criterion within schools, applicants just above and just below the cutoff for eligibility could be studied using a regression discontinuity design (RDD) approach to evaluate impact. Future work based on this program will exploit that approach, and will be able to use data from the first and second cohorts of students who applied to the program. 11 Table 1 reports the full set of variables included in the calculation of the score. 12 The weights were determined by estimating a model predicting the probability that a student would drop out of school during grades 4 to 6 since addressing this dropout was the stated goal of the program. Strictly speaking, therefore, the score should be referred to a dropout-risk score. However, the risk is essentially a set of household characteristics that capture the socioeconomic status of a household weighted to capture those elements that predict dropout best. For convenience and ease of exposition, the score is referred to in this paper, as well as in program documents, as a poverty score. 13 The National Assessment was implemented nationwide in Grade 3 in a sample of schools during the 2005/06 school year (Royal Government of Cambodia 2006). 14 The number of scholarships is not equal to half the number of applicants because (1) the rule was to allocate scholarships to all applicants who had the cutoff score and ties mean that more applicants would receive a scholarship offer, or (2) because of changes in enrollment numbers from year-to year. 6

Implementation and data This study evaluates the impact of the program on first cohort of students who filled out application forms when the program began implementation in December 2008/January 2009. At the time, these students were in the 4 th grade. 15 All 4 th grade students in program schools filled out the application forms as well as took the assessment test both students in Phase 1 and Phase 2 schools, as well as students in poverty-targeting and merit-targeting schools (see Figure 2). Recipients received scholarships disbursements (on condition of remaining in school, attending regularly, and maintaining passing grades) during the 2008/09, 2009/2010 and 2010/11 school years. We use three main data sources to evaluate program impact. First, we use the full set of data collected at the time students applied for the scholarships. That is, we have information on baseline household characteristics as well as baseline math and Khmer language test scores for all applicants. Second, we use the official list of students who were offered a scholarship. Third, we use endline data that were collected specifically for this evaluation. These data are derived from a survey of a random subsample of students from each program school that was administered at the end of the 2010/11 school year, when a student who would have stayed in school in the correct grade would have been finishing (or just finished) grade 6. The survey was administered in households (i.e. not in schools) to the child who applied for the scholarship, and included a household module administered to their mother, father or other caregiver. In total 3,618 applicants were interviewed. 16 For the bulk of the analysis we use data from 1,377 students in grade 4 at baseline who were offered a poverty- or merit-based scholarship, or would have been offered a poverty- or merit-based scholarship had they been going to a program school. The survey asks about a broad range of issues, both in terms of school participation (for example the intensity of school participation though questions relating to time spent in school), other activities such as labor market participation, as well as various measures of cognitive development and learning achievement. Importantly, given that these data are collected at the household level the questions can be asked of both recipients and non-recipients whether they are in school or not. As such, another contribution of this study is that it avoids the problem of selecting only children who are enrolled and attending school when analyzing learning achievement data collected at the school (Kremer, Miguel and 15 These students were supposed to have filled out application forms prior to the beginning of the school year, i.e. when they were still in 3 rd grade. Because of delays in effectiveness and implementation of the overall project, the application process could only be implemented once the students had begun 4 th grade. 16 Attrition was 15%. Analysis of attrition patterns show that the share of attritors is not different by Phase 1/Phase 2 schools, nor is it related to poverty/merit status of the school. 7

Thornton, 2009). Three learning achievement tests were administered: a mathematics test, a Digitspan test and Ravens Progressive Matrices test. 17 The items on the Mathematics test were drawn from a variety of sources including the baseline mathematics test; questions from the national grade 6 assessment; questions drawn from publicly released items from the Trends in International Maths and Science (TIMSS) Grade 4 Assessment. Items were tested during a pretest and only items with adequate properties were retained for the final test. It is a multiple choice test, measuring both knowledge and capacity to use this knowledge to solve specific problems. Presumably, this is the measure of the most immediate academic impact of the intervention, since exposure of the program can directly affect the ability to solve mathematical problems. The Digitspan test is a test in which a series of numbers are read to a respondent who is then asked to repeat the numbers back to the enumerator. The series increases from 2 numbers to a larger and larger number, until 9 digits. Respondents are also asked to repeat the numbers back in reverse order. The test is typically interpreted as a measure of short term memory and working memory capacity. In the Ravens Progressive Matrices test respondents are shown a set of three images each with a pattern that links to the others. They are then shown a set of potential images, one of which link to the three original images, and are instructed to tell the enumerator which one completes the first three. The test is typically interpreted as a measure of logical reasoning. 3. Empirical strategy Empirical Strategy We estimate the reduced-form of the program impact on enrollment and attendance outcomes; on test scores; and on potential mechanisms of transmission (school / teacher effort and student / household effort). The estimation is based on the equation (1) denotes the outcome variable for individual i at follow-up (t1); is in indicative of treatment status; is a vector of controls measured at baseline; and captures unobserved students 17 A vocabulary test was also included. The results of this test were in general terms very imprecise and tending towards zero possibly because of problems in translating words and concepts into Khmer. For the sake of simplicity, we do not report these results. 8

characteristics and idiosyncratic shocks. The controls includes baseline values for gender, number of minors in the household, indicators for whether the household owns a motorcycle, a car/truck, an oxen/buffalo, a pig, an ox or buffalo cart; indicators for whether the house has a hard roof, a hard wall, a hard floor, an automatic toilet, a pit toilet, electricity, piped water; as well as the overall poverty index and test scores at baseline. The estimation is done separate for each targeting mechanism. Accordingly, for students in meritbased targeting schools, is equal one if offered the merit-based scholarship, and zero for untreated students in the control schools who would have been eligible for merit-based scholarships based on their test scores had they attended a treatment school. An analogous treatment indicator is built for the poverty-based targeting mechanism. Given that the treatment variable identifies at baseline all individuals offered the scholarship, the estimation is in practice an intention to treat estimator (ITT). Errors are clustered at school level, and each estimation includes a district level fixed effects. In order to gain efficiency, we run seemly unrelated regressions (SUR) in estimating the impact on test scores. The design of the intervention also allows for directly estimating the effects of the program on non-treated students. In each treatment school, approximately half of the students were not treated. Therefore, given the random assignment of schools into treatment, we can compare the non-poor (and non-treated) students in the poverty treatment schools with non-poor (and non-treated) students in control schools. Similarly, we can compare students who did not received scholarships in merit-treated schools with control school students who would not have received merit-scholarships in control schools. In order to estimate this, we used Equation (1), but replacing with the appropriate group of students. In this program the scholarship is offered after the baseline test is done, and during the duration of the program, non-scholarship recipients cannot change their status. Therefore, any effects on non-treated students emanate from complementarities and interactions between treated and non-treated students during the academic year. 18 Baseline balance and characterization of the study sample This section presents the general characteristics of the study sample and the validation of the random assignment by comparing treatment and control students at the baseline. In Table 1, columns (1) and (2) are based on all students in the control and the treatment schools, whereas columns (3)-(6) only 18 These peer effects are different in nature with the externality effects estimated in Kremer et al (2009). The authors of that paper estimate the effect of the promise of scholarship on students with low scores pre intervention. Those effects presumably emanate from the effort that all students may exercise in order to get the scholarship. 9

use information of the treated students in the treatment schools and untreated students in the control schools who would have been eligible for treatment based on their poverty index score or on the baseline test score had they attended a treatment school. The sample is restricted to Grade 4 students at baseline, and as described above, the control group of students was not part of the phase-in expansion of the program. Columns (1), (3), (5) show the means and standard deviations of household and individual characteristics prior to the intervention. Columns (2), (4) and (6) show the differences and standard errors at baseline between the treatment and control groups. That is, the values are the coefficient estimates from a regression of each characteristic on a dummy variable equal to 1 for a treatment school. Two main features emerge from Table 1. First, treatment and control groups are similar in observed characteristics. Only a few coefficients in columns (2), (4) and (6) are statistically significant (of the 48 differences reported, 5 are significant at the 10% level, and among them, 1 is significant at the 1% level). The results in Table 1 confirm the validity of the random assignment since both control and treatment groups are similar in their observed characteristics. More important, in means, the poverty index and the test score are equal between treatment and control groups. Figures 3 and 4 present the density of poverty index at baseline for treatment and control schools. There is a clear overlap along the whole distribution. We cannot reject equality of both distributions using a Kolmogorov-Smirnov test. Second, on average, the recipients who are offered scholarships on the basis of the merit-based targeting have more assets, a lower poverty index score and better performance on the baseline test than students in the poverty-based treatment. For instance, the poverty index, which ranges from 0 (wealthiest family in the sample) to 292 (poorest family in the sample), has a mean of 245.13 for poverty based students, and 218.2 for merit-based students. Likewise, the baseline test (ranging from 0 to 25) has a mean of 19.77 for merit-based students and 17.74 for poverty-based students. We return to this issue in the discussion below where we discuss tradeoffs between the two targeting approaches. The main finding from Table 1, however, is that it shows a balanced sample between treatment and control groups at baseline, which is a key determinant of the random assignment approach being a valid identification strategy. 4. Results Impacts on enrollment and attendance The intervention is aimed directly at incentivizing higher enrollment and attendance. In order to keep the scholarship, the selected students must stay enrolled, attend school regularly, and maintain 10

passing grades until they graduate from primary school (sixth grade). We focus on three enrollment and attendance proxies: the proportion of students reaching 6 th grade, the highest grade completed, and the hours of school attended in the past seven days. In order to provide a baseline for assessing the relative magnitude of impacts, Table 2 reports outcome variables of the students at the follow up in the control group. Between 61% and 64% of the students reported reaching at least sixth grade and the average grade completion is around 5.4.. The third outcome variable was constructed from a question that asked students how many hours they attended school the past seven days, conditional on being enrolled. Depending on the control group, students reported an average having attended school for about 8.83 hours (poverty) and 9.27 hours (merit) in the past week. Table 3 reports the program impacts on the enrollment and attendance proxies. Columns (1) and (2) present the effects of treatment of the poverty and merit-based interventions, respectively, when controlling for baseline student characteristics (i.e. the variables through the application forms, reported in Table 1), poverty index and test scores at baseline, and province fixed effects. As discussed above, the randomized assignment was successful in that it produced a balanced sample. Controlling for additional variables in the program impact regressions should therefore only affect the precision of the estimates, not the magnitude of the estimated effects. Appendix 1 presents the results without controls; as expected, the results are very similar to the results in Table 3. Overall, Table 3 shows consistent evidence of positive impacts from the interventions on enrollment and attendance. The proportion of students reaching grade 6 increased with both treatments, and the effects are similar in magnitude. The estimated impacts range between 12% point and 17% point increase from a counterfactual of around 61%-64%. Similarly, the intervention increased the average highest grade completed in both treatment samples, with effects ranging from 0.332 in poverty-based intervention to 0.187 grades merit-based, from a counterfactual of about 5.4 years. These impacts are similar than those found in the context of the Secondary School scholarships program (where impacts on enrollment were on the order of 20-25% point increase). These impacts are larger than most documented in countries elsewhere in the world (Fiszbein and Schady 2011), and should be assessed against the very small size of the transfer considered (i.e. $US20 per year). The measure of attendance (number of hours in school in the past seven days) shows positive estimates, but none of these are statistically significant, with the exception of the estimate without controls for poverty treatment. Nevertheless, taking the point estimates at face value, the results suggest 11

that the intervention increases attendance by on the order of 2.9 hours per week for poverty-based treatment and 0.64 for merit-based. In sum, there is strong evidence that the program increased enrollment and suggestive evidence that it also improved attendance rates, regardless if the scholarship is based on merit or on poverty status the targeting approach did therefore not affect the extent to which the program increased measured school participation. Results on test scores There are two main channels through which the program could impact test scores. First, by incentivizing enrollment and attendance, students are more exposed to school and through that additional schooling acquire more learning. Second, by requiring that the students maintain passing grades, the program may give students an incentive to study more. Table 4 presents impacts of the interventions on the three measures of academic and cognitive achievement: Mathematics test, Digitspan test, and Raven test. (Table 4 has the same structure as Table 3; it present impact coefficients after controlling for baseline characteristics and district fixed effects; Appendix A presents results without controls). Given that we use three, likely correlated, measures of test scores, we estimate the model using Seemingly Unrelated Regressions (SUR) to gain efficiency. All three measures are standardized using the mean and deviation of the respective control group impacts can therefore be interpreted as changes in a standard deviation of the achievement measure. In contrast to the results on enrollment and attendance, Table 4 reveals different impacts on test scores between the poverty and the merit treatment group. While the impact estimates show no effects from treatment for students treated based on their poverty status, there is a clear trend of positive effects from the intervention for students treated based on merit. All the point estimates for the merit treatment are positive and statistically significant. For the merit sample when using additional controls, the effect of the intervention on the math test is 0.170 standard deviations, the effect on the Digitspan test is 0.149 standard deviations, and the effect for the Raven test is 0.178 standard deviations. The results on math are of particular interest, since math is potentially the measure that is the most sensitive to exposure to additional schooling. Moreover, households have potentially less ability to substitute for teaching math than vocabulary or other types of learning. These effects on test scores are similar in magnitude to the merit-based scholarship program evaluated in Kenya (Kremer, Miguel and Thornton 2009). 12

In sum, the results suggest that the program incentivized both types of students those from poorer households and those with higher academic merit to enroll in additional years of schooling and have higher attendance. However, only students who received treatment based on merit show any gains in test scores from the intervention. Heterogeneity All in all, it seems that the program incentivized both types of students those with higher academic merit and those from poorer households to enroll and attend additional years of school. However, only students who received treatment based on merit show any gains in academic achievement from the intervention. That is, additional schooling (e.g. enrollment and attendance) results in better learning outcomes (as captured by our four tests) only for those students who had better skills at the baseline. At the light of this result, it is important to explore the heterogeneity of effects by baseline skill and poverty levels. These effects are presented in Table 5. For each poverty school (treated and control) we identify students that were above and below each school s median baseline test, and then we run separate regressions to estimate the effect for the two types of students (high baseline achievers and low baseline achievers). For each follow-up test (math, Digitspan and Raven), Column (1) presents the results on test scores of the effects of the poverty-targeting mechanism for low achievers at baseline and Column (2) for high baseline achievers. In an analogous way, for merit-based schools (treatment and controls) we identify students above and below each school s median poverty index, and then, we run separate regressions for the two populations. For each follow-up test, Column (3) presents the impact of meritbased treatment for those above the school s median poverty index (non-poor population) and Column (4) presents the impact of merit-based treatment for students below the school median poverty index (poor population). 19 As Table 5 shows, the merit-based treatment has either similar or larger positive impacts on the poor population than in the non-poor population. In contrast, the poverty-based treatment does not elicit positive results from either the high baseline achievers or the low baseline achievers. In other words, the asymmetry in test results for poverty-based and merit-based targeting mechanisms persists. More importantly, the poverty-based treatment does not induce better test results among high baseline performers, whereas the merit-based treatment does induce better test scores among baseline poor individuals, despite the fact that the conditions of both scholarships merit and poverty are the same. 19 An alternative specification is to run the pooled regression with a dummy variable for treatment, a dummy variable indicating the status at base line, and the interaction term. The results are very similar to the ones presented in Table 5. 13

Given the results for poor individuals in the merit-based treatment, similar results were expected for the high achievers in the poverty-based treatment. Different mechanisms can explain the asymmetry in tests results. As argued before, both types of scholarships provide incentives for students to increase effort. For instance, it is plausible that, due to the scholarship, students increase hours of studying outside school. Also, families may be motivated to invest more in education expenditures, textbooks and such and as a result, help the student in conserving the scholarship. Likewise, the program can impact directly the behavior of the school and teachers. For instance, under an altruistic model, teachers can increase attention to students with scholarships with the hope that they can retain the money. Also, it is possible that presents from scholarship winners parents to teacher can induce higher effort. As such, the school can change behavior. Banerjee and Duflo (2006), while presenting the results of Kremer et al (2009), discuss changes in teacher motivation and higher control of families. As mentioned before, the follow-up data comes from household interview. As such, the drawback of household interviews is the lack of school level information. However, household interviews allow us to gather evidence regarding potential channels that may explain the results, such as, student effort outside of school and a household s investment in education. Students effort is captured by the amount of time spent studying, doing homework and taking private lessons outside of school. The household s effort is measured by the total amount of education expenditure by the household and the proportion of this expenditure spent on textbooks. The effects of treatment on each of these three variables are presented in Table 6. On average, control students spent 3.5 hours per week doing school tasks outside the school. Merit-treated students spent more time doing academic work outside of school (an increment of 0.579 hours). The household response to scholarships in terms of education spending and the nature of that spending differs across the scholarship types. On average, household s education expenditures are approximately U$17. Households with a merit scholarship recipient spent U$ 5 more on education, and a higher proportion (1% more) of the expenditure was on textbooks, than control students households. 20 In contrast, there were no impacts on these outcomes in households with poverty targeted recipients. The results suggest that only household with merit-based students increase effort. All in all, it seems that families in the merit-based treatment invest more in the education of students; in contrast to what happen in families with the poverty-based treatment. Also, students in the merit-based treatment put more effort outside school than the poverty-based students. These findings are 20 For the text book expenditures, the p value of the coefficient is 0.105 14

compatible with a motivation hypothesis: merit-based students are motivated to work more, and their families are motivated to invest more. Ideally we would like to disentangle the motivation of students and from the motivation of teachers, however, the data available do not allow us to carry such analyses. Peer effects on enrollment, attendance and achievement One concern with a program such as this is that there might be negative spillover effects. If increased enrollment and attendance leads to classroom overcrowding, then this may hurt the learning opportunities afforded to other students. At the same time, there might also be positive spillover effects if scholarships create a positive energy in favor of schooling and learning which could affect all children in a classroom. 21 Moreover, peer effects might differ by the design of the targeting approach: for instance applicants denied a merit-based scholarship might become discouraged, and eventually perform worse. Since there will always be limited scope for containing spillover effects, they should be estimated directly, and factored into the equity/efficiency tradeoff in program design. As discussed above, the design of the intervention allows us to test directly for peer effects. Table 9 presents the coefficient estimates for the indicator variable that is one for those students not treated in treated schools, and zero for students in control schools with analogous baseline test scores or poverty index scores. The dependent variables are proportion of students reaching 6 th grade, the highest grade completed, and the hours of school attended. The estimates can be read similarly to those reported in Table 3, but estimated off of the sample of non-recipients and their counterfactual. There seems to be a positive peer effect in poverty-treated schools: the three coefficients are positive; and one of them is significant at the 5% level. Moreover, as expected, the point estimates are lower than the direct effect of the program (Table 3). Regarding the merit-based treatment, we do not find any evidence of peer effects, negative or positive. 22 Table 10 is the analogous table for the three measures of test scores. None of the coefficient estimates are statistically significant. All the coefficients are very close to zero. In sum, it seems that the poverty-based treatment may induce more attendance from non-treated peers. No other peer effects are detected, neither positive nor negative, from the program. 5. Conclusions 21 An additional positive spillover might be on younger cohorts who stay in school longer in order to potentially benefit from scholarships. We don t have the data available to address this issue. 22 The results therefore dispel the notion that there was discouragement among merit scholarship applicants who did not receive a scholarship. 15

The fact that some students were able to take better academic advantage from more exposure to school than others highlights an issue rarely addressed in previous evaluations of conditional cash transfer programs. Recent evidence on monetary incentives for schooling shows that students are able to change their behavior on the margins that are under their control for example enrollment and attendance. However, these positive effects do not necessarily translate into test score gains. For example, despite the fact that Mexico s Oportunidades program a rigorously evaluated conditional cash transfer program induced students to enroll and attend more to school, the program did not induce higher test scores. A recent set of papers has argued that education systems in developing countries are typically tailored towards better-off and better-skilled students. Specifically, Glewwe, Kremer and Moulin (2009) show that only the strongest students at baseline were able to take advantage of textbooks that were provided to schools in Kenya; Duflo, Dupas and Kremer (2011), while studying the effects of tracking students into classrooms according to initial achievement (also in Kenya), show that teachers who were assigned to students at the bottom of the achievement distribution were less likely to teach. Our findings based on the Cambodian Primary Scholarships Pilot add to this discussion. On the one hand, additional exposure seems to pay off in terms of test scores to those students who are more academically ready to take advantage of the opportunity. On the other hand, poorer students who are not academically prepared are not able to measurably gain in terms of test scores from the additional schooling. This evidence is compatible with the idea that teachers are not prepared, or do not have the pedagogical skills, to take on the challenge of reaching the more academically challenged students. Clearly more work is needed to establish how best to prepare and incentivize teachers to reach these students and this would be an important area for Cambodia (and other countries) to generate knowledge. At the same time, however, this evaluation suggests that for students who are better academically prepared including poor students incentivizing school attendance can pay off in measurable learning outcomes. This suggests that remedial lessons for students in the early grades, or increasing school readiness among poorer students, for example through early child development programs, might be complementary approaches to increasing the impact of schooling, and programs that incentivize schooling. Indeed, data from Cambodia suggest that children suffer from substantial delays in cognitive development, which hampers school readiness (Naudeau and others 2011). The Cambodian program uses two targeting approaches setting up a potential tradeoff between efficiency defined as achieving more learning per dollar transferred versus equity defined as reaching the poorest population. Analysis of the socio-economic profile of program applicants and recipients under the two targeting schemes and comparing those to the national distribution of socioeconomic characteristics suggests that both targeting approaches are heavily weighted to the poor. The 16

first panel of Figure 5 shows that 50% of those who applied to the program are within the poorest nationally-benchmarked quintile; fewer than 3% of applicants were from the richest quintile. Clearly the program was targeted to poor areas and poor schools. Unsurprisingly, targeting the scholarships further to the poorest from within each school yields an even greater pro-poor distribution of benefits: 85% of applicants who were in the poorest half in their school (i.e. those targeted by poverty scholarships) were from the poorest two quintiles of the population 63% were in the poorest quintile (Panel 2). Meritbased targeting is not as pro-poor but is still largely able to reach the poorest groups in the population: 76% of applicants who were in the top merit half of their school (i.e. those targeted by merit scholarships) were from the poorest two quintiles of the population 54% were in the poorest quintile (Panel 7). A complementary analysis of the within-school correspondence between high/low poverty applicants and high/low test scoring applicants yields a similar conclusion: wealthier applicants were not necessarily higher-scoring. 23 Given the relatively effective geographic targeting it is unclear whether this result is generalizable. In other settings (e.g. where there is more heterogeneity in student poverty levels) the result may not hold. Nevertheless, the results suggest that for this program, the tradeoff between efficiency and equity was not particularly stark. Scaling up an approach that targets students with high academic potential while ensuring that the poorest student are among that set is likely to be the approach that maximizes both the equity and effectiveness objectives of the program. 23 See Appendix B for further details. 17

References Baird, Sarah, Craig McIntosh and Berk Ozler. 2009. Designing Cost-Effective Cash Transfer Programs to Boost Schooling Among Young Women in Sub-Saharan Africa. World Bank Policy Research Working Paper No. 5090. The World Bank. Baird, Sarah, Craig McIntosh and Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1709-1753. Banerjee, Abhijit, and Esther Duflo. 2006. "Addressing Absence." Journal of Economic Perspectives, 20(1): 117 132. Behrman, Jere R., Susan W. Parker, and Petra E. Todd. 2005. Long-TermImpacts of the Oportunidades Conditional Cash Transfer Program onrural Youth in Mexico. Discussion Paper 122, Ibero-America Institute for Economic Research, Göttingen, Germany. Behrman, Jere R., Piyali Sengupta, and Petra Todd. 2000. The Impact of PROGRESA on Achievement Test Scores in the First Year. Unpublished manuscript, International Food Policy Research Institute, Washington, DC. Chaudhury, Nazmul and Dilip Parajuli. 2008. Conditional Cash Transfers and Female Schooling: The Impact of the Female School Stipend Programme on Public School Enrolments in Punjab, Pakistan. Applied Economics. Duflo, Esther, Pascaline Dupas and Michael Kremer. 2011. Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya. American Economic Review. 101(5): 1739-74. Ferreira, Francisco H., Deon Filmer and Norbert Schady. 2009. Own and Sibling Effects of Conditional Cash Transfer Programs: Theory and Evidence from Cambodia World Bank Policy Research Working Paper No. 5001. The World Bank, Washington, DC. Filmer, Deon and Lant Pritchett. 2001. Estimating Wealth Effects without Expenditure Data or Tears: With an Application to Educational Enrollments in States of India. Demography. 2001. 38(1):115-132. Filmer, Deon and Kinnon Scott. 2011. Assessing Asset Indices. Demography. 2011. 49(1), 359-392. Filmer, Deon, and Norbert Schady. 2008. Getting Girls into School: Evidence from a Scholarship Program in Cambodia. Economic Development and Cultural Change 56(2): 581 617 Filmer, Deon and Norbert Schady. 2009. School Enrollment, Selection and Test Scores. World Bank Policy Research Working Paper No. 4998. The World Bank, Washington, DC. Filmer, Deon and Norbert Schady. 2011. Does more cash in conditional cash transfer programs always lead to larger impacts on school attendance? Journal of Development Economics. 96(1): 150 157. Fiszbein, Ariel and Norbert Schady. 2009. Conditional Cash Transfers: Reducing Present and Future Poverty. The World Bank. Washington, DC 18

Glewwe, Paul, Michael Kremer and Sylvie Moulin. 2009. Many Children Left Behind? Textbooks and Test Scores in Kenya. American Economic Journal: Applied Economics. 1(1): 112-135. Kremer, Michael, Edward Miguel, and Rebecca Thornton. 2009. Incentives to Learn. Review of Economics and Statistics. 91(3): 437-456. Naudeau, Sophie, Sebastian Martinez, Patrick Premand, and Deon Filmer. 2011. Cognitive Development among Young Children in Low-Income Countries in Alderman, Harold ed. No Small Matter: The Impact of Poverty, Shocks, and Human Capital Investments in Early Childhood Development. The World Bank. Washington, DC. Orfield, Gary, Foreword, in Donald E. Heller and Patricia Marin (Eds.), Who Should We Help? The Negative Social Consequences of Merit Aid Scholarships (2002) (Papers presented at the conference State Merit Aid Programs: College Access and Equity at Harvard University). Document found in http://civilrightsproject.ucla.edu/research/college-access/financing/who-should-we-help-the-negativesocial-consequences-of-merit-scholarships/ Royal Government of Cambodia. 2006. Student Achievement and Education Policy: Results from the Grade Three Assessment Final Report. Cambodia Education Sector Support Project National Assessment Component. Phnom Penh, Cambodia. 19

Figure 1: Proportion of 15 to 19 year olds who have completed each grade, by quintile. 1 0.8 0.6 0.4 Poorest quintile Quintile 2 Quintile 3 Quintile 4 Richest quintile 0.2 0 1 2 3 4 5 6 7 8 9 Source: DHS 2010 20

Figure 2. Design of the intervention Poverty-based schools TIME=0: BASELINE INFORMATION; LOTTERY Phase 1 Phase 2 Grade 4 Grade 3 Grade 4 292 0 Poverty index Poverty index 292 Poverty index TIME=1: PHASE-IN Grade 5 Grade 6 Grade 4 Grade 5 Grade 5 TIME=2: FOLLOW-UP Grade 6 Grade 3 0 Grade 4 Grade 5 209 Schools Lottery Grade 4 Grade 3 Poverty index 29 Test score 0 Test score Grade 5 Grade 4 Grade 6 Grade 5 Phase 1 Merit-based schools Phase 2 Grade 4 29 Test score Grade 5 Grade 6 = Treated = No treated Grade 3 0 Test score Grade 4 Grade 5 21

0 kdensity pov_scor.005.01 Figure 3: Poverty score at baseline, Treatment versus Control 0 100 200 300 Poverty score at baseline by school-level treatment status Treatment (both merit and poverty) Control Source: Students at follow-up, using baseline information 22

0 kdensity khm_math.02.04.06.08.1 Figure 4: Test scores at baseline, Treatment versus Control 0 5 10 15 20 25 Test score at baseline by school-level treatment status Treatment (both merit and poverty) Control Source: Students at follow-up, using baseline information 23

Figure 5: Distribution of selected populations across nationally benchmarked quintiles 70 60 50 40 30 20 10 0 (1) Program applicants (2) High Poverty (3) High merit Poorest quintile Quintile 2 Quintile 3 Quintile 4 Richest quintile Source: Analysis of Cambodia DHS 2010 and Primary Scholarship Application forms. Quintiles are defined on the basis of an index of household wealth-related variables that are collected in both the DHS 2010 as well as on the scholarship program application forms. 24

Table 1. Baseline Balance and mean and standard deviation of baseline characteristics School Level Student Level Control Difference Control Difference Control Difference with treatment Poverty with treatment Merit with treatment (1) (2) (3) (4) (5) (6) Gender 0.49 0.037 0.52 0.100*** 0.49-0.03 '(0.50) '(0.02) '(0.50) '(0.03) '(0.50) '(0.04) No of minors 1.69-0.026 1.79 0.075 1.73-0.097 (1.11) (0.09) '(1.12) '(0.12) '(1.12) '(0.12) Own motorcycle 0.42 0.008 0.28-0.035 0.42 0.003 (0.49) (0.04) '(0.45) '(0.05) '(0.49) '(0.05) Own car/truck 0.16 0.017 0.04 0.01 0.13 0.028 (0.37) (0.03) '(0.19) '(0.03) '(0.34) '(0.04) Own oxen/buffalo 0.55 0.032 0.39 0.109* 0.53 0.033 (0.50) (0.05) '(0.49) '(0.06) '(0.50) '(0.06) Own pig 0.56 0.028 0.43 0.117** 0.55 0.029 (0.50) (0.04) '(0.50) '(0.06) '(0.50) '(0.05) Own ox or buffalo cart 0.31 0.02 0.19 0.058 0.29 0.009 (0.46) (0.04) '(0.40) '(0.05) '(0.45) '(0.05) Hard roof 0.49 0.064 0.32 0.047 0.48 0.102** (0.50) (0.04) '(0.47) '(0.05) '(0.50) '(0.05) Hard wall 0.54 0.032 0.38 0.045 0.55 0.018 (0.50) (0.04) '(0.49) '(0.06) '(0.50) '(0.05) Hard floor 0.85 0.039 0.79 0.037 0.84 0.068* (0.36) (0.03) '(0.41) '(0.05) '(0.37) '(0.04) Have automatic toilet 0.07-0.02 0.02-0.01 0.05 0.005 (0.25) (0.02) '(0.13) '(0.01) '(0.22) '(0.02) Have pit toilet 0.12 0.018 0.11 0.02 0.13 0.001 (0.32) (0.03) '(0.32) '(0.03) '(0.34) '(0.04) Electricity 0.25 0.011 0.16-0.01 0.23-0.002 (0.43) (0.04) '(0.37) '(0.04) '(0.42) '(0.05) Pipe water 0.06-0.001 0.03-0.012 0.06-0.013 (0.24) (0.02) '(0.17) '(0.01) '(0.23) '(0.02) Poverty Index (o to 292) 210.16-1.609 245.13-2.924 218.2-11.771 (60.18) (5.43) '(32.73) '(5.14) '(51.66) '(8.76) Test score (0 to 25) 17.47 0.534 17.74 0.888 19.77 0.028 (4.81) (0.52) '(4.71) '(0.68) '(3.22) '(0.48) Number of students 940 2448 431 883 474 940 Number of schools 101 204 67 119 67 118 Columns (1), (3), (5): means and standard deviation, control group. Columns (2), (4), (6): difference with treatment, estimated by regressing each variable against corresponding treatment variable; standard error in parenthesis 25

Table 2. Outcome variables at follow-up Poverty-targeting Student Level Merit-targeting Control Treatment Control Treatment (1) (2) (3) (4) Reach grade 6 0.61 0.8 0.64 0.77 '(0.49) '(0.40) '(0.48) '(0.42) Completed grades 5.38 5.73 5.45 5.68 '(1.22) '(0.93) '(1.23) '(0.94) Number of hours 8.83 12.29 9.27 10.64 '(12.97) '(14.87) '(13.22) '(14.38) Math test 0.02-0.02 0.16 0.32 '(1.01) '(0.94) '(1.04) '(1.04) Digitspan 0.02 0 0.08 0.23 '(0.98) '(1.00) '(0.99) '(0.97) Raven Test -0.02-0.07 0.11 0.21 '(0.98) '(0.92) '(0.99) '(1.15) Number of Students 431 452 474 466 Mean and () standard deviation 26

Table 3. Impact on Enrollment and Attendance Reach Grade Six Highest Grade Completed Number of hours in school, last 7 days (conditional on enrollment) (1) (2) (1) (2) (1) (2) Poverty-targeting treatment 0.170*** 0.332*** 2.865 '(0.04) '(0.11) '(1.87) Merit-targeting treatment 0.120*** 0.182* 0.635 '(0.04) '(0.10) '(1.55) Constant 1.764 0.514 8.161** 5.285** -100.065** 0.575 '(1.42) '(0.83) '(3.35) '(2.33) '(45.16) '(28.25) Control Variables Yes Yes Yes Yes Yes Yes No. Obs 883 940 831 897 665 713 F() 6.435 4.872 2.271 1.759 1.246 2.026 R2 Adj 0.18 0.155 0.145 0.122 0.131 0.199 Regression coefficient of dependent variable against treatment indicator controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level. 27

Table 4. Impact on test scores Mathematics Digispan Test Raven Test (1) (2) (1) (2) (1) (2) Poverty-targeting treatment -0.041-0.059-0.021 '(0.06) '(0.07) '(0.06) Merit-targeting treatment 0.170*** 0.149** 0.178*** '(0.07) '(0.06) '(0.07) Constant -3.204 1.831-1.977 0.000*** 0.000*** 2.055 '(3.26) '(2.19) '(3.42) '(2.19) '(0.00) '(2.19) Control Yes Yes Yes Yes Yes Yes No. Obs 883 940 883 940 883 940 Chi_ 2 177.525 178.627 112.442 122.093 130.253 169.017 R2 Adj 0.167 0.16 0.113 0.093 0.126 0.152 Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation 28

Table 5. Impact on test scores, by baseline test score and poverty index Mathematics Digispan Test Raven Test Baseline test score Baseline poverty index Baseline test score Baseline poverty index Baseline test score Baseline poverty index Nonpoor Non- Non- Low High Poor Low High poor Poor Low High poor Poor (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) Povertytargeting treatment -0.019-0.069-0.077-0.077 0.163-0.103 '(0.09) '(0.09) '(0.11) '(0.09) '(0.10) '(0.08) Merittargeting treatment 0.053 0.233** 0.194* 0.142* 0.107 0.222** '(0.09) '(0.09) '(0.10) '(0.08) '(0.10) '(0.09) Constant 0.000*** -7.26 0.000*** 0.000*** 1.716 0.000*** 0.000*** -2.981 7.520* 0.000*** 0.000*** 0.000*** '(0.00) '(5.64) '(0.00) '(0.00) '(4.80) '(0.00) '(0.00) '(0.00) '(4.49) '(0.00) '(0.00) '(0.00) Control Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No. Obs 332 551 427 513 332 551 427 513 332 551 427 513 Chi_ 2 108.149 119.13 128.474 182.44 78.243 95.644 84.833 89.501 81.429 99.055 96.842 149.375 R2 Adj 0.209 0.178 0.203 0.212 0.191 0.144 0.148 0.149 0.197 0.152 0.169 0.207 Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation 29

Table 6. Impact on test scores on Non-treated Peers Mathematics Digitspan Test Raven Test (1) (2) (1) (2) (1) (2) Poverty-targeting treatment -0.116-0.08-0.173* (0.09) (0.09) (0.09) Merit-targeting treatment 0.033 0.091 0.076 (0.09) (0.10) (0.09) Constant 0.000*** 0.000*** 0.000*** -1.202 1.385 2.511 (0.00) (0.00) (0.00) (0.00) (2.44) (0.00) Control Variables Yes Yes Yes Yes Yes Yes No. Obs 591 503 591 503 591 503 F() 131.4 122.862 94.067 101.866 121.76 92.4 R2 Adj 0.181 0.127 0.137 0.168 0.171 0.155 Regression coefficient controlling by Table 1 baseline characteristics and district fixed effects. All estimators using SUR 30

Table 7. Impact on test scores, by baseline test score and poverty index Mathematics Digitspan Raven Test Baseline test score Baseline poverty index Baseline test score Baseline poverty index Baseline test score Baseline poverty index Low High Rich Poor Low High Rich Poor Low High Rich Poor (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) Povertytargeting treatment -0.011-0.128-0.102 0.007 0.053-0.076 (0.11) (0.12) (0.11) (0.13) (0.11) (0.12) Merittargeting treatment -0.022 0.259** 0.201 0.046 0.063 0.185 (0.12) (0.11) (0.13) (0.11) (0.12) (0.12) Constant 0.000*** 5.126 0.000*** 4.871 0.000*** 0.000*** 0.000*** -5.445 0.000*** 0.000*** 0.000*** 2.074 (0.00) (11.38) (0.00) (5.28) (0.00) (0.00) (0.00) (5.28) (0.00) (0.00) (0.00) (5.28) Control Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No. Obs 319 352 312 394 319 352 312 394 319 352 312 394 Chi_ 2 83.269 103.635 127.389 135.978 60.946 81.716 92.082 79.902 84.686 80.668 129.416 122.469 R2 Adj 0.204 0.227 0.265 0.257 0.158 0.187 0.21 0.169 0.204 0.186 0.279 0.237 Regression coefficient controlling by Table 1 baseline characteristics and district fixed effects. All estimators using SUR 31

Table 8. Potential mechanisms for impact School Open Yesterday School Responses Teacher Absence/Sick Yesterday Time doing homework, studying or private lessons Student and Household (hh) effort Total expenditure in education, hh Percentage of expenditure in textbooks, hh (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) Poverty-targeting treatment 0.039 0.044 0.314-4013.14-0.001 (0.07) (0.03) (0.45) (14468.91) (0.01) Merit-targeting treatment 0.072 0.023 0.321 20936.620* 0.014* (0.06) (0.03) (0.40) (11708.08) (0.01) Constant -3.323-0.019-0.679 1.232-19.035 3.64 1.35E+05 1.23E+06-0.237* -0.095 (2.07) (1.22) (0.55) (0.88) (17.32) (11.76) (212286.02) (790015.70) (0.13) (0.13) Control Variables Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No. Obs 518 540 671 706 498 534 671 706 550 595 F() 3.529 3.072 1.883 1.236 8.016 3.39 4.487 3.207 3.368 1.878 R2 Adj 0.166 0.249 0.101 0.159 0.176 0.195 0.131 0.134 0.097 0.095 Regression coefficient controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level. 32

Table 9. Impact on Enrollment and Attendance on Non-treated peers Reach Grade Six Highest Grade Completed Number of hours in school, last 7 days (conditional on enrollment) (1) (2) (1) (2) (1) (2) Poverty-targeting treatment 0.082** 0.058 0.529 '(0.04) '(0.11) '(1.82) Merit-targeting treatment -0.009-0.099 0.291 '(0.05) '(0.12) '(1.67) Constant 0.168 2.201** 4.030** 8.599*** 12.924 14.213 '(0.83) '(0.94) '(1.92) '(1.81) '(29.16) '(28.46) Control Variables Yes Yes Yes Yes Yes Yes No. Obs 785 678 732 633 576 486 F() 7.528 6.94 1.603 1.765 2.166 3.195 R2 Adj 0.172 0.183 0.125 0.118 0.191 0.21 Regression coefficient of dependent variable against treatment indicator controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level. 33

Table 10. Impact on test scores on Non-treated Peers Mathematics Digispan Test Raven Test (1) (2) (1) (2) (1) (2) Poverty-targeting treatment -0.105-0.009-0.09 '(0.07) '(0.07) '(0.07) Merit-targeting treatment -0.061-0.048-0.028 '(0.07) '(0.08) '(0.08) Constant 0.000*** 0.000*** -0.922-0.302 0.359 2.405 '(0.00) '(0.00) '(1.93) '(0.00) '(1.91) '(0.00) Control Variables Yes Yes Yes Yes Yes Yes No. Obs 785 678 785 678 785 678 F() 143.244 148.59 101.985 101.983 132.628 97.794 R2 Adj 0.153 0.129 0.115 0.131 0.145 0.126 Regression coefficient of dependent variable against treatment indicator, controlling by Table 1 baseline characteristics and district fixed effects. All estimators using Seemingly Unrelated Regression (SUR) estimation 34

Appendix A. Enrollment and test result, without controls Impact on Enrollment and Attendance Reach Grade Six Highest Grade Completed Number of hours in school, last 7 days (conditional on enrollment) (1) (4) (1) (4) (1) (4) Poverty-targeting treatment 0.186*** 0.349*** 3.466* '(0.04) '(0.11) '(1.80) Merit-targeting treatment 0.131*** 0.234** 1.374 '(0.05) '(0.11) '(2.01) Constant 0.613*** 0.635*** 5.377*** 5.448*** 8.829*** 9.270*** '(0.03) '(0.03) '(0.09) '(0.08) '(1.15) '(1.14) Control Variables No No No No No No No. Obs 883 940 831 897 665 713 F() 18.154 7.753 9.334 4.572 3.691 0.465 R2 Adj 0.042 0.02 0.025 0.011 0.015 0.002 Columns (1) and (3), regression coefficient of dependent variable against treatment indicator without controls; Columns (2) and (4), controlling by Table 1 baseline characteristics and district fixed effects. All estimators using clusters at school level. Impact on test scores Mathematics Digispan Test Raven Test (1) (2) (1) (2) (1) (2) Poverty-targeting treatment -0.035-0.019-0.047 '(0.07) '(0.07) '(0.06) Merit-targeting treatment 0.150** 0.147** 0.109 '(0.07) '(0.06) '(0.07) Constant 0.018 0.165*** 0.015 0.081* -0.023 0.106** '(0.05) '(0.05) '(0.05) '(0.05) '(0.05) '(0.05) Control No No No No No No No. Obs 883 940 883 940 883 940 Chi_ 2 0.291 4.925 0.085 5.273 0.54 2.426 R2 Adj 0 0.005 0 0.006 0.001 0.003 Regression coefficient of dependent variable against treatment indicator without controls; All estimators using Seemingly Unrelated Regression (SUR) characteristics 35

Appendix B: Equity and efficiency trade-off Benefit incidence The primary scholarships pilot was in part motivated by the fact that secondary school scholarships failed to reach the poorest of the poor as they had dropped out of school before becoming eligible for those programs. In order to assess the benefit incidence of the primary scholarships pilot, the data from the applicants and recipients need to be compared to a national survey. The Demographic and Health Survey (DHS) data collected in 2010 provide a useful comparison point. These data are national in scope, and include the same set of variables than were collected from program applicants, and the timing of the survey overlaps with the Primary Scholarships Pilot. The following procedure was followed to place the DHS and application form data on a common metric. First, the DHS variables (e.g. ownership of a motorbike, a car, a pig; material of dwelling floor, walls, roof) were aggregated using principal components (following the approach described in Filmer and Pritchett 2001, and Filmer and Scott 2011). 24 The first principal component from this procedure is interpreted as a wealth index from which quintiles can be derived. The first three panels of Figure A1 show the percentage distribution across quintiles of the Cambodian population of 8-15 year olds, of those in rural areas, and of those in the program Provinces. 25 Clearly the population in areas served by the program is poorer than the other areas of the country: while the distribution of 8-15 year olds mirrors the population as a whole (i.e. 20% in each quintile), and the rural population is slightly more skewed towards the poorer quintiles (with only 12% of the rural population being in the richest quintile), the population of the program Provinces is heavily concentrated among the poorest quintiles. Indeed, 30% of 8-15 year olds within program provinces are in the poorest quintile, and only 9% are in the richest quintile (Panel 3 of Figure A1). Considering children who have completed grade 3, however, reduces the share in the poorest quintile the poorest of the poor do not even make it to grade 3. While 30% of children in the program 24 The procedure was run on the sample of 8 to 15 year olds to reflect the age of those who applied to the program. The full set of variables used is: number of household members 0-14; ownership of motorbike, car, oxen/buffalo, pig; dwelling roof made of hard materials; dwelling walls made of hard materials (e.g. concrete); dwelling walls made of wood; dwelling floor made of hard materials (e.g. concrete); dwelling floor made of wood; flush toilet, pit latrine; availability of electricity; drinking water from pipe (either in house or yard). 25 The DHS data do not distinguish between Preah Vihear and Stung Treng, and these numbers therefore include both Provinces, in addition to Ratanakiri and Mondolkiri. 36

provinces are in the poorest quintile, only 20% of those who have attained grade three are in that quintile (Panels 3 and 4 of Figure A1). The variables captured in the scholarship application forms can be aggregated using the same weights derived from the national sample in the principal components procedure described above. Applicants can therefore be assigned to nationally-benchmarked quintiles, and the distribution of applicants and recipients can be compared to the national distribution. The program was able to reach the poorest schools within the program provinces: 50% of all those who applied to the program are within the poorest nationally-benchmarked quintile; fewer than 3% of applicants were from the richest quintile (Panel 5 of Figure A1). Unsurprisingly, targeting the scholarships further to the poorest from within each school yields an even greater pro-poor distribution of benefits: 85% of applicants who were in the poorest half in their school (i.e. those targeted by poverty scholarships) were from the poorest two quintiles of the population 63% were in the poorest quintile (Panel 6). Merit-based targeting is not as pro-poor but is still largely able to reach the poorest groups in the population: 76% of applicants who were in the top merit half of their school (i.e. those targeted by merit scholarships) were from the poorest two quintiles of the population 54% were in the poorest quintile (Panel 7). 26 There are two main conclusions that can be drawn from this analysis. First, compared to targeting schemes in other countries, the benefit incidence of the scholarships pilot is very pro-poor. 27 Of course, the program was implemented in some of the poorest and remote Provinces and Districts; therefore the universe from which the merit-based recipients were selected was relatively poor. Scaling up the program would not necessarily achieve a similarly pro-poor benefit incidence as expansion would mean operating in less poor areas, and therefore the baseline poverty of the population served would be less severe. Second, the fact that the benefit incidence of the merit-based approach to targeting is largely pro-poor (and not particularly less pro-poor than the poverty based approach) suggests that the tradeoff between equity (i.e. pro-poorness of the program) and efficiency (i.e. the impact on learning outcomes) might not that stark. Poverty- versus merit-based targeting at the school level 26 A recent review of programs globally reported the share of Conditional Cash Transfer program benefits that were received by the poorest 20% of the population (World Bank Social Protection Atlas, http://data.worldbank.org/datacatalog/atlas_social_protection). Globally the average was that 47% of benefits reached the poorest 20%; with a range from 24% in Bangladesh, to 58% in Panama. These findings suggest that the Cambodian program performs relatively very well. 27 For a review of the incidence across a variety of programs see World Bank Social Protection Atlas (forthcoming). 37

The finding that merit-based targeting did not result in an overall regressive scheme is a reassuring result. Figure A2 shows that within schools, the association between poverty and test scores is not as close as might have been feared. The horizontal axis of Figure A2 is the relative poverty ranking of an applicant, where 0 is the 50 th percentile (which was the cutoff for scholarship eligibility in povertytargeting schools), +1 is the applicant ranked one position higher on the poverty scale, and -1 is the applicant ranked one position lower. The vertical axis is the relative ranking of an applicant on the merit scale, again with 0 being the 50 th percentile (which was the cutoff for scholarship eligibility in the merittargeting schools). If only wealthier children were to score high on the merit test, and poorer children low, then all the observations would be in quadrants (A) and (D) of Figure A2 (respectively: Low Poverty/High Merit and High Poverty/Low Merit). Clearly this is not the case: the observations are roughly equally distributed across the four quadrants. 28 This means that a merit-based targeting approach (which targets children in quadrants A and B) includes children from both wealthier backgrounds (quadrant A) as well as children from poorer backgrounds (quadrant B). Analogously, a poverty-based targeting approach includes both higher scoring (quadrant B) and lower scoring (quadrant D) applicants. These school-specific rankings are consistent with the benefit incidence analysis. The equity/efficiency tradeoff between poverty- and merit-based targeting is not particularly stark. Nevertheless, if the purely merit-based approach is adopted, it must be borne in mind that roughly half of the recipients will come from better off families. 28 In fact the regression line for this figure has a mildly positive slope: the regression of relative merit ranking versus relative poverty ranking yields a coefficient of 1.2 with a standard error of 0.17 (significant at the 1% level). 38

-15-10 Relative merit ranking -5 0 5 10 15 Figure A1: Distribution of selected populations across nationally benchmarked quintiles 70 60 50 40 30 20 10 0 (1) All (2) Cambodia rural (3) Program Provinces (4) Program provinces, attained grade 3 (5) Program applicants (6) High Poverty applicants (7) High merit applicants Poorest quintile Quintile 2 Quintile 3 Quintile 4 Richest quintile Source: Analysis of Cambodia DHS 2010 and Primary Scholarship Application forms. Quintiles are defined on the basis of an index of household wealth-related variables that are collected in both the DHS 2010 as well as on the scholarship program application forms. Figure A2: The association between applicants relative poverty and relative merit rankings A) 21% B) 27% C) 27% D) 25% -15-10 -5 0 5 10 15 Relative poverty ranking 39