Andhra Pradesh School Choice Project Proposal 1. Background: In recent years, access to primary education has expanded tremendously in India and gender gaps have narrowed. Approximately 95% of both boys and girls in the 7-10 age range are now enrolled in school, according to the recent ASER Report (2006). However, the quality of primary education in most parts of India leaves a lot to be desired and is in serious need of improvement. A recent survey found that a quarter of teachers in government schools were entirely absent from school during surprise visits, and only about half were in the classroom teaching (Kremer et al (2005)). Learning levels in government schools are also very low with 61% of the children rural India in grades 1-8 not being able to read at grade 1 level and over 80% not being able to read at grade 2 levels (ASER Report, 2006). The Andhra Pradesh Randomized Evaluation Study (APRESt) is a three-way partnership between the government of Andhra Pradesh, the Azim Premji Foundation (a leading nonprofit organization in India working to improve primary education outcomes) and the World Bank to pilot alternative policy options to improve rural primary education and rigorously measure their impact in rural areas of Andhra Pradesh. The first year of APRESt (2005 06) saw the implementation and evaluation of four different policy options to improve the quality of primary schooling in Andhra Pradesh, including performance pay for teachers based on group and individual performance, use of contract teachers, and provision of cash grants to schools (Muralidharan and Sundararaman (2006)). Following the completion of the first phase of APRESt, a Memorandum of Understanding (MoU) has been signed between the partners to continue to pilot and rigorously evaluate (using randomized allocation of programs) the most promising options in primary education policy over a period of 5 years under the APRESt. One of the proposed initiatives is to study the impact of providing disadvantaged and underserved children with opportunities for school choice that are similar to those available to other non-disadvantaged children. This would be done by offering scholarships that would allow them to shift to schools of their choice (if they wish to) in addition to the option of continuing in the existing government school. Such a program would provide opportunities for children from disadvantaged families to attend private 1
schools. The research study involves a rigorous evaluation of the impact of school choice both on children who receive the choice as well as on the aggregate impact on education outcomes for all children in villages where the school choice program is implemented. This is an important topic to understand, because private education has grown rapidly in India, with over 18% of children aged 7-16 in rural areas currently attending private schools (ASER Report, 2006). One possible reason for the expansion of private schools is the poor quality of education in government schools. Private schools generally have lower teacher salaries, much higher teacher accountability, significantly lower pupilteacher ratios, and higher student attendance and test scores than government schools (Muralidharan and Kremer, 2006). However, access for the disadvantaged is limited, since private schools charge fees. The Indian government is currently considering various policy options designed to increase access to private schools for weaker sections of the population. However, it is difficult to compare the performance of private and government schools since the better performance of children in private schools could potentially reflect family background or other student characteristics as well as what goes on in school. Thus, policies that hope to leverage the existence of private schools for universal quality education need to be designed on the basis of rigorous evidence regarding the relative performance of public and private schools. The Andhra Pradesh School Choice Project aims to provide such evidence. 2. School Choice Project: 2.1. Project Description The proposed school choice project would identify a set of 200 villages with existing private schools. Scholarship programs would be established in half of these villages (chosen by a random lottery). Approximately 40 scholarships will be made available in each village. Eligibility will be determined on the basis of current enrollment in a government school (and never having been enrolled in a private school in the past). Thus, children enrolled in grades 1-3 at the end of the school year will be eligible for the scholarship, and the choice over schools will be exercised in the subsequent year (in grades 2-4). A lottery will be used to allocate scholarships if there is excess demand 2
among eligible households. A child who receives a scholarship to attend a private school will continue to be eligible for the entire duration of primary schooling (till the end of grade 5), subject to meeting attendance requirements and participating in the end of year learning assessments. Thus, children in earlier grades at the time of launching the study will have a longer exposure to the program than those in later grades. The proposed sample size is based on the prior work done under APRESt which suggests that a sample of 100 scholarship villages will be sufficient to detect effects of reasonable magnitude. Each private school that participates in the project will be required to sign an agreement with the Foundation with regards to the terms for participation. This agreement will include details of how many scholarship places each private school is willing to offer at the given scholarship amount, the systems for verification of student attendance, protocols of provision of non-fee inputs to students (such as textbooks, notebooks, and other learning materials), and the procedures for payments to be made to the private schools. To summarize, the key design features of the scholarship/choice program are: The universe of children eligible for the scholarship will be limited to children currently in a government school and who have never enrolled in a private school o Identifies disadvantaged children without having to do means testing The number of scholarships to be offered in a village will be capped at either a fraction of the current capacity of all the private schools in the village or of the enrollment of the government schools o Don t want to empty out the government school Once a child receives a scholarship, he/she will continue to receive a scholarship till the end of primary school subject to meeting attendance requirements and taking the end of year tests Hope to avoid cream skimming problems by using a design analogous to those of many charter schools in the US o If a village has more than one private school, and if a school has more applicants than places, then the places in that school are allocated randomly, and the remaining students who get scholarships can go to the other private school or stay in the public school. 3
o Schools can choose to increase the number of spots for scholarship students (and accommodate all applicants), but as long as the number of applicants is more than the number of spots, the places will be allocated by lottery. No topping up scholarship amount will be set at around the 80th percentile of the private school fee distribution across all villages o Private school can determine the number of such places at this scholarship rate o But it has to accept all the children who are awarded the scholarship by lottery All expenses for books, uniforms, and school supplies are being covered by the scholarship. A transport subsidy may be provided in some cases, but may not be required if the choice is being exercised within the village. o The total scholarship spending per child (all inclusive) is expected to be around Rs. 3,200/child per year (USD 80/year). This is significantly less than the spending per child in the government schooling system (which is at least Rs. 4,000/year counting only pure variable costs, and over Rs. 5,000/year including various overhead costs) Fraud prevention on the basis of independent checks on student attendance. 2.2. Evaluation: A range of outcomes will be monitored during the life of the project both to accurately measure the impact of school choice on the recipients, and to allow the lessons from this program to feed into the policy process. Learning will be measured for children in all 200 villages via independently administered assessments that will be conducted by evaluators who will be hired, trained, and supervised by the full time project team of the Azim Premji Foundation. The project will also track various process variables, such as changes in number of teachers and their qualifications, school fees at private schools, attendance and activity of students and teachers, as well as classroom processes and pedagogy to see how the private schools respond to the program. Finally, the project will also track longer-term outcomes such as drop out rates, primary school completion rates, and rates of transition to post-primary education. The random allocations of scholarships through a lottery will allow the impact of the scholarships both on the individuals receiving the scholarships and on the larger community to be accurately measured. Since the scholarship is randomly assigned to a subset of the potential recipients, the remaining potential recipients provide a valid comparison group as there is no selection bias in who received the scholarship and who 4
did not. Similarly, the random selection of villages to receive the scholarship program provides a valid comparison group at the village level. Comparing the outcomes of children who received a scholarship to those who did not receive one in the same village can be used to estimate the impact of being able to exercise school choice and to look at how this varies with gender, family background, and other characteristics such as the baseline level of learning. The differential duration of exposure of different cohorts to private schooling will also allow estimation of the cumulative impact over several years of attending private schools on learning outcomes. Comparing the aggregate performance of all children in villages that received scholarships with the aggregate performance of all children in the villages that did not receive the scholarships will allow estimation of the total impact of the scholarship program, at the village level (including the impact on the remaining children in the government schools). A key question for the long-term viability of choice based school systems is to understand the bases by which parents choose schools for their children. Detailed data on baseline characteristics of families, schools, and distances will be collected and the take up rate will be modeled as a function of distance, socio-economic characteristics, as well as the gap in baseline scores, facilities, etc between the public and private schools in the village. The random increase in demand (due to the random choice of villages for scholarships) combined with the short run inelasticity of new school construction will shed light on the overall market for education in the area. At this point, the project does not envisage varying the information provided to parents who are receiving the scholarships but it might be possible to embed some variations on this dimension as well, which we expect to put in place after the first 1 or 2 years of the project. The findings from the evaluation will be relevant not only for education policy in Andhra Pradesh and India, but also in other countries as described below. 2.3. International Evidence on School Choice Programs Considerable evidence has been accumulated on the impact of scholarships on individual pupils; but the impact on the larger system, including that on students who originally would have been in public schools or would have been in private schools anyway, is unknown. School choice advocates argue that the increased competition will spur 5
performance improvements across all schools [Hoxby (2003)], while opponents argue that public schools will suffer as more motivated students leave for private schools [Hsieh and Urquiola (2001)]. Providing evidence on the aggregate impact of school choice will fill a key gap in the literature on school choice and allow policy makers to assess the case for expansion of the program. School choice programs have been implemented on a large scale in two developing countries, Chile and Colombia. Colombia s program offered vouchers to attend private secondary schools to more than 125,000 students from poor urban neighborhoods. In most communities, the demand for vouchers exceeded the supply, so voucher eligibility was determined by a lottery, generating a natural experiment. Angrist, Bettinger, Bloom, King, and Kremer (2002) find lottery winners were between 15 percent and 20 percent more likely to be in private schools, 10 percent more likely to complete eighth grade, and scored 0.2 standard deviations higher on standardized tests, equivalent to a full grade level. Using administrative records on registration and test scores on a centralized college entrance examination, Angrist, Bettinger, and Kremer (2004) find the lottery program increased secondary school completion rates by 15-20 percent. A key open question is the overall impact of vouchers. While Angrist et al.(2002) examine the effect of vouchers on participants in voucher programs, such programs may also affect children who do not participate and instead stay in public schools. One possibility is that these children may be adversely affected by the loss of their more motivated peers, while another possibility is competition from private schools might improve public schools, as argued by Hoxby (2003). Hsieh and Urquiola (2001) argue that higher private enrollment rates negatively affect the relative test scores, repetition rates, and socioeconomic status of students in public schools. They also find that higher private enrollment rates did not affect the average outcomes of municipalities. They interpret these results as evidence that Chile s voucher program merely increased sorting rather than adding value to education. To convincingly assess of the impact of voucher programs on non-participants requires randomization evaluations at the level of local communities, which could allow the estimation of the total program effect. This is the design that is proposed for the Andhra Pradesh School Choice Project and the research findings will therefore be relevant not only for school choice programs in India, but in other developing countries as well. 6
3. Project Implementation Roadmap Since careful communication at the household and school level is a critical prerequisite for the success of the project, it has therefore been decided that the project should be implemented in 2 phases. Phase 1 of the project will cover 40 villages in 1 district and will start in the school year 2007-08 and continue till March 2011 (by which time children who start receiving scholarships at the end of grade 1 will have completed grade 5). Phase 2 will expand coverage to the full sample of 200 villages and start in March 2008 across multiple districts and continue till March 2012. Since the operational details of implementing the scholarship project are critical; the smaller scale of Phase 1 will enable tight control on processes and creation of a blueprint for scaling up, before expanding to 200 villages under Phase 2. The school choice project will be managed within the framework of APRESt and the scholarships and evaluation are being funded by Legatum Global Development a private philanthropic organization who is the lead financial partner for the project. The participation of the Azim Premji Foundation, the Government of Andhra Pradesh, and the World Bank will be instrumental in ensuring efficient operation on the ground, proper accounting and disbursement policies, and in helping the results feed into the policy process through the institutionalization of APRESt in GOAP s education program. APRESt has established a successful track record in implementing and evaluating education policy options on a large scale. In the course of the work so far APRESt has developed a systematic outcome measuring infrastructure that employs 30 full time staff and nearly 300 part-time evaluators. In particular, the administration of the school choice project will be overseen by an experienced project manager who has overseen the first 2 years of APRESt. The evaluation of the project will be carried out with the support of a technical team comprising of Michael Kremer and Karthik Muralidharan of Harvard University, and Venkatesh Sundararaman of the World Bank. This technical team will be responsible for ensuring the scientific and technical validity of the evaluation. The World Bank will continue to lead discussions with the Government on the policy front and facilitate all interactions of the project team with the Government of Andhra Pradesh (and the Government of India as required), and be responsible for all policy dialogue with the center and state governments. 7
Appendix A: Phase 1 Timeline of first year activities By May 31, 2007 Identify the 40 villages for Phase 1 of the project Conduct learning assessments in all schools in 40 villages Household census of children in all schools in the 40 villages to receive scholarships and create the database of scholarship-eligible children, and collect basic demographic data on all the children in these 40 villages. o Invite applications from parents for potential awarding of a scholarship to attend a private school of their choice in the same village Discuss with the private schools regarding the possibility of an NGO like the Azim Premji Foundation offering scholarships to some government-school children to attend private schools. Determine the number of places that each private school can offer at the given fee. Determine the protocols for payment and verification Draft and sign terms of agreement between private school managements and the Foundation for participation in the scholarship program. June, 2007 Award scholarships (stratified random allocation) for the school year 2006-07 Communicate the news regarding selection for receiving scholarships to the selected households and let them know that they have the option of sending their children to (any of) the private schools in the village. Be available to answer questions regarding the process from the government school, private schools, and households June 2007 February 2008 Households choose schools, payments made to private schools based on choice of students to go there and independent verification of student attendance Tracking surveys to understand process variables in the schools in all 40 villages March April, 2008 Conduct end of year learning assessments in pilot villages March 2008 onwards Launch program across 200 villages in Phase 2 of the project. 8