Dial Mobile for Monitoring: Using Technology to Increase Transparency in Public Service Delivery. July 18, 2015

Transcription

1 Dial Mobile for Monitoring: Using Technology to Increase Transparency in Public Service Delivery Sisir Debnath Sheetal Sekhri July 18, 2015 This is a preliminary draft. Abstract This paper studies the role of technology in improving transparency and accountability in public service delivery. Using the roll-out of a technology enabled monitoring mechanism (the Interactive Voice Response System or the IVRS) in the Mid Day meal provision in Bihar, we find that a simple mechanism that aids in cross tallying the information provided by the middle tier of the delivery chain in welfare programs can reduce leakages and increase the efficacy of the programs. Using independently collected data by an NGO Pratham, we find that the technology enabled policy change increases the likelihood of meal provision in a school in Bihar by 21 percentage points, and the likelihood of meal being cooked on the day of surprise visit to school by 15 percentage points. These results are robust to a number of specifications which include matching based difference-indifference specifications, and control district specific trends. The increase in the take-up of the program by beneficiaries is also accompanied by an improvement in the quality and sufficiency of meals. Using central government commissioned audits data, we find an increase of 47 percentage points in fraction schools serving good quality of meals in Bihar schools post IVRS and an increase of 48 percentage points in fraction of schools serving sufficient quantity meals. In contrast, using state official records, we find that number of children availing MDM per school reduces post IVRS and percentage of schools providing meals is trending down. Surprisingly, the amount of rice consumed and cooking costs per school within district increase. Our results provide evidence that the IVRS resulted in reduction in leakage in the delivery system. Key Words: Welfare programs, Mid-Day meals, ICT, Leakages JEL Codes: D73, I38, O31, H53, H51 We would like to thank Mr R. Lakshmanan (former director of MDM Bihar) for generously giving us time to discuss the operational details of the IVRS program. We are also grateful to the directorate of Mid Day Meals in Bihar for sharing the IVRS data and answering our numerous queries and the staff of e-horizon Private Limited for technical help with the extraction of the data. Funding by International Growth centre (grant number 1-VCG- VINC-V ) is greatly appreciated. Anup Tiwari provided excellent research support. Indian School of Business, sisir debnath@isb.edu University of Virginia, ssekhri@virginia.edu 1

2 1 Introduction Social welfare programs in developing countries are infamous for poor administration and leakages from the distribution networks. Large swathes of benefits do not reach the intended beneficiaries (World Bank, 2003). In many countries, institutional capacity to implement safety net programs successfully is diminutive to the point of being deemed failing (Pritchett, 2009). Development outcomes tied to the public institutions that deliver them are undermined as a result of this failing. Yet, there is little empirical evidence uncovering what can improve the efficacy of public service delivery and importantly, reduce leakages in the delivery system. Finding low cost ways in which social program implementation can be improved remains an important constraint in improving development outcomes. Devolving the power of the government machinery by contracting delivery to private vendors (Galiani et al., 2005), involving NGOs in distribution (Bloom at el, 2006), encouraging community participation in monitoring (Björkman and Svensson, 2009) and incentivizing public delivery agents (Debnath, 2014) have been demonstrated as effective strategies in certain contexts. But these approaches are not policy-driven mechanisms that strengthen the public institutions so delivery can be improved across a broad range of services. In this study, we focus on a state driven technology enabled reform and show that information and communication technologies (ICT)can be harnessed to improve transparency and accountability in public delivery systems. We demonstrate that technology enabled improvements in delivery system enhance the efficacy of the system leading to significant increase in the delivery to the beneficiaries. Our paper uses the statewide roll-out of Interactive Voice Response System (IVRS) introduced by the state of Bihar in India to improve the functioning of the Mid Day Meal Program to establish that ICT technologies can improve service delivery in developing countries. Even though India is one of the top three global producers for wheat, rice, and pulses, the incidence of malnutrition among children in India is very high. By one estimate, India accounted for 40 percent of malnourished children in the world (von Braun, Ruel, and Gulati, 2008). In order to combat mal-nutrition, the Indian government launched the National Program of Nutritional Support to Primary Education (popularly known as the Mid-day Meal or MDM Scheme) in The scheme entitles each enrolled child to a meal on the school premises each school day. The program currently benefits 120 million primary school children across the country making it one of the largest school feeding programs in the world (Ministry of Human Resource Development, India). However, this program is fraught with corruption and inefficiencies. 1 Improving the effectiveness of this program can potentially induce better nutritional and educational outcomes for children. 2 A key challenge faced by the top tier administration is that they have to rely on the take-up estimates provided by the middle level delivery machin- 1 See for example 2 Figlio and Winicki (2005) show that improving accountability in US schools leads to better nutritional outcomes. But this study does not address how best to improve accountability. 2

3 ery to determine the future allocations and performance of the program. This leaves room for rigging the statistics, siphoning off from the system, and inaction or shirking by the middle tier monitors of the system. As a result, a first order policy goal is to ascertain how to improve information flow from the grass root level to the top tier. To this end, in 2012 the state of Bihar side lined take-up information provision by middle tier of the delivery machinery and introduced a fully automated platform that calls the school teachers in every school every day to record whether the meal was provided by the school. There are two ways in which this system can curtail leakages and improve the efficacy of the delivery system. First, the IVRS provides a way to cross tally information provided by the middle tier of the distribution system. Second, since beneficiary take-up information is available at much disaggregated level under the new system, it is possible to identify the schools that are not providing meals consistently. The middle tier bureaucrats are then held accountable for the performance of the school there by inducing them to increase their effort to implement the MDM effectively. We use state government reports for MDM and independent assessments of MDM by central government and an independent NGO that monitors schooling outcomes in India every year to analyze the impact of this policy reform. We utilize the introduction of the program in 2012 in Bihar and compare the outcomes in districts of Bihar to comparable districts in its neighboring states before and after the introduction of the program. We find evidence of malfeasance in reporting. As per the government data, before the program the number of children availing the MDM per school was trending upwards in Bihar and the increase was statistically significant. However, after the program there was a substantial significant decline. This event study evidence is mirrored in our Difference-in-Difference estimates using district and time fixed effects, where we find a decline in the number of children availing MDMs per school post IVRS in Bihar. Analogously, the percent of schools providing MDM also declined. In Bihar, we find over reporting of the number of schools prior to the program. In independent assessment of the program where we employ Annual Status of Education Report (ASER) data collected by Pratham, a national NGO of repute, we find significant improvement in delivery. ASER team visits schools in a district randomly every year and assesses a number of schooling outcomes. Using this data, we find improvements in the percentage of schools providing MDM, the likelihood of whether MDM was cooked on the day of surprise visit, and the likelihood of evidence that MDM was observed in schools in Bihar over time. Our DID estimates echo these event study results and show that the percentage of schools providing MDM increased by 21 percentage points, the likelihood that MDM was cooked by 15 percentage points and the likelihood that evidence of MDM being cooked was observed by 22 percentage points all significant at conventional levels of significance. These results are robust to a number of alternate specifications including controls for school characteristics, district specific trends, and a generalized DID where we match districts based on observables before implementing the DID estimates. 3

4 Although government estimates show a decline in number of children availing meals per school, surprisingly, amount of rice consumed and cooking cost which includes cost of other materials added to meals increased. The delivery system that delivers MDM is also responsible for providing Vitamin A and deworming tablets in schools and periodically conducting health check-ups in schools. Although these services were not targeted by IVRS, we find evidence that provision of these services also improved after the IVRS introduction in Bihar over time. In addition, in our DID specifications, we find large and economically significant effects on these outcomes. 3 Finally, we examine the effects on quality of meals using independent assessment conducted by the central government who routinely audits the schools for quality. We find that quality of good meals went up dramatically and the quality of bad meals dropped commensurately. Also, provision of sufficient quantity of meals went up and insufficient quantity meals declined in our DID specifications. Using school level administrative data and IVRS data, we find that the improvements accrue due to changes in the middle tier machinery s operations. Using school and year fixed effects, we compare changes in MDM provision by school before and after the policy change in Bihar employing (i) annual school administrative data alone for pre- and post- policy change periods, and (ii) using administrative data for pre-period and IVRS for post period. Since, head masters reporting in these specifications is no different in the post policy period, we interpret this as evidence that head masters do not change behavior post IVRS. We also geo-referenced the administrative data of approximately 70,000 schools in Bihar and examined if the improvements in Bihar accrued in schools closer to the district head quarters and do not find evidence of such heterogeneity. This further casts doubt on changes in head masters behavior. Our paper extends two strands of literature. The first strand has focussed on establishing low cost monitoring or other mechanisms to improve the provision of public service delivery. Technology based monitoring by beneficiaries coupled with non-linear incentives has been demonstrated in inducing agents to exert effort (Duflo et al., 2012), but this mechanism may not work so well where agents can also be co-opted to steal from the system. Encouraging communities to hold service providers more accountable may improve accountability in public service provision (Björkman and Svensson, 2009). Informal networks may facilitate monitoring and enforcement (Nagavarapu and Sekhri, 2015). But when large swathes of data and information are required to facilitate such empowerment, ICT technologies can reduce costs and be useful. More importantly, we show that ICT technologies can also reduce the level of bureaucratic involvement in information flows within the systems, and thus reduce possibilities of malfeasance and shirking. Another salient contribution of our paper is to isolate a technology based policy change that can increase state capacity in delivering services. Using an experiment, Muralidharan et al (2014) show that payment infrastructures can be improved using biometric 3 In contrast, when health services were contacted out to NGOs in Cambodia, only those outcomes improved which were targeted (Bloom et al, 2006). 4

5 payment cards. Complementing this new evidence, our paper shows that technology can be used to design simple mechanisms that can increase state capacity to monitor agents in welfare delivery programs so that agency problems can be curtailed. Our paper also extends the burgeoning literature examining the effects of ICT enabled platforms on development outcomes. ICT enabled technologies have been demonstrated to improve producer surplus and reduce price dispersion in markets in India (Jensen, 2007; Goyal, 2010). Mobile phones have been demonstrated to increase access to credit, financial services like banking, information, and have served as reminders in health care provision. 4 However, role of ICT technologies in improving transparency and public service delivery is not well established. Our study addresses this gap. Rest of the paper is organized as follows: Section 2 provides a background highlighting the features of the MDM program and the IVRS reform in Bihar. Section 3 discusses our data sources. In section 4, we provide a discussion of our estimation strategy. Section 6 discusses the results, and Section 7 concludes. 2 Background 2.1 MDM Program and Subsidies The National Program of Nutritional Support to Primary Education, commonly known as the Mid-Day Meal Program, was launched in India in The program entitled each school going child with a free meal each school day. Initially the program was introduced in 2,408 blocks but was extended nationwide in 1997, to cover all primary-school (grade 1 through 5) going children in government and local body public schools. The original program provided 100 grams of take home food grains per child per day including summer breaks. In September of 2004, the program transitioned from raw grains to cooked meals, that provided a minimum of 300 calories and 8-12 grams of protein per child. In 2006 the provisions under the program were changed again requiring 450 calories and 12 grams of protein per child per day with special stipulations to provide iron, folic acid, and other essential micronutrient. The 2006 revision also entailed subsidies to states for cooking and preparation costs. 5 The coverage of the program now includes upper primary classes in Government and Government aided schools. The meals are supposed to provide 700 calories and 20 grams of protein for children in upper primary grades. Under the program the central government provides free grains, while state governments are responsible for financing cooking costs. States that were unable to finance the costs were 4 Aker and Mbiti (2010) provide an excellent overview. 5 These subsidies consisted of 1.80 rupees per child per school day for States in the North Eastern Region, conditional on the state governments contributing 0.20 rupees per child per school day, and 1.50 rupees per child per school day for all other states, conditional on these other state governments contributing 0.50 rupees per school day. 5

6 initially allowed to distribute raw grain conditional on attendance. However, a Supreme Court judgement in 2001 mandated that all public schools must provide cooked meals within six months. In order to comply with this, the state provides subsidies to the schools in forms of financial grants. These financial grants, given to schools to provide meals, can be broadly classified as recurring and non-recurring grants. Recurring grants include cooking costs and salaries for the cooks and helpers. While non-recurring grants include infrastructure costs such as kitchen devices and kitchen sheds. These grants vary by school and are based on school level consumption. Schools estimate and report their annual requirement based on the number of students consuming meals and the number of working days. These data are aggregated at the block, district, and state levels. Based on these aggregated reports, state governments prepare an Annual Work Plan and Budget (AWP&B) and submit them to the Project Approval Board (PAB) of the Ministry of Human Resource Development (MHRD) for review and approval. Actual allocations and release of funds are based on the PAB approvals. 2.2 Monitoring and Allocation Disbursal This program is extensively monitored by states and the central government authorities. Some of the monitoring mechanisms for the implementation of the program are uniform across the states. For example the Government of India (GOI) has empaneled several Monitoring Institutes that are responsible to assess the progress and quality of the midday meal program at the district level. These institutes are required to cover 5 percent of total schools that serve midday meals in a period of two years. States submit their Quarterly Progress Reports (QPR) to the GOI that is used to assess performance. States may also have their own monitoring systems. 6 Under the system of manual monitoring, which was prevalent in India in 2012, schools submitted monthly reports to the Block Resource Person (BRP), a government functionary responsible for assisting the implementation of the mid day meal (MDM) at the block level. These reports measure the performance of the MDM provision and report the number of beneficiaries, details of meals served, and the status of funds and food grains supplied. Monthly requisitions for funds were collected by the BRPs from schools and were submitted to district offices. Based on school specific requirements, cheques were prepared manually. This entire process involved to man days. School head masters deposited the cheques in bank accounts after receiving them from the BRPs. The process of requisitioning funds and crediting it to the schools account, took somewhere between 2 to 3 months. The monthly progress reports were not available and quarterly progress reports were compiled with a delay. Capability to take preemptive actions was also limited as monthly reports allowed only post facto analysis and could not adequately predict upcoming supply chain bottlenecks. BRPs were also required to inspect certain number of schools randomly each month 6 For example the state of Orissa has opened a control room and installed toll free telephones for students as a measure for grievance redressal. To promote compliance, all schools are also required to publicly display information on the meals and to expect periodic visits by State Government officials. 6

7 in order to audit the school records. However, there was no overall system for tracking number of inspections, inspections with adverse remarks that required additional action, and the status of any follow-up actions. Under manual reporting, late submission of reports by schools was typically the norm. These delays in turn led to shortage of funds and foodgrain. There was also a significant scope for siphoning both funds and food grains from this system. Data provided by the intermediary officials on the number of requisitions submitted and processed, student attendance, number of beneficiaries, number of meals served, and infrastructure provided (such as kitchen devices) were routinely inflated and reported inaccurately as there were no means to authenticate the data. 2.3 IVRS Reform The state government used the IVRS system in several ways to improve the provision of MDM in the state. First they hired district coordinators to monitor the implementation of the MDM program using the IVRS data. They are are hired on a contract basis and given an unique username and password. Coordinators periodically visit schools Bihar streamlined the requisitions and allocations by making use of an Management Information System (MIS) for the requisition and transfer of funds and food grains. The middle tier machinery of the system was still the main conduit of beneficiary take-up data to the top tier adminstration. Thus to better assess and monitor school level provision of meals on real time basis, Bihar MDM authority implemented Dopahar, an Interactive Voice Response System (IVRS) that collected real time data directly from the head masters. Prior to this system, monitoring authorities had to rely on complaints by the public and audit inspections by BRPs to asses MDM performance. Although, audit inspections are supposed to cover roughly 8-9 percent of all schools, these are not conducted frequently as BRPs are severely time constrained owing to the large number of other duties. Also, it was easy to siphon off from the distribution system. As a result, figures of expenditure, beneficiaries served, number of days the school was open, and other performance indicators were not very accurate or reliable and often were inflated. IVRS was a simple technology enabled mechanism that would allow to cross tally the reports of the BRPs. Under the IVRS, each school had to register 5 points of contact (cellular numbers) including one head master, 2 teachers, and 2 para-teachers. The IVRS calls any one of the five teachers in each school at random and collects data on the numbers of meals served in each school every day. If no meals were served the teacher is supposed to press 0 and provide reason for the same. All the responses are categorical and pre-coded for the ease of data collection through mobile phones. In 2012, there were about 70,000 schools in Bihar that were supposed to serve meals to students. Table A.1 shows the cell phone penetration of the head masters collected from the Dopahar records. Over 99 percent of the head masters had 7

8 a mobile phone and were contacted every day. After completing the calls to all of the schools, the IVRS system summarizes the data and generates reports at the district, block, village, and school level on attributes such as number of attendees, meals served, adherence to menu etc. The district level reports are ed and sent to the District Magistrate as an SMS, while the block level reports are sent to the Block Education Officer (BEO). The mid-day meal department hires District Coordinators (DC) on a contract basis to monitor the implementation of the MDM program. They are given a unique username and password to log into the Dopahar website and monitor the implementation of the program. The system counts the number of times these coordinators log into the system during a particular period. These coordinators visit schools and check the veracity of the data. Later several add on features were added to the IVRS to increase the efficiency of data collection. 7 Since IVRS collected data from all schools it was possible to identify the schools that are not providing meals consistently. The middle tier bureaucrats could be held accountable for the performance of the school inducing them to exert more effort. Dopahar data is also used to crosscheck the MIS data. It, therefore, acts as a supplement to existing monitoring systems. IVRS data is shared freely with the public through a web-interface. Thus making the headmaster s report open to public scrutiny. Complaints or grievances from the public are recorded and followed up. 3 Data We use three primary sources of data for carrying out our analysis. The independent assessment is based on the Annual Status of Education Report (ASER), carried out by an NGO Pratham to primarily collect information about the educational achievement of primary and upper primary school children in every rural district in India. Each year the survey roughly covers 570 districts, 15,000 villages, 15,000 government schools, 300,000 households and 700,000 children between We use ASER survey data to estimate the effects of the IVRS on provision of Midday meals. These surveys are available from 2005 onward but the survey instruments are uniform and variables of interest are readily comparable for the period Therefore, we restrict our estimation sample to this period. Furthermore, we restrict our analysis to five states in India which are similar to the state of Bihar in terms of many socio-economic aspects and geographical proximity. Our sample includes: Bihar, Chhattisgarh, Jharkhand, Orissa, and Madhya Pradesh. 8 The estimation sample is a representative repeated cross section at the school and household level. 7 These included miss call facility and increased call response time. If a headmaster could not receive the call from IVRS she could received it at a convenient time by dialing a particular number. 8 Uttar Pradesh is also comparable and is an immediate neighbor. We do not include Uttar Pradesh in our sample as it introduced IVRS in However, we do not have access to the government records for the state.as a result, we are not able to include it in our sample. 8

9 For each village surveyed under ASER, one government school (if any) is inspected randomly. For each of the inspected schools between , ASER collected information on three vital variables: whether MDM was served in a school on the date of interview, whether the interviewer found any evidence of meals being served to the students on that day such as used utensils and meals bought from outside, and whether the interviewer found food being cooked in the school premises. Other than these observations on MDM, ASER also collects information on physical school infrastructure such as source of drinking water, provision of toilets, whether the school has a boundary wall; and questions regarding teaching staff such as the number of teacher appointed and the number of teacher present on the day of interview. The sample includes 5041 schools for Bihar and schools for the entire sample of five states. Our second dataset comes from government records. We use district level Annual Work Plan Budgets (AWPB) that are submitted by the state midday meal authorities to the Government of India for review of their performance and approval of their budget for the period Each state has a district wise annual target of number of children that they want to serve MDM. In addition to these targets, these reports include total number of schools serving midday meals, children availing meals, total amount of rice consumed, total amount of cooking cost, expenditure on cook s wages. The MDM distribution channel is also responsible for school health program that distributes vitamins and deworming tablets in schools. This AWPB data also provides total number of schools and children under school health programme, total number of schools where vitamin A and deworming tablets were distributed, and total number of children who consumed vitamin A and deworming tablets. These data are provided for both primary and upper primary schools at district level. Our Bihar data has 38 districts and 189 district-year observations. Our event study analysis are based on this sample. Overall, there are 157 districts and 785 district-year observations in our sample of five states. For the health care program, we have 152 district-year observations for Bihar and 632 in the overall sample. In order to shed light on the quality of meals provided, we make use of data collected by the central government s independent auditing of the state MDM programs. The Government of India has appointed several independent institutes to monitor the government and aided schools. Each district is assigned to one of the empaneled monitoring institutes and within a period of two years, 5 percent of the elementary schools in the district are inspected by them using surprise visits. Most of these institutes are headed by tenured professors at state universities. Post monitoring the institutes submit half yearly reports to the authorities. Apart from inspecting the daily operations of a schools, these reports assess the quality and quantity of midday meals on the day of their visit. These assessments are qualitative and the reports publish the number of schools where the quality and quantity of the meals are found to be good, satisfactory, or bad. We use these reports to create a panel dataset that entails information about fraction of the inspected schools that serve good and bad quality meals, and sufficient and insufficient quantity meals. 9 We have 180 district-year observations in this sample. 9 These reports are available at While it is possible that these data are not accurate 9

10 As described previously, the IVRS system in Bihar introduced in year 2012 collects data on the numbers of meals served for every school day using a technology enabled reporting system. We use this data in conjunction with another administrative data discussed below to asses the discrepancy between reporting of midday meal status by the school headmaster to the local authorities vis-a-vis IVRS. 10 This data consist of information at a monthly level on number of days the system attempted to call the headmaster, number of days calls were actually received by the headmaster, attendance and number of student availing mid day meal. Using this monthly data for the period between April 2012 to November 2014 we calculate status of midday meal at the school level for the academic years 2012 to 2014, which is a binary variable taking the value one if any student availed midday meal in a school in a given academic year and zero otherwise. Finally, we also use school level administrative data from the District System for Education (DISE) reports for the states of Bihar, Jharkhand, Chhattisgarh,Orissa and Madhya Pradesh. DISE is published annually by the National University of Education Planning and Education (NUEPA). Each year the report roughly covers 662 districts, 1.4 million schools, million students, and 7.35 millions teachers. We use this data for the period 2009 to 2013 to cross tally the status of midday meals reported by the headmasters to the IVRS and to the DISE authorities. Like with IVRS data, the status of midday meals is a binary variable taking the value one if meals are served in a school in a given academic year and zero otherwise. 4 Estimation In order to assess the effect of the IVRS on provision of midday meals, we estimate two types of empirical models. First, we use a event study framework to investigate the effects of the program on outcomes in Bihar before and after the introduction of the IVRS examining how the outcomes trend over years, and whether changes in outcomes occur precisely after the policy change. The three outcomes are: percent of schools serving meals, average number of children availing meals, and a measure of performance defined as the ratio of actual meals served against the target. The empirical model is specified as: y dt = α k 1 α k IVRS operational for k periods + α X X dt + δ d + ε dt, (1) where y dt is an outcome variable measuring provision of meals in district d in year t. IVRS operational for k periods is an indicator variable equal to one if IVRS is in effect since k years and zero otherwise. A negative value of k implies it is a pre-ivrs period. X dt is a vector due to co-opting of monitors, these data were used to flag non-compliance and quality issues to Bihar government in 2010 preceding an accident where due to poor storage, school food rations got mixed with fertilizers and caused the death of several students who consumed the food. 10 IVRS data is obtained from e-horizon Private Limited, the company responsible to maintain the IVRS technology in Bihar. 10

11 of schools characteristics at the district level. 11 The specification also includes district fixed effects (δ d ). We estimate equation 2 after restricting data to the state of Bihar. The parameters of interest are α k s. Second, in order to allay the concern that secular trends over time may be responsible for the changes that we are attributing to the IVRS, we estimate a difference-in-difference model. We compare the effects of the IVRS on meals provision across the districts of Bihar and control states before and after the policy change. 12 The empirical model is as follows: y dt = β k 1 β k Bihar IVRS operational for k periods + β X X dt + δ d + ε dt, (2) While other control variables remain the same as in the event study, here we interact the indicator for IVRS operational for k years with an indicator for treatment which is equal to 1 for Bihar and 0 for the other control states. Thus, the parameter of interest, β k is the treatment on treated DID estiamte which shows the effect of IVRS on outcomes in districts of Bihar after the program has been implemented. Any secular unobserved changes in outcomes are controlled for under the standard difference-in-difference identifying assumption. Our other auxiliary outcome variables, for which we estimate analogous empirical models, are rice consumed, cooking costs, cook s wages, health checkups, distribution of vitamin A, and deworming tablets. 5 Results 5.1 Main Results We show our main results using three different datasets. The first two aid in understanding if there are any discrepancies in the assessment of beneficiary take-up based on state s official and independently collected data. The third sample is used to determine the impact on quality metrics based on independent assessments conducted by institutes engaged by the central government Independent Data Based Assessment In Table 2, we use school level ASER data for Bihar with district fixed effects to conduct an event study analysis. In column 1, we estimate whether a school provides MDM or not. In the pre-policy years, this estimate is negligible and statistically insignificant. In 2012, there is a 17 percentage point increase in the likelihood of a school serving MDM. This effect is statistically 11 School level characteristics include fraction of schools with separate girls toilet, drinking water, playground; number of head teachers, total appointed teachers. 12 Districts are the administrative unit under the state. Our control states are in close proximity of Bihar and have a comparable poverty profile. 11

12 significant at the 1 percent significance level and persists in the first post policy year The pre-post comparison indicates an improvement of 18 percentage points. On a baseline average of 0.562, this is a 9.5 percent increase. This is robust to inclusion of school level controls reported in column In column 3, we examine the effect on whether MDM was cooked on the day of the surprise visit by the survey team. Prior to the policy change, this estimate was also close to 0. This jumps to 0.19 in 2012 and and persists at 14 percentage points in 2013 both significant at the 1 percent significance level. Comparison of pre-post coefficients indicate an improvement of 12 percentage points. This is an 6 percent increase over a base of Again, the estimates are robust to including school level controls in column 4. An analogous estimations for whether the team observed evidence of MDM served in the school on the day of the visit is presented in columns 5 and 6 without and with school level controls respectively. The results echo the previous findings. The estimates jump from nearly 0 in 2011 to around 0.20 in 2012 and 0.14 in 2013 significant at the 1 percent significance level. The pre-post comparison reveals an improvement of 10 percentage points. This is a 4 percent increase over a base of Overall, we find consistent evidence that MDM provision in treatment schools improved after the policy was implemented. 14 Next, we focus on our DID estimation and report the results in Table 3. Here, our sample includes 19,771 schools. 15 Our DID estimates are remarkably similar to the event study estimates reported in Table 2. For the first outcome whether a school provides MDM or not, the estimates in column 1 indicate interaction coefficients of 0 prior to the policy change, and an 18 percentage points increase in the likelihood of serving MDM in 2012 in Bihar districts, which persists at 16 percentage points in Both these estimates are statistically significant at the 1 percent level. Results are robust to inclusion of school level controls in column 2. A pre-post difference in coefficients of 23 percentage points, on a baseline average of.562, indicates an improvement of 13 percent in the likelihood of provision of MDM in schools in Bihar. Similarly, we find evidence of an increase in the likelihood of MDM being cooked on the day of the surprise visit by the survey team in the school in districts of Bihar. Point estimates in columns 4 reveal that the interaction coefficient for 2011 is negative -0.08, statistically significant at the 10 percent significance level. Then it changes sign and is estimated to be 0.16, significant at the 1 percent significance level in Subsequently, it is 0.14 and statistically significant at the 1 percent significance level in A comparison of the interaction effects for the pre and post IVRS years indicate that the likelihood of MDM being cooked on the day of inspection increased by 20 percentage points. On a baseline of.503, this is a 10 percent increase. For whether the team observed evidence of MDM served in the school on the day of the 13 These controls include indicators for blackboards in grade 2, drinking water facilities, toilets for girls and boys, and school type fixed effects. 14 The number of observations is different in columns 1,3, and 5 because of missing values of outcome variables and further reduced in columns 2, 4, and 6 due to missing values of control variables in the data. 15 The data is a repeated cross section of schools in a district level panel data. 12

13 visit, as seen in columns 5 and 6, there is a negative and statistically significant trend in Bihar prior to the policy change. The 2010 interaction coefficients are and in columns 5 and 6 respectively (both significant at the 1 percent significance level), and the 2011 are and -0.14, significant at conventional levels of significance. There is a change in this trend in 2012, and the interaction coefficients are now positive 0.11 and 0.08 in columns 5 and 6, significant at the 1 and 10 percent significance levels respectively. The pre-post comparison of the interaction coefficients corroborates the findings in the event study analysis reported earlier Official (State Government) Data Based Assessment We present the results of our event study, where we compare outcomes related to MDM provision before and after the policy change in Bihar, in Table 4. In column 1, the number of children availing MDM per school for primary schools was trending upwards in a statistically significant manner in 2010 and 2011 relative to In 2011, one year before the policy change, there was an increase of 57 children availing MDM per school and the estimate is statistically significant at 1 percent significance level.in 2012, the year IVRS was introduced, this was still positive but statistically indistinguishable from 0. In the first post year, this dropped precipitously to 74 fewer children per school availing MDM. The estimate is significant at the 1 percent significance level. A comparison of the pre-post coefficients reveals that on average 130 fewer children were availing meals per school one year after the IVRS was introduced. On a baseline of 353 children per school this translates to a decline of 37 percentage points. In upper primary schools (column 2), the estimates for 2010 to 2012 are small and statistically insignificant. However, in the first year after the policy change, almost 57 fewer children availed MDM per school. On a baseline of 177 children per upper primary school this was a decline of 28.5 percent. In rest of columns we turn to examining what happens to other inputs such as rice and cooking costs which include costs of other materials used in the meal preparation. Results are provided separately for primary and upper primary schools. For primary schools in column 3, rice consumed in 2010 relative to 2009 was higher but imprecisely measured. In 2011, the estimate was statistically indistinguishable from 0. In 2012, it was 289 but statistically insignificant. This jumped to 1182 in 2013, a year after the policy change, and is now statistically significant at 1 percent significance level. For upper primary schools, this estimate is small and statistically insignificant in the pre years and jumps to 460, significant at the 5 percent significance level. It dramatically increases to 1284, significant at 1 percent significance level in the same year when the number of children availing MDM per school decline. A prepost comparison of these coefficients indicate that consumption of rice increased by 1258 and 1180 metric tonnes at primary and upper primary schools, significant at 1 percent level. Given the baseline average consumption levels this was an increase by 52 and 117 percent. Cooking costs show a similar pattern. For primary schools, in column 5, one year before the policy 13

14 change the coefficient was 8.04 statistically indistinguishable from 0. One year after, in 2013 it jumped to 61.4, statistically significant at the 1 percent significance level. For upper primary schools, in column 6, the estimates are similar. In Table 5, we document the results for the DID analysis in which we use as controls, districts from 4 states from the same geographical area that have comparable poverty profile as Bihar. In column 1, we see that relative to 2009, there was a statistically significant increase of 61 in the number of children availing the MDM per school in 2011, a year before the policy change. This estimate dropped to 30 in 2012 but was still positive and statistically significant at the 5 percent significance level. In the first post year in 2013, the number dropped to 68 fewer children availing MDM per school. The coefficient is significant at the 1 percent significance level. The difference-in-difference estimate between 2011 and 2013 is 129 fewer children reported at bottom, and is significant at the one significance level. In column 2, for upper primary schools, the estimate for 2011 is 2 and statistically insignificant. This drops to in 2012 but again statistically indistinguishable from 0. In 2013, 52 fewer children availed MDM per school, significant at 1 percent significance level. The difference-in-difference estimate between 2011 and 2013 is suggests that 54 fewer students availed MDM in Bihar, and it is precisely estimated. The rest of the columns in Table 5 show the DID estimates for other inputs used to prepare meals. The results mirror the event study evidence reported earlier. The interaction coefficient for 2011 is for primary schools and statistically insignificant. This jumps to 670 for 2012, and for 2013 both significant at the 1 percent significance level. The analogous interaction coefficients for upper primary schools are in 2011 (statistically insignificant) and in 2012 and 1082 in 2013 both significant at conventional levels of significance. The coefficients for cooking costs increase from statistically insignificant in 2011 to 19.3 in 2012 and 53 in 2013 both significant at the 1 percent significance level. In upper primary schools, the estimated interaction coefficient is -14 in 2011, statistically significant at 1 percent level. It dropped to in 2012 but not distinguishable from 0, and jumped to 22.7, statistically significant at 1 percent significance level. The difference in the pre-post interactions are similar to the estimated differences in the event study analysis and they are statistically significant at 1 percent level Improvements in Other Services Using the Official Data The MDM machinery is also responsible for running another health program ongoing since 2010, where health check-ups are done for children in school, and vitamin A and deworming tablets are distributed for consumption by the school going children. IVRS did not directly target these services. By examining the impact of IVRS on the delivery of these services, we want to shed light on whether the overall system efficacy improved. In Table 6, we report the results for health check-ups, vitamin A take-up, and de-worming 14

15 take-up with 2010 as reference year. Specifications in columns 1, 3, and 5 control for total number of schools within each district, and 2, 4, and 6 control for enrollment in primary schools. Column 1 shows that number of schools providing health check-ups dramatically improved from 93.7 in 2011 to 1113 in Number of children who were provided health check-ups increased from to Number of schools distributing vitamin A went from (statistically significant at the 5 percent significance level) to a positive 218.2, significant at the 1 percent significance level. Number of children receiving vitamin A went from -488 in 2011 to The latter estimate is statistically significant at the 1 percent significance level. The number of schools providing deworming tablets increased from 134 to 454 and and number of children receiving deworming tablets went from in 2011 to in Our DID estimates are reported in Table 7. The interaction coefficient on number of schools providing health check-ups went up from a statistically insignificant in 2011 to in 2013 significant at the 5 percent significance level. The number of children who were provided health check-ups also shows such patterns although estimates are imprecisely estimated (column 2). The estimate for the interaction coefficient for children receiving vitamin A in column 3, is (statistically insignificant) in 2011 relative to 2010, but changes to in 2013, significant at the 10 percent significance level. In case of deworming tablets, the interaction coefficient for the number of schools for 2011 is statistically indistinguishable from 0 but jumps to 540 for 2013 (column 4), significant at the 5 percent significance level. The interaction coefficient for number of children (reported in column 5) receiving deworming tablet went from (statistically insignificant) to , significant at the 5 percent significance level Central Government Commissioned Audits Data Based Quality Assessment The quality audit data contains an unbalanced panel of districts from 2010 to 2013 for the states in our sample and has a total of 180 district year observations. Hence, we are not able to estimate an event study empirical model due to the small number of observations per year. We show our condensed DID estimates in Table 8. Fraction of schools among the audited schools that served good quality meals increased post IVRS policy change increased in Bihar relative to other states. The point estimate is 0.47 implying a 47 percentage points improvement in Bihar (column 1). The coefficient is significant at the 1 percent significance level. This is commensurate with a decline in bad quality meals. 16 In column 2, bad quality meals declined by 67 percentage points. In column 3, we report the estimate for fraction of audited schools providing sufficient quantity of meals. This went up by 48 percentage points, statistically significant at the 5 percent significance level. The fraction providing insufficient meals declined by 60 percentage points. The interaction coefficient is significant at the 1 percent significance level. 16 Since these measures in the reports are conducted from qualitatively assessments, both good and bad quality schools are reported. 15

16 5.2 Selection and Endogeneity Concerns One of the main empirical concerns might be that the districts of Bihar are trending differently and the results are confounded by these trends. We allay this concern for our independent assessment based results shown in Table 3 in two ways. First, we include district specific trends in our specifications. On inclusion of district specific trends, the results remain remarkably similar. Second, we match districts on observable school characteristics and estimate a generalized DID model on this sample. We provide results in Table 9. In the first panel, we show our baseline specification results condensing years into pre (2009, 2010, and 2011) and post (2012, 2013) policy change. We include a state specific trend in this specification in addition to the district and year fixed effects. Standard errors are clustered at the district level. Additionally, we control for average district school characteristics. 17 We find a 23 percentage points increase in the likelihood that the schools in the treated areas serve MDM post treatment (column 1), 29 percentage points increase in the likelihood that MDM was cooked on the day of the surprise visit (column 2), and 33 percentage points increase in the likelihood that the enumerators saw evidence of MDM being cooked in the school on the day of the visit (column 3). All three estimates are statistically significant at the 1 percent significance level. In the Panel B, we include the district specific trends. The results remain remarkably similar to the estimates reported in Panel A. In addition, we match the districts using propensity scores which we calculate using the above mentioned control variable. We trim the observations which are outside the common support of the propensity score distribution. Figure A.1 shows the relative distributions for treated (Bihar) districts and the control (other states) districts and highlights the common support. In the third panel, we restrict the sample to this common support and estimate a DID model. 18 Our results, shown in the third panel, are very similar to the results documented in the first panel. In the last panel, we use a generalized DID method proposed by Heckman (reference) to estimate the treatment effect. We use a kernel based matching algorithm and employ a gaussian kernel for the procedure. We present bootstrap standard errors. Our previous results are confirmed using this specification as well. 6 Underlying Mechanisms and Alternate Hypothesis Our hypothesis is that there is less leakage from the MDM delivery machinery because the intermediate tier of the delivery system now has incentives to siphon off and shirk less. here, we investigate if the school headmasters change their reporting behavior as well. We rely on the 17 We include district level school infrastructure such as fraction of schools in a district with tap or hand-pump for drinking water, common toilet, separate toilet for boys and girls, and boundary wall; and average number of appointed teachers, teachers present during the survey as controls. 18 We show covariate balance in Table A.2. 16

17 fact that the head masters and the staff report the MDM provision to two different sources after the policy change but not necessarily at the same time. IVRS collects data on MDM provision every day from a randomly chosen teacher among the 5 designated teachers, and the school headmasters also manually report MDM take-up to the DISE authorities every year. We compare the effect of the policy change on MDM provision in Bihar pre and post policy change using the DISE data controlling for school and year fixed effects. Results are reported in the top panel of Table 10. There is an 11 percentage points increase in the likelihood that schools provides MDM in this sample and this is statistically significant at the 1 percent significance level (column 1). In column 2, we compare pre and post school level MDM provision in Bihar, except here we use school specific IVRS data for the post period. The effect is identical to what we reported in column 1. We infer from this that the school head masters do not change their reporting pattern after the policy change. 19 We carry out an additional test to shed light on this question. If headmasters expect more vigilance and monitoring after the policy change consonantly, change their reporting then the reporting chnage might be more pronounced in schools closer to district head quarters as these are relatively easy to physically audit. We geo-coded all 70,000 schools in Bihar and calculated their distance from the district headquarters. We then interact our post indicator with this distance measure. Bottom panel of Table 10 shows the results for the interaction effects for both the specifications used in the top panel described above. The double interaction with distance is negligible and statistically insignificant. The post indicator is identical to the top panel. Hence, results for schools closer to the district headquarters are no different than those farther from the district headquarters. 7 Future Directions We did not find any available observational data to investigate the impact of this policy reform on malnutrition or anthropometry of children. Most data sets such as the RCH focus on children between the ages of o to 5 years when they do not attend schools. There is a general need to conduct surveys which can help us to identify how nutritional status of children is changing in schools. With the help of IGC funding, we are conducting a survey to shed light on this aspect for IVRS. 8 Conclusion This paper studies the role of IT technology in improving transparency and accountability in public service delivery. We use the roll-out of a technology enabled monitoring mechanism 19 Although it is possible that school headmasters and the staff that IVRS calls keep extensive diaries so that they report consistently to each system. A very large number of headmasters would have to do this to get identical results here across samples and yet generate the improvements we observe using the ASER data. 17

18 (the Interactive Voice Response System or the IVRS) in the Mid Day Meal provision in Bihar, and show that a simple mechanism that aids in cross tallying the information provided by the middle tier of the delivery chain in welfare programs can reduce leakages and increase the efficacy of the programs. Using independently collected data, we find that the technology enabled policy change increases the likelihood of meal provision in a school in Bihar by 21 percentage points, and the likelihood of meal being cooked on the day of surprise visit to school by 15 percentage points. These results are robust to a number of specifications which include matching based difference-in-difference specifications, and control district specific trends. The increase in the take-up of the program by beneficiaries is also accompanied by an improvement in the quality and sufficiency of meals. Using central government commissioned audits data, we find an increase of 47 percentage points in fraction schools serving good quality of meals in Bihar schools post IVRS and an increase of 48 percentage points in fraction of schools serving sufficient quantity meals. In contrast, using state official records, we find that number of children availing MDM per school reduces post IVRS and percentage of schools providing meals is trending down. Surprisingly, the amount of rice consumed and cooking costs per school within district increase. Our results provide evidence that the IVRS resulted in reduction in leakage in the delivery system. Our findings have important policy implications. This study demonstrates that a policy driven reform initiated by a state government succeeded in improving the delivery in of a very important public service. Hence, state capacity can be increased by reforming the existing public institutions. Second, these improvements might be portable to other arenas of public service delivery which have similar delivery channels such as the public distribution system. 18

19 References Aker, J. C. and Mbiti, I. M. (2010). Mobile phones and economic development in africa. Björkman, M. and Svensson, J. (2009). Power to the people: Evidence from a randomized field experiment on community-based monitoring in uganda. The Quarterly Journal of Economics, 124(2): Duflo, E., Hanna, R., and Ryan, S. P. (2012). Incentives work: Getting teachers to come to school. American Economic Review, 102(4): Figlio, D. N. and Winicki, J. (2005). Food for thought: the effects of school accountability plans on school nutrition. Journal of Public Economics, 89(23): Galiani, S., Gertler, P., and Schargrodsky, E. (2005). Water for Life: The Impact of the Privatization of Water Services on Child Mortality. Journal of Political Economy, 113(1): Goyal, A. (2010). Information, direct access to farmers, and rural market performance in central india. American Economic Journal: Applied Economics, 2(3): Jensen, R. (2007). The digital provide: Information (technology), market performance, and welfare in the south indian fisheries sector. The Quarterly Journal of Economics, 122(3): Nagavarapu, S. and Sekhri, S. (2015). Informal monitoring and enforcement mechanisms in public service delivery: Evidence from the public distribution system in india. Journal of Development Economics, 111. Pritchett, L. (2009). Is India a Flailing State?: Detours on the Four Lane Highway to Modernization. Technical report. 19

20 FIGURE 1: Percentage of Primary Schools Serving Mid-Day Meals. 1 Average Provision of Mid-Day Meals Year Bihar Other States Notes: We use school level data from the Annual Status of Education Report (ASER) for the years Other states include Jharkhand, Madhya Pradesh, Orissa and Chhattisgarh. 20