School Choice and College Attendance

Transcription

1 DRAFT DO NOT CIRCULATE September School Choice and College Attendance Evidence from Randomized Lotteries David Deming Justine Hastings Thomas Kane Douglas Staiger This is a preliminary draft. Please do not circulate or quote it without prior permission. Comments are welcome and appreciated. ABSTRACT In 2002, Charlotte Mecklenburg school district implemented an open enrollment policy that allocated slots at oversubscribed schools via random lottery. To assess the impact of gaining admission to a highly demanded high school, we match administrative data from the district to the National Student Clearinghouse, a national administrative database of postsecondary enrollment. We find strong evidence that high school lottery winners from neighborhoods assigned to the lowest-performing schools benefited greatly from choice. Girls are 12 percentage points more likely to attend a four-year college. Boys are 13 percentage points more likely to graduate from high school but are less likely to attend a four-year college. We present suggestive evidence that changes in relative rank within schools may explain these puzzling gender differences. In contrast with the results for students from low-performing home school zones, we find little evidence of gains for students whose home schools are of even average quality.

2 September School choice is an increasingly important feature of the U.S. education policy landscape. Scarce public resources and the rising return to education have led to a focus on policies that can enhance schools productivity. Proponents of school choice espouse policies which decouple neighborhood residence and school attendance, breaking the monopoly power of local school districts and causing schools to compete for students (Hoxby, 2003.) Aside from competitive pressure, school choice could also enhance welfare by improving match quality between students and schools (Hoxby, 2003). Improvement in outcomes for student applicants is a necessary condition for choice to be efficiency-enhancing. Yet evidence on the benefits of public school choice, at least in the U.S. setting, is weak. The most broad-based form of school choice is known as open enrollment. Open enrollment allows public school students to attend magnet or other public schools that are outside of their neighborhood zone, subject to the availability of slots. A number of school districts have implemented some form of open enrollment in recent years. Cullen, Jacob and Levitt (2006) study one such program in Chicago public high schools. They use lottery-based random assignment to oversubscribed schools to investigate the causal impact of being offered admission to a non-home school, and find no evidence of benefits across a number of outcomes despite sizeable changes in measured peer quality (Cullen, Jacob and Levitt 2006). Voucher programs in Milwaukee (Witte 1997, Rouse 1998), New York City (Howell and Peterson, 2002) and Washington DC (Wolf et al, 2008) have found mixed impacts on student achievement. Some of the best evidence for the benefits of school choice comes from recent studies of lottery-based admission to charter schools. Hoxby and Rockoff (2004) and Hoxby and Murarka (2007) find modest effects of charter schools on student in Chicago and New York City respectively. Very recent results from Boston (Angrist et al, 2009) and the charter schools in the Harlem Children s Zone (Dobbie and Fryer, 2009) have found very large yearly gains on standardized test scores, particularly in middle school math. Among studies of school choice that do find positive impacts, they are largely limited to test scores as an outcome. Usually this is by necessity, since few other indicators of progress for school-age children are available. Although past research has demonstrated the connection between test scores and wages (Murnane et al 1995;

3 September Krueger 2003), postsecondary outcomes such as educational attainment, earnings, and health are of direct interest. Furthermore, some of the best evidence concerning long-term benefits of social programs comes from interventions where test score gains faded out over time (Anderson 2008; Belfield et al 2006; Deming 2009). Recent studies of school responses to high-stakes accountability policies have found evidence of teaching to the test (Jacob 2005), or purely non-productive test score gains through strategic assignment to test-taking (Figlio 2006, Jacob 2005) and teacher cheating (Jacob and Levitt 2003). This raises concerns that schools may be inefficiently multi-tasking by acting to maximize measured achievement at the expense of postsecondary outcomes of direct interest such as college attendance (Holmstrom and Milgrom, 1991). In this paper we study the impact of open enrollment in high schools in Charlotte Mecklenburg school district (CMS) on subsequent college attendance. In 2002, CMS implemented a district-wide school choice plan. Students were guaranteed admission to their neighborhood school but were allowed to choose and rank up to three other schools in the district, including magnet schools. Nearly half of all rising ninth graders chose a non-guaranteed school. Where demand for school slots exceeded supply, allocation was determined by random lottery. Furthermore, the great majority of students who were offered admission chose to enroll, leading to a large one-year change in student assignments in the district. Since the youngest students in this sample were rising 9 th graders in 2002, we are able to observe college-going for a minimum of two years after lottery participants were scheduled to graduate from high school. These are critical times in students lives. Over 90 percent of eventual bachelor s degree recipients enter college within the first two years of graduating from high school (College Board, 2007). We match a long ( ) and detailed panel of administrative data from CMS, including the lottery numbers and admission status of all applicants, to the National Student Clearinghouse (NSC), a national database of postsecondary enrollment. The match is done directly using personal identifying information. As a result, attrition from the college attendance data is limited only to colleges that are not covered by the NSC. This is a very small fraction of four-year and public two-year colleges nationwide, and an even smaller fraction of colleges in North Carolina and the surrounding states.

4 September The central finding of this paper is that students with the lowest performing guaranteed (or home ) schools benefit greatly from choice. We define low performing schools by their average test scores and rates of college attendance, but also by value added measures and estimates of parental demand from the choice lottery. Students from neighborhoods that are assigned to one of the four lowest-performing schools stay in school longer, graduate at higher rates, and are more likely to attend college. In contrast, we find no evidence of gains for students whose home school is of even average quality. In a pattern that is consistent with many previous studies, the effects of school choice are markedly different by gender (Kling et al 2007; Hastings, Kane and Staiger 2006; Schanzenbach 2007; Angrist, Lang and Oreopoulos 2009). Females from lowperforming schools who win the lottery to attend their first-choice school are about 12 percentage points more likely to attend a 4 year college. Although these students are still completing their schooling, the effects on persistence in college for females are proportionally even larger than for the enrollment margin. Male lottery winners are about 13 percentage points more likely to graduate from high school. In contrast we find no increase in college enrollment for males, and some evidence that lottery winners are actually less likely to persist in 4 year colleges. We present some evidence on changes in class rank for male lottery winners that may provide a possible explanation for this puzzling result. The findings are corroborated by impacts on grades, course-taking patterns and disciplinary incidents. Lottery winners from low-performing neighborhood school zones had higher grade point averages, took more math and science classes, and were less likely to be absent and suspended from school in the two school years following the lottery. In all cases, effects were large for children from low-performing school zones and small or near zero for all others. Changes in average peer characteristics also match these patterns lottery winners from the lowest-performing schools had peers with significantly higher test scores and graduation and college enrollment rates, but this was much less true for lottery winners from other schools. This paper makes several contributions. The first is to provide evidence on the benefits of a specific form of school choice open enrollment in public high schools. The only other study to look at high school choice in a similar setting found no impact of

5 September winning the lottery to attend a first-choice high school in Chicago Public Schools (Cullen, Jacob and Levitt 2006). There are two possible reasons for this discrepancy. The first is that school choice has a disproportionate impact on students with a low quality default option. In Chicago, students could apply to many schools, the chances of acceptance were much lower, and a relatively low fraction of accepted students actually enrolled. Thus lottery applicants may have been weighing several other acceptable alternatives. Because of the twenty year history of school choice in Chicago, parents had many years to exercise Tiebout choice through residential location. Although this is true in CMS as well, the sudden opportunity to attend a non-home school may offer immediate benefits that are less available in equilibrium, once parents have re-sorted and admission probabilities have adjusted. The second possibility is that school choice has the strongest impact on postsecondary outcomes that have not been available in previous studies. The NSC data are available for all students in CMS in multiple years, enabling a comprehensive examination of college-going in a large urban school district. In contrast to a model of strict peer achievement maximization, parents seem to choose schools that are demographically similar but do a much better job of sending children to college. This may not be surprising if students test scores are relatively fixed by 9 th grade yet institutional features of a school (i.e. a good guidance counselor) could still have a large impact on college enrollment. The results here also contrast with Hastings, Kane and Staiger (2008), who study the 2002 lottery for elementary and middle school students and find test score gains only for children whose parents place a high implicit weight on academic achievement. These children are also more likely to be white, upper income and have high test scores. Thus school choice appears to benefit more advantaged children in the short-run but more disadvantaged children in the long-run. 1 Finally, this paper provides an in-depth look at the relationship between school success, course-taking and the growing reverse gender gap in educational attainment. While boys and girls choose similar schools and enter high school with similar test scores, female lottery winners accumulate more credits and have higher GPAs, and this 1 Another possibility is that preferences and choice patterns for high schools are different than for elementary and middle schools.

6 September leads to increases in four year college enrollment. In contrast, boys are no more likely to accumulate math and science credits after 9 th grade and have only marginally higher GPAs. One important possible explanation is that highly demanded schools are also more challenging, and that to succeed students must adjust their level of effort. We find that lottery winners of both genders enter school at a lower relative rank than lottery losers on an initial test score. However, female lottery winners adjust by improving their grades so that they are similar to lottery losers in the GPA distribution, whereas male lottery winners lose GPA rank. This may be a partial explanation for the pattern of findings on college attendance by gender. 2. Institutional Detail and Data Description 2.1 Background and Details of the Choice Plan Charlotte-Mecklenburg is the 20 th largest school district in the nation. The school district encompasses all of Mecklenburg County, which includes both the inner city areas of Charlotte and the more affluent suburbs surrounding it. Thus neighborhoods in CMS and the schools that are assigned to them vary widely by race and income. In 1971, the Supreme Court (in Swann v. Charlotte-Mecklenburg Board of Education) ruled that this variation resulted in neighborhood schools that were de facto segregated, and for over 30 years CMS schools were forcibly desegregated under a court order. Students were bused all around the district to preserve racial balance in the schools. Particularly at the high school level, this meant in practice that inner-city and largely African-American neighborhoods were divided up and bused out to more affluent and white suburbs in different parts of the county. After several years of legal challenges, the court order was overturned and CMS was declared unitary and ordered to dissolve its busing plan. In 2001 the State Supreme Court declined to hear an appeal, signaling the end of desegregation. In December of 2001 the CMS School Board voted to move forward with district-wide open enrollment for the school year. Because CMS was no longer allowed to use race explicitly in student assignments, they redrew school boundaries as traditional contiguous neighborhood school zones. Children who lived within each zone received guaranteed access to their neighborhood school. This resulted in a change in assigned neighborhood

7 September high school for about 35 percent of households. Figures 1a and 1b provide a visual illustration of this dramatic change. Figure 1a is a map of the high school boundaries in the year before the choice plan ( ), and Figure 1b shows the boundaries in the following year. The hatched areas on both maps are areas that experienced a change in high school assignment. As the figures show, the majority of central city areas and substantial shares of the suburbs were reassigned. The school choice lottery took place in the spring of In order to maximize the number of parents that exercised choice, CMS conducted an extensive information campaign. They held a well-advertised fair at the Charlotte convention center, set up choice booths in local shopping malls, and sent volunteers door-to-door in low-income and non-english speaking neighborhoods to talk to families about the plan (Hastings, Kane and Staiger 2008). CMS also developed a comprehensive booklet with information about each school, as well as smaller brochures for individual schools. As a result, over 95 percent of parents submitted a choice application in the spring of Parents were allowed to submit up to three choices, which included schools as well as special programs within schools. 2 Students were guaranteed admission to their neighborhood school, and admission for all other students was subject to grade-specific capacity limits that were set by the district beforehand but unknown to families at the time of the lottery (Hastings, Kane and Staiger 2008). Children with siblings already in enrolled in a school also received guaranteed access. CMS was also divided into four choice zones and transportation was provided by the district only within each zone, although families were free to provide their own transportation to any school. 3 The district expanded capacity at schools where they anticipated high demand in an attempt to give every parent one of their top three choices. Still, most high schools were oversubscribed. When demand for slots among non-guaranteed applicants exceeded supply, admission was allocated by random lotteries that occurred within the following lexicographic priority groups. The first three groups consisted of students who had 2 Parents who listed 3 non-guaranteed choices were automatically assigned their home school as a 4 th choice. 3 The choice zones were constructed so that there was at least one predominately white suburban and at least one predominantly black inner-city school in each zone.

8 September attended the school in the previous year, with the highest rising grade (12 th ) given the first grouping. In practice, this meant that most lotteried applicants (about 60 percent) were rising 9 th graders. The next priority group consisted of free-lunch eligible (i.e. low income) students applying to schools where less than half of the school population was free-lunch eligible. The final priority group was students applying to a school within their choice zone. Applicants were sorted by priority group according to these rules, and then assigned a random lottery number. Slots at each school were first filled by students with guaranteed access, and then remaining slots were allocated within each priority group according to students lottery numbers. If all members of a priority group could be offered admission, slots were allocated to the next priority group in the order of lottery numbers. CMS administered the lottery centrally (i.e. schools did not conduct their own lotteries) and applied an algorithm known as a first choice maximizer (Abdulkadiroglu and Somnez, 2003). This meant that CMS first allocated slots to all those who listed a school as their first choice and only then moved to second choices. As the name indicates, this maximized the percentage of students who received their first choice, but it also meant that students who lost the lottery to attend their first choice school often found that their second choice had already been filled up in the previous round. While there is the potential for strategic choice with this type of lottery mechanism, Hastings et al (2006) show evidence that this is not likely to be a large problem in CMS, at least in the first year of the choice plan. In any case, even if parents are of two types ( sincere and sophisticated ), each has the same ex ante probability of winning the lottery so it will not bias the results, although it may complicate their interpretation (Pathak and Somnez, 2008). 2.2 Data Description and Summary Statistics We match the lottery applicant file, with individual lottery numbers and priority groupings, to a panel of administrative data from CMS. The data span approximately six years before and after the choice lottery (from 1995 to 2008) and contain detailed information on students enrollment histories, test scores, course-taking and other outcomes of interest. The North Carolina Department of Public Instruction requires all

9 September school districts to assemble and send them a standardized set of files under the state s accountability regime. In addition to enrollment records, this includes students scores on standardized End-of-Grade (EOG) exams in math and reading for grades 3-8 and End-of- Course (EOC) exam scores in high school subjects such as Algebra I and II, Geometry, English I, Biology and Chemistry. These tests are administered to all public school enrollees and schools are required to use the scores as some component of students grades. Importantly, these reporting requirements help to ensure that CMS s records are of high quality, with longitudinally linked and consistent student records. See the Data Appendix for a more thorough description of the CMS administrative data. We match these files to information on college attendance from the National Student Clearinghouse (NSC), a non-profit organization that maintains enrollment information for over 90 percent of colleges nationwide. In collaboration with CMS, we provided each student s full name, date of birth and (when applicable) high school graduation date, which the NSC used to match to their database. The data contain information on enrollment spells, full or part-time status and (in some cases) degree receipt for all covered colleges that a student has ever attended. Although not all colleges provide information to the NSC, the coverage is very good in North Carolina and the surrounding states. The Data Appendix contains a list of colleges by coverage and a detailed analysis of the match process using data from the Department of Education s Integrated Postsecondary Data Source (IPEDS) as a reference. 4 Thus students who leave CMS are also followed in these data, and attrition is subject only to the NSC s coverage and the quality of the match. However, unless coverage is differential for lottery winners and losers, the results may be attenuated but will not be biased. CMS received high school lottery applications from 29,584 students. We first limit the sample to students who were enrolled in any CMS school in the previous year. About five percent of applicants come from outside the district, and these students are much less likely to be enrolled in CMS the following fall. Since previous enrollment status is fixed at the time of the lottery, this sample restriction does not affect the validity 4 The major two-year college in Charlotte, Central Piedmont Community College (CPCC), did not provide information to the NSC until To fill in this gap, I obtained enrollment data directly from CPCC for all years which was more detailed than what is typically provided by the NSC. The data from CPCC contain information on type of enrollment (i.e. degree-seeking or correspondence course) and credit accumulation and GPA. I also used this data to verify the NSC s match process. See the Data Appendix for details.

10 September of the randomization. We also drop the less than one percent of students who choose alternative schools. Finally, about five percent of this sample does not show up in any CMS school in the fall of Although these students can still be matched to the NSC and arrest record data and are included in those analyses, we have no other outcome information for them. Thus selective attrition is a concern for analyses that use CMS administrative data. This final restriction results in a sample of 26,242 rising 9 th -12 th graders. Even with mandatory busing, the demographic composition of CMS high schools varied widely. The redrawing of school boundaries as contiguous neighborhood school zones caused these already sizeable gaps to widen further. The first and fourth columns of Table 1 report the average 8 th grade math score (in standard deviation units) of rising 9 th graders and the percent of black students in each high school for the year before the choice plan ( ). The second and fifth columns show the change in composition for each school in the year of the choice plan ( ). The third and sixth columns compute the average characteristics of students in each high school s neighborhood zone. This shows what CMS schools would look like if the boundaries were redrawn but there was no school choice. The schools are ranked by the average test score of their 2002 neighborhood zone. The difference between the top and bottom school was already large before open enrollment (about 1 standard deviation), but it grows further to about one and a half standard deviations in the year of the choice plan. Similarly, racial sorting increased sharply. This is not surprising given the composition of each school s neighborhood zone. The seventh through tenth columns of Table 1 show enrollment patterns. The utilization rate is simply the number of children who attend a school divided by the total number zoned to that school. Lower test-score schools are under-enrolled, with capacity at or just above 50 percent in a few cases. This is partly due to the 3 magnet high schools in the district, which are located in inner city areas. One school, Berry Academy of Technology, was opened for the first time in Still, several lower-test score schools saw large drops in enrollment from 2001 to 2002, which can only be partially accounted for by increases in the supply of magnet slots. In contrast, most of the higher-scoring schools were close to capacity or oversubscribed.

11 September Patterns of Choice in CMS There was a great deal of churning in CMS schools in the year of the choice plan. Over 35 percent of rising 9 th graders chose a school other than the one to which they were assigned. This was due partly to the large one-year change in school boundaries, but choices were very heterogeneous even among parents whose home school remained the same. Table 2 shows these patterns of choice by neighborhood school zone. The first two columns show the number of rising 9 th graders assigned to each neighborhood school, and the fraction that chose their neighborhood school first. This share, while never close to 100 percent for any school, varies greatly across schools. Like Table 1, schools are sorted by the average 8 th grade test scores of students in the neighborhood zone. Not surprisingly, higher test-score schools are chosen more frequently. What is notable, however, is how few of the students in the low test-score neighborhoods choose their home school. Among the four lowest-scoring neighborhoods, only about 45 percent of residents choose to attend their home school. For rising 9 th graders, that number is less than 30 percent. As a consequence, a disproportionate share of the lottery sample consists of students from low-scoring neighborhoods. Two conditions must hold for students to be in the lottery sample. First, a student must apply to a non-guaranteed school. Second, the probability of admission for that student s priority group must be greater than zero and less than one (i.e. a non-degenerate lottery). In the lottery analysis specification in Section 3, the weight given to each school in the overall analysis is the number of students in the lottery times p*(1-p) where p is the probability of winning the lottery (Cullen, Jacob and Levitt 2006). Intuitively, this is because lotteries with a more balanced (i.e. closest to 50 percent) proportion of winners and losers will have greater power to detect true differences in outcomes. The result, in column 7 of Table 2, is that students from the lowest-scoring neighborhoods constitute a disproportionate share of the lottery sample. Table 3 presents choice patterns organized by first choice rather than neighborhood school. The first thing to note is that the 4 highest-scoring schools admitted over two-thirds of non-guaranteed applicants. Three of these 4 schools admitted 100

12 September percent of rising 9 th grade applicants. This suggests that parents are not all strictly maximizing on average test scores since those schools were available but not chosen. Instead, the most frequently chosen schools rank in the middle in terms of test scores but are demographically similar to the lowest scoring neighborhoods. The set of lottery weights calculated in column 7 reveals that the 3 magnet schools account for about 60 percent of the lottery sample. It is also important to note that enrollment conditional on admittance is very high for all schools. This suggests that students were not relying on unobserved outside options, and is important for interpretation of the results presented in Section 4. Table 4 compares the characteristics of all students in CMS to the lottery sample. Columns 1 presents the average characteristics of the full 9 th -12 th grade sample, and column 2 restricts to students who chose their home school. We can see that students who choose their (guaranteed) home school are more likely to be white, less likely to be free lunch eligible, and have higher average test scores. Columns 3 through 5 organize students who chose a non-guaranteed school into 3 categories. The first and second categories are students in degenerate lotteries, where everyone was admitted and everyone was denied, respectively. Column 5 shows the average characteristics of the lottery sample, which consists of 1,230 rising 9 th graders and 786 rising 10 th -12 th graders. All three categories of non-guaranteed applicants are more disadvantaged than students who chose their home school. However, students in the lottery sample look relatively similar on observables to other non-guaranteed applicants. 5 To create a summary statistic of prior academic preparation, we estimate a logistic regression of four-year college attendance on a polynomial in prior test scores, student demographics and prior school and census tract fixed effects. We then plot this predicted probability for students who choose their home school and then for the three groups of non-guaranteed applicants in Figure 3A. We can see that applicants who choose their home school are much more likely to attend college than others, but that the three groups 5 Among those who chose a non-guaranteed school, the all admitted group is more likely to be free-lunch eligible. This is because free lunch students who applied to a school with a low fraction of free-lunch eligible students were given an explicit priority boost in the lottery. This doesn t affect the identification strategy since we estimate the effect of the winning the lottery for applicants within the same priority groups.

13 September who choose non-guaranteed schools (all admitted, all denied, and the lottery sample) are very similar. Figure 3B plots this same predicted probability for students from lowperforming school zones. Here we can see that there is very little difference between any of the groups. Although lottery applicants are clearly different in some way since they chose not to attend their home schools, it is important to note that they are similar on an index of observable predictors of college attendance. Finally, Column 6 shows the characteristics of the less than five percent of the sample who did not submit a choice. Nearly 50 percent are free lunch eligible, and they have average test scores about 0.75 standard deviations below the average. We cannot say anything about the effect of winning the lottery for this population. 2.4 Measures of School Quality Schools with low average test scores are not necessarily low-performing. If parents know this, they may choose schools that provide better value added over schools with better peers. Still, there is plenty of evidence that parents value test scores in their choice of school, either through higher housing prices in the residential location decision or through school choice programs (Black 1999, Bayer et al 2006, Hastings et al 2006.) It is less clear, however, that gains on peer quality measures are productive for students, despite parental demand. Perhaps incremental improvements are effective but large changes lead to mismatch (Cullen, Jacob and Levitt 2006). Alternatively, mean peer achievement in levels may not be as informative as in gains. Yet parents also show weak responsiveness to value added measures in existing studies of school choice (Cullen, Jacob and Levitt 2006; Hastings, Kane and Staiger 2008). This could be due either to a lack of information about school effectiveness (Hastings and Weinstein, 2008), because they do not value effectiveness as much as other neighborhood features (Rothstein 2006), or because value added is not actually a good measure of school quality. Table 5 measures school value-added and parental demand. Value-added is calculated as the school-level mean residual from a student-level regression of each outcome on a 3 rd order polynomial in 7 th and 8 th grade reading and math test scores, demographic characteristics, and census tract fixed effects. This effectively compares students who live in similar neighborhoods, with similar observable measures of

14 September academic preparation, but who go to different high schools. These estimates are likely to suffer from bias due to nonrandom sorting into high school zones that is not picked up in test scores (Rothstein 2009). Still, they may be informative and are likely to be less biased than simple comparisons of average test scores. We calculate value-added measures for two outcomes. The first is students standardized score on the English I EOC exam, which is taken by nearly every student in the 9 th grade. The second is the predicted probability that students will have enrolled in four total semesters of any college by the spring of Alternative measures of college enrollment, such as any enrollment or at least one semester at a four-year college, yield similar results. 6 Although we do not report scores based on the th grade cohort, the across-year correlation in value-added for both English I and college attendance is reasonably high, about 0.75 and 0.6 respectively. This is notable because both rezoning and choice greatly affected the selection process of students to schools. We also calculate parental demand, first as the percentage of families that select each school as their first choice, weighted by the size of each schools neighborhood zone and then standardized. This is reported in column 4. Column 5 reports the mean school residual from a conditional logistic regression which predicts the probability that families will choose each high school, controlling for a 4 th order polynomial in travel time, plus home school and choice zone fixed effects. This is intended to account for the fact that some schools are not as highly demanded because they are located in less dense parts of the district. See Long (2004) for an explanation of the conditional logit setup and Hastings, Kane and Staiger (2008) for a richer specification of parental choice that uses all 3 choices and allows for substitution patterns in a mixed logit framework. There are 3 important takeaways from Table 5. The first is that value-added measures are positively correlated with average test scores. This is a combination of true productivity differences between the schools and the bias from unobserved sorting across neighborhoods that is not captured by test scores and other covariates. Interestingly, the correlation is modestly higher for college-going than for the 9 th grade English score 6 Two year enrollment often includes things like non-degree correspondence courses and GED retraining, which are arguably not measures of the academic contribution of a school. Few of these enrollees stay for more than a semester or two, however. One option is to only count 4-year college enrollment, yet some schools might specifically target 2-year colleges. In any case the correlation between different value-added measures of college enrollment is always over 0.9.

15 September value-added (0.67 vs. 0.53). The second thing to note is that value-added measures also predict parental demand. Again, college value-added is a stronger predictor than English (0.46 versus 0.24), and it is actually more correlated with parental demand than average test scores (0.25). The last important point is that there are some schools which rate poorly on all measures. Strikingly, the same four schools rate at the bottom on average test scores, college value added, and parental demand. Students from these neighborhood zones also make up nearly two-thirds of the lottery sample. Clearly, parents are choosing to exit these low test-score and low value-added schools. 3. Empirical Strategy If winning the lottery is randomly assigned, the winners and losers of each lottery will on average have identical observed and unobserved characteristics. Thus in expectation, the effect of winning the lottery can be estimated as a simple difference in means. With sufficient sample size, one could estimate this difference for every lottery, but here the sample is not large enough for such a calculation. Instead, following Cullen, Jacob and Levitt (2006), we estimate: Y ij = βx ij + δ(win Lottery) ij + Γ j + e ij (1) where Y is the outcome variable of interest, X ij is a vector of covariates which includes gender, race, free or reduced price lunch status, special education and limited English proficiency status, prior math and reading test scores, suspensions, absences and school fixed effects. Win Lottery ij is an indicator variable that equals 1 if the student won the lottery, Γj is a set of lottery fixed effects and e ij is a stochastic error term. The number of observations is equal to the number of students in the lottery sample since there is only one first choice application per student. In principle we could estimate a nested model that incorporates multiple choices. However, students who lost the lottery to attend their first choice were only rarely randomized into another school. Thus restricting to first choices captures nearly all the experimental variation in school

16 September attendance. Standard errors are clustered at the lottery (i.e. grade by choice by priority group) level. Lottery fixed effects are necessary to ensure that the ex ante probability of admission to a first-choice school does not differ between losers and winners. If, for example, savvy families had some prior knowledge about the chance of admission, they might (all else equal) apply to schools with a higher probability of acceptance. Thus comparing winners and losers across lotteries may induce bias. In this specification, the δ coefficient gives the weighted average (with weights approximately equal to column 8 of Table 3) of treatment minus control differences summed over each individual lottery. Here δ corresponds to an intent-to-treat (ITT) effect of attendance at a first choice for students who choose to apply to schools where the probability of admission is greater than zero and less than one. We cannot estimate the effect of attendance for students who apply to a guaranteed school or to a school with a degenerate lottery. The ITT does not capture the effect of attending a first-choice school, but rather the effect of being offered admission. Not all students who win the lottery will subsequently enroll. In Table 3 we saw that enrollment conditional on admission was very high however, some lottery losers may also attend the school, largely by moving into the neighborhood zone after the lottery or in subsequent years. An alternative to estimating the ITT is to use lottery status as an instrument for attendance at the first-choice school. Under certain assumptions the ITT can be scaled up by the difference in attendance to become a local average treatment effect (LATE). However, some lottery winners stay in school all four years and some leave much sooner. If better schools are also more academically challenging, the effect of winning the lottery may depend on a student s prior preparation. In this case, withdrawal is also part of the treatment. Perhaps the best reason to report ITT estimates is that the effect of being offered admission is a direct policy parameter. In any setting where a school choice plan is implemented, not all students will apply nor will all applicants take up an offer of admission. School districts cannot force students to attend a particular school, and since

17 September length of enrollment among lottery winners is not randomly assigned, the LATE may be of limited external validity. 3.1 Verification of Randomization and the Effect of Winning on Attendance If the lotteries were conducted appropriately, there should be no difference on average between winners and losers on any characteristic that is fixed at the time of application. Table 6 tests this proposition by estimating equation (1) with a set of fixed demographic covariates as outcomes. Column 1 shows the mean value of each variable for lottery losers, and Column 2 shows the estimated difference and standard error for lottery winners. Reassuringly, mean differences are small and not statistically different from zero, suggesting that the randomization worked. The estimated effect of winning the lottery could still be biased by selective attrition if leaving the CMS school district is correlated with lottery status. Columns 3 and 4 of Table 6 reestimate covariate differences for the approximately 95 percent of the sample that was still in enrolled in CMS in the fall of 2002, after the lottery was conducted. The bottom two rows test the hypothesis that lottery winners are more likely to remain in CMS, About 94 percent of lottery losers are still enrolled, and winning the lottery has a precisely estimated zero impact on remaining in the district. Furthermore, as stated in Section 2, we have data on college attendance even for this small fraction of students who leave CMS after losing the lottery. 7 Thus, selective attrition is unlikely to be a concern here. In Columns 5 and 6 we conduct a similar randomization check for the subsample of students who are assigned to the four lowest-performing high schools, as defined in Table 5. Here there is some evidence of imbalance. Although lottery winners score no higher on 8 th grade math or reading test scores and are no more likely to be absent or suspended from school in the previous year, they are somewhat more likely to be both male and African-American. While this may just be a statistical fluke, the difference is troubling. We went back through the lottery files to see if the imbalance was concentrated in one lottery, and it was not. We estimated the regressions in Table 6 for 8 th grade scores and predicted 4-yr enrollment within male and African-American applicant groups. No 7 Subject to coverage see Appendices on NSC and Mecklenburg County Arrest data for details.

18 September estimates were statistically different from zero. As a final robustness check, we estimated regressions for all the main outcomes using race and gender as additional priority groups in the lottery, which effectively compares winners and losers only within these groups. The main results were unaffected. Table 7 presents estimates of the effect of winning the lottery on initial fall 2002 enrollment and characteristics of the school attended. We split the sample into students from one of the four lowest-performing school zones (Columns 3 and 4) and all other students (Columns 1 and 2). Overall, lottery winners were 55 percentage points more likely to subsequently enroll in their first-choice school. This shows that winning the lottery had a large and highly significant impact on initial enrollment. About 35 percent of lottery losers still manage to enroll in their first choice school. 8 Lottery winners are about 36 percentage points less likely to be enrolled in their neighborhood school and about 30 percentage points more likely to be enrolled in a magnet school. They are also enrolled in schools that are about 1.5 miles farther away from their residence. Interestingly, the difference in distance to assigned school is near zero for the group of applicants from low-performing neighborhood schools. This reflects the fact that the three highly demanded magnet schools are all located in inner city areas. Since previous research has shown that low-income parents value proximity very highly, this may be an important explanation for the pattern of parental demand (Hastings, Kane and Staiger 2008; Hastings and Weinstein 2008). Lottery winners experienced large changes in average peer characteristics. 9 Importantly, this was concentrated mostly among winners with low-performing neighborhood school assignments. On measures such as average 8 th grade math scores, grade point average, high school graduation and college attendance, average peer characteristics of lottery winners were about 0.5 school-level standard deviations better than for losers. Proportional gains were particularly large (about 0.63 school-level 8 This is due to a combination of factors. In a few cases, students lost the lottery for a special program within a particular school (such as the International Baccalaureate program) when regular admission to that school was not oversubscribed. The most common ways that lottery losers managed to enroll in their first choice is through subsequent admission off the waiting list and relocation to the school s home zone after the lottery. Separate estimates for 9 th grade applicants show that only about 20 percent of losers managed to enroll, compared to nearly 50 percent for 10 th -12 th graders. 9 These average peer variables are calculating using enrollment in the fall of 2002, and they exclude students in the lottery sample.

19 September standard deviations) for four-year college matriculation. The differences for winners from all other areas, while often significantly different from zero, were much smaller. Finally, although lottery winners experienced large increases in peer quality across a wide variety of outcomes, it is important to note that the schools they attended were demographically similar to the ones attended by lottery losers. This is in contrast to Cullen, Jacob and Levitt (2006), who found that lottery winners attended schools with modestly higher high school graduation rates (about 2 percent) but also lower fractions of African-American, Hispanic and free lunch eligible children (about 3-4 percent). 4. Results 4.1 Persistence, Graduation and Transfer In Section 3 we show that winning the lottery had a strong and statistically significant impact on enrollment and on measures of peer quality for students with lowperforming neighborhood schools. This shows that lottery winners from these areas experienced a significant change in school environment. However, higher peer quality does not guarantee improvement on outcomes like test scores and educational attainment. Cullen, Jacob and Levitt (2006) find no gains for lottery winners despite significant gains in average peer test scores and graduation rates. In Tables 8 and 9 we evaluate the effect of winning the lottery on educational attainment, overall and then separately for students whose neighborhood school assignment is one of the four lowest-performing schools in CMS. We also split that sample by gender, since many educational interventions have found that girls benefit much more than boys, particularly when the sample is very disadvantaged (Kling, Ludwig and Katz 2005; Hastings, Kane and Staiger 2006; Schanzenbach 2003; Angrist et al 2009). Table 8 shows the effect of winning the lottery on persistence in school and graduation. The first three rows estimate the effect of winning the lottery on being in the first-choice school in each subsequent school year. The effect of winning the lottery could be attenuated over time, either because lottery winners transfer to another school or drop out of the CMS system altogether, or because lottery losers successfully enroll in subsequent years. In general, however, winning the lottery in the spring of 2002 has a strong and persistent impact on subsequent enrollment. Even by the fall of 2005, when

20 September rising 9 th grade lottery applicants would be entering their senior year of high school, 9 th grade lottery winners are about 34 percentage points more likely to be enrolled in their first choice school. 10 Overall, winning the lottery led to substantial increases in grade attainment. Column 2 shows that lottery winners were four to five percentage points more likely to stay in school through grades 10, 11 and 12. In Columns 4 and 6, we see that these gains are concentrated among lottery winners from low-performing neighborhood school zones. The point estimates are of similar magnitudes by gender but more precise for males, who were 9.6 and 9.4 percentage points more likely to make it to 11 th and 12 th grade respectively. Estimates for grade repetition are consistently negative but only statistically significant at the ten percent level for 9 th grade girls, who were about 9 percentage points less likely to be retained if they won the lottery. The bottom panel of Table 8 estimates the effect of winning the lottery on students withdrawal status from CMS. These data directly come from the district s administrative records, which are maintained and reported out to the North Carolina Department of Public Instruction as part of the state s accountability system. There are five mutually exclusive and collectively exhaustive possibilities graduate, transfer in state, transfer out of state, dropout and no code. High school graduation is defined conservatively, as students who have a confirmed diploma date and were present through the end of 12 th grade in a CMS school. In-state transfers are also defined conservatively, because CMS is required to verify that a student has enrolled at another school in the state of North Carolina before they can designate a student as an in-state transfer. Out-ofstate transfers are not verified as rigorously. A small share (about 6 percent) of CMS enrollees simply stop showing up in the data without receiving a withdrawal code. While it is difficult to know exactly how the decision is made to call a student a dropout versus not assigning a code, we can compare them on observables as well as match each category of student to the National Student Clearinghouse and see how many of them show up in a (North Carolina or other state) college. In the low-performing 10 All estimates exclude rising grades that, in the absence of grade repetition, would have already graduated and left CMS. For example, the effect of winning the lottery on fall 2005 enrollment is estimated for 2002 rising 9 th graders only. Likewise, the estimates for grade attainment in rows 4-6 exclude applicants who had already made it to 10 th, 11 th or 12 th grade when the lottery was conducted.

21 September subsample, the average 8 th grade math scores of students with no withdrawal code are very close to those of dropouts (about versus -0.75). 2.1 percent of girls and 8.5 percent of boys with no withdrawal code show up in a 4 year college, compared to about 1 percent of those coded affirmatively as dropouts. Based on these data and on the pattern that high risk males are more likely to not receive a withdrawal code, it is likely that a substantial fraction of these students have dropped out of school. Overall, the pattern of increased grade attainment among lottery winners also holds for high school graduation. Lottery winners are about 4 percentage points more likely to graduate from a CMS high school. Part of this effect, however may be offset by transfers. Winners are a statistically insignificant 2 percentage points less likely to transfer in-state, and they are no more likely to transfer out of state or have no code. This estimate represents some combination of true increases in graduation and differences between winners and losers in attrition from CMS over time. If we assume that all instate transfers graduate, for example, the mean difference in graduation would be about 2 percentage points. Again, however, the effects are concentrated entirely in the low-performing subsample. Female lottery winners from these schools are about 11 percentage points more likely to graduate from high school and about 8 percentage points less likely to transfer in-state. This large difference in transfers for females makes it difficult to know how to interpret the net effect of winning the lottery on high school graduation. A conservative approach might look only at the dropout variable, in which case the effect is about 3 percentage points and statistically insignificant. Thus the true effect is likely to be between 3 and 11 percentage points for females from low-performing schools. For males, however, the effects are much less ambiguous. Male lottery winners are about 13 percentage points more likely to graduate from a CMS school, and the effect is statistically significant at the p<0.01 level. They are only 1.7 percentage points less likely to transfer in state and over seven percentage points less likely to be affirmatively coded as a dropout. Since slightly less than half of the control group graduates from a CMS high school, 13 percentage points represents a very large and economically significant effect.