Graduation Rates for Choice and Public School Students in Milwaukee,

Transcription

1 JANUARY 2011 Overall, had MPS graduation rates equaled those for MPCP students in the classes of 2003 through 2009, the number of MPS graduates would have been about 18 percent higher. Graduation Rates for Choice and Public School Students in Milwaukee, John Robert Warren, Ph.D. University of Minnesota G r a d u a t i o n R a t e s f o r C h o i c e a n d P u b l i c S c h o o l S t u d e n t s i n M i l w a u k e e,

2 INTRODUCTION In coming months the future of education in Milwaukee and Wisconsin will receive much attention as elected officials seek to raise academic outcomes while facing a multi-billion dollar state budget deficit. In this challenging environment, Governor Walker and members of the Legislature would be wise to consider the results reported here on high school graduation rates in Milwaukee. Using seven years of data, University of Minnesota Professor John Robert Warren, a recognized expert in the field, tracks graduation rates for students in the Milwaukee Parental Choice Program (MPCP) and the Milwaukee Public Schools (MPS). Professor Warren estimates that low-income school choice students were about 18% more likely to graduate from high school than students from across the economic spectrum in MPS. Significantly, he reports that these results occurred during a period when the historically low MPS graduation rate was increasing. Thus, in one of the most important measures of educational achievement high school graduation recent developments in Milwaukee are positive, both for choice students and for students attending MPS. Professor Warren explains that separate research being conducted at the University of Arkansas will address whether expanded choice for Milwaukee parents has caused the higher rates reported here. The MPCP, now twenty-one years old, serves more than 20,000 students. It saved state taxpayers $37 million in FY As this report shows, it achieved higher graduation rates than MPS in six of seven years studied. Had MPS attained the same graduation rate achieved in the MPCP, an additional 3,939 Milwaukee students would have received diplomas between 2003 and According to authoritative 9TH GRADE CHOICE ENROLLMENT research cited in the Milwaukee Journal Sentinel, the annual impact from an additional 3,939 MPS graduates would include an additional $24.9 million in personal income and approximately $4.2 million in extra tax revenue. 1,400 1,200 1, Unfortunately, benefits for high school students in the MPCP are at risk. This is because increased regulation and funding cuts threaten the viability of private high schools participating in the MPCP. For example, tax support for these schools is less than 45 per cent of the public support for MPS schools. This is not financially sustainable, a fact that has caused private high schools in the MPCP to reduce freshmen enrollment despite high parent demand. For the first time in several years, the number of 9th graders entering the MPCP actually decreased in Without regulatory relief and increased financial support, the kind of positive results reported here are in jeopardy. Susan Mitchell President, School Choice Wisconsin Sources: The Fiscal Impact of the Milwaukee Parental Choice Program: Update and Policy Options, Robert M. Costrell, University of Arkansas School Choice Demonstration Project, SCDP Milwaukee Evaluation Report #22, December Dropouts lose millions in pay; Study also estimates more graduates would boost tax revenue, too Erin Richards, Milwaukee Journal Sentinel, December 4, 2009 ( Tax support/pupil: $6,442 for MPCP in vs. $15,034 for MPS: See PAGE 1

3 The annual impact from an additional 3,939 MPS graduates would include an additional $24.9 million in personal income. EXECUTIVE SUMMARY This study compares graduation rates of students in Milwaukee who use vouchers to attend private high schools with those who attend public high schools. In 2009, we reported graduation rates for six years through for students in the Milwaukee Parental Choice Program (MPCP) and students in the Milwaukee Public Schools (MPS). The current study, which provides updated evidence for , reinforces the earlier conclusion that students in the MPCP are more likely to graduate from high school than MPS students. Like the earlier report, this study incorporates reasonable assumptions about the direction and magnitude of biases inherent in the type of graduation rate measure that is used. Overall, had MPS graduation rates equaled those for MPCP students in the classes of 2003 through 2009, the number of MPS graduates would have been about 18 percent higher. That higher rate would have resulted in 3,939 more MPS graduates during the years. A recent analysis of the economic impact of high school dropouts suggests that the annual impact from an additional 3,939 MPS graduates would include an additional $24.9 million in personal income and about $4.2 million in extra tax revenue. 1 Valuable additional research on this topic is ongoing as part of a longitudinal study of the MPCP directed by the School Choice Demonstration Project (SCDP) at the University of Arkansas. 2 1 Dropouts lose millions in pay; Study also estimates more graduates would boost tax revenue, too Erin Richards, Milwaukee Journal Sentinel, December 4, 2009 ( For information on the study, see Milwaukee s Path to Economic Growth The Economic Benefits of Reducing Milwaukee s Dropout Rate, Alliance for Excellent Education, November The report estimated ( publication_material/econmsa) a per capita annual income and tax revenue gain, respectively, of $6,322 and $1,079 for high school graduates. 2 One component of this five-year longitudinal study will track ninth graders in MPCP, matched to similar students in the Milwaukee Public Schools (MPS). The study will follow both groups over several years to generate comparative rates of high school graduation. As this study is only in its fourth year, those results are unavailable. PAGE 2

4 PREVIOUS RESEARCH Do students benefit by using the Milwaukee Parental Choice Program (MPCP) to attend a private school instead of a Milwaukee Public Schools (MPS) school? In addressing that question, most prior evaluations focus on whether students in the MPCP score better on tests of academic achievement than students in MPS schools. As reviewed in our 2007 report, two studies 3 based on randomized trials each demonstrated significantly higher mathematics test scores for MPCP students as compared to MPS students four years after enrolling in the program; one study also showed significantly higher reading test scores. Evidence from randomized trials in a number of other cities including Charlotte, NC, the District of Columbia, Dayton, OH, and New York City supports the conclusion that programs like the MPCP have the effect of improving academic achievement, at least for some (generally disadvantaged) subgroups of students. More recently, second year reports from the School While academic achievement is important, it is at least as important to ask whether participating in a program such as the MPCP matters for students chances of completing high school. Choice Demonstration Project (SCDP) found test score gains attributable to the MPCP in 7th and 8th grade math. The SCDP also has found a statistically significant positive impact on MPS test scores as a result of MPCP competition. Fourth year results of the SCDP study are expected in the Spring of While academic achievement is important, it is at least as important to ask whether participating in a program such as the MPCP matters for students chances of completing high school. This is because completing high school is essentially a prerequisite for success in modern America. If it is true that participating in programs like the MPCP improves academic achievement, we might rightly ask whether there are tradeoffs for rates of high school completion. After all, if participating in programs like the MPCP improves students test scores but hurts their chances of graduating, then this may not be a tradeoff we are willing to accept. In our February 2009 report, we showed that graduation rates between and were higher for students in MPCP schools as compared to students in MPS schools, even after testing important assumptions that underlie the type of graduation rate measure used in this report. The present report updates those findings for the graduating class of Complete data for the 2010 graduation class are not yet available. 3 Jay P. Greene, Paul E. Peterson, and Jiangtao Du Effectiveness of School Choice: The Milwaukee Experiment. Education and Urban Society 31: ; Cecilia Elena Rouse Private School Vouchers and Student Achievement. Quarterly Journal of Economics 113: See, for example: John Barnard, Constantine Frangakis, Jennifer Hill, and Donald Rubin Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City. Journal of the American Statistical Association 98: ; Joshua Cohen. Forthcoming. School Choice as a Latent Variable: Estimating the Complier Average Causal Effect of Vouchers in Charlotte. Policy Studies Journal; Jay Greene Vouchers in Charlotte. Education Matters 1: 55-60; Alan Krueger and Pei Zhu Another Look at the New York City School Voucher Experiment. The American Behavioral Scientist 47: ; and Patrick Wolf, Babette Gutmann, Michael Puma, Lou Rizzo, Nada Eissa, and Marsha Silverberg Evaluation of the DC Opportunity Scholarship Program: Impacts After One Year. US Department of Education, Institute of Education Sciences. Washington, DC: US Government Printing Office. 5 John F. Witte, Patrick J. Wolf, Joshua M. Cowen, David J. Fleming, and Juanita Lucas-McLean The MPCP Longitudinal Educational Growth Study Second Year Report. The University of Arkansas: The School Choice Demonstration Project. 6 Jay P. Greene and Ryan H. Marsh The Effect of Milwaukee s Parental Choice Program on Student Achievement in Milwaukee Public Schools. The University of Arkansas: The School Choice Demonstration Project. 7 John Robert Warren Graduation Rates for Choice and Public School Students in Milwaukee. Milwaukee: School Choice Wisconsin. ( schoolchoicewi.org/library/research.cfm) PAGE 3

5 Academics and policy makers have called attention in recent years to the need for better measures of high school graduation rates. ESTIMATING GRADUATION RATES Academics and policy makers have called attention in recent years to the need for better measures of high school graduation rates. In part, this is because independent studies of high school graduation rates have produced widely differing estimates, usually lower than graduation rate estimates issued by school districts and state education agencies. 8 A forthcoming report by the National Research Council notes that [d]espite the strong need for sound and reliable measures of high school dropout and completion, there has been widespread disagreement among researchers, statisticians, and policy analysts about the true rates, how they are best measured, and what trends are evident over time. 9 The national media and prominent education officials have called attention to this situation and to the need for reform. In Honest Data on High School Dropouts, the New York Times editorialized on this subject in its April 28, 2008 edition. The paper concluded that procedures followed by the U.S. Department of Education had initially allowed states to phony up even the most basic data on graduation rates. As a result, the Times explained, [S]ome states wrote off students who dropped out in grade 9, 10 or 11, which allowed them to report a bogus graduation rate based on the number of graduates who began the year in the senior class. Other states brightened a grim picture by including G.E.D. recipients, who were actually dropouts and should have been counted as such. Not surprisingly, the state-reported rates were nearly always higher than the estimates derived from more comprehensive efforts such as those by Greene, Warren, and Swanson. The Times praised then-education Secretary Margaret Spellings for taking a welcome step in the right direction by issuing new rules for how those rates are calculated. According to the paper: By the school year, states will have to use the generally accepted way of computing their dropout rate. That means tracking students from the day they enter high school until the day they receive regular diplomas, counting as nongraduates those who leave without the diploma. This method was endorsed three years ago by the National Governors Association, which realized that accurate graduation rates were a vital indicator of how well the schools were doing. Until states have faithfully and completely implanted the new federally-mandated technique for tracking and classifying students and reporting graduate rates, it will remain difficult to make meaningful comparisons of graduation or dropout rates over time or across schools, districts, or states. 8 See, for example, Jay P. Greene Graduation Rates for Choice and Public School Students in Milwaukee. Manhattan Institute for Policy Research and School Choice Wisconsin; John Robert Warren State-Level High School Completion Rates: Concepts, Measures, and Trends. Education Policy Analysis Archives (Retrieved March 13, 2006 from Christopher B. Swanson and Carole Bausell Diplomas Count: Ready for What? Preparing Students for College, Careers, and Life after High School. Bethesda, MD: Editorial Projects in Education. 9 National Research Council and National Academy of Education High School Dropout, Graduation, and Completion Rates: Better Data, Better Measures, Better Decisions. Committee for Improved Measurement of High School Dropout and Completion Rates: Expert Guidance on Next Steps for Research and Policy Workshop. Robert M. Hauser and Judith Anderson Koenig, Editors. National Research Council and National Academy of Education. Center for Education, Division of Behavioral and Social Sciences and Education. PAGE 4

6 LONGITUDINAL MEASURES The best way to measure graduation rates involves longitudinal tracking systems that monitor the school enrollment, school completion, and geographic location of every incoming high school student from the day they enter high school until the time they permanently leave the secondary schooling system (regardless of how, when, or where they do so). Although such systems provide precise information about the share of incoming high school students who go on to graduate, in practice they are difficult and expensive to implement. One reason is that school districts have a very difficult time monitoring the activities of students who leave their jurisdictions or who repeatedly leave and re-enter. As noted above, this situation should improve as states move to the federally-mandated use of longitudinal student tracking systems and rigorous, standardized methods for reporting graduation rates at the state, district, and school-house level. In the meantime, we believe that it is useful to make use of existing data to best understand patterns and trends in Milwaukee s graduation rates. One alternative technique for measuring Milwaukee s graduation rate involves comparing the number of graduates from Milwaukee high schools in the spring of calendar year X to the number of students who entered those schools for the first time as ninth graders in the fall of calendar year X-4. If none of those ninth graders die, if none are made to repeat any high school grades, and if the number of students who move away from Milwaukee is exactly equal to the number of new students who move into Milwaukee, then this simple technique yields precisely accurate information about Milwaukee s high school graduation rate. Of course, some ninth graders do die, some are made to repeat grades, and net migration rates are not exactly equal to zero. Thus, a simple comparison of the number of Milwaukee high school graduates in the spring of calendar year X to the number of enrolled ninth graders in the fall of calendar year X-4 is potentially biased by (1) mortality; (2) grade retention; and (3) net migration. A number of researchers have developed techniques for statistically adjusting for these sources of bias 10 in such measures, with varying degrees of success. As demonstrated elsewhere, the most accurate and empirically valid of these adjustment techniques is Warren s Estimated Completion Rate (ECR). 11 However, computing the ECR for the Milwaukee schools is not technically possible. For one thing, it requires precise information about mortality rates and net migration rates among school-aged individuals, separately for those attending MPCP and MPS schools. It also requires information about the number of first time ninth graders (as opposed to the total number of ninth graders) in the fall of calendar year X-4. This information is not readily or completely available for the Milwaukee schools. As described in the Appendix, the methodology for this study makes reasonable adjustments to account for such factors in estimating MPS and MPCP graduation rates. 10 See Jay P. Greene, and Marcus A. Winters Public High School Graduation and College-Readiness Rates: New York: Center for Civic Innovation, Manhattan Institute. Walt Haney, George Madaus, Lisa Abrams, Anne Wheelock, Jing Miao, and Ileana Gruia The Education Pipeline in the United States, Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College. Marilyn Seastrom, Lee Hoffman, Chris Chapman, and Robert Stillwell The Averaged Freshman Graduation Rate for Public High Schools From the Common Core of Data: School Years and NCES Washington, D.C.: U.S. Department of Education, National Center for Education Statistics. Christopher Swanson and Duncan Chaplin Counting High School Graduates When Graduates Count. Washington, D.C.: Education Policy Center, Urban Institute. 11 See John Robert Warren State-Level High School Completion Rates: Concepts, Measures, and Trends. Education Policy Analysis Archives (Retrieved March 13, 2006 from PAGE 5

7 The estimated graduation rate in MPCP schools for % -- compares with the MPS rate for that year of 70%. Table 1 presents MPCP and MPS graduation rate estimates for the seven-year period of through Estimates for the first six years were reported in our previous 2009 study; estimates for are new. As discussed in the Appendix, Table 1 assumes: (1) a 25% ninth grade retention rate for MPS students; (2) a 5% ninth grade retention rate for MPCP students; and (3) no net migration among either MPS or MPCP students. TABLE 1. Graduation Rates after Adjusting for Ninth Grade Retention (Schools with complete data for all years) UPDATED MILWAUKEE RESULTS MPCP SCHOOLS (Assuming 5% Ninth Grade Retention Rate) MPS SCHOOLS (Assuming 25% Ninth Grade Retention Rate) Graduation Rate 63% 61% 62% 64% 87% 77% 82% 49% 67% 54% 57% 60% 65% 70% Adjusted 9th Graders ,591 4,928 5,333 5,618 5,613 5,462 4,773 Observed 9th Graders ,455 6,570 7,110 7,491 7,484 7,282 6,364 Graduates ,741 3,326 2,906 3,229 3,380 3,567 3,320 Number of Schools The new estimate in Table 1 shows that the estimated graduation rate in MPCP schools for % compares with the MPS rate for that year of 70%. Overall, the MPCP rate was higher than the MPS rate in six of the seven years. As shown in Figure 1, both the MPS and MPCP graduate rates have been gradually trending upward for the past seven academic years. PAGE 6

8 100% 80% 60% 40% 20% 0% FIGURE 1. Graduate Rates in MPS and MPCP Schools, MPS MPCP TABLE 2. Hypothetical Number of MPS Graduates Based on MPCP Graduation Rates What if the MPS graduation rate in these seven years had been equal to the rate for high school students participating in the MPCP? If, for example, the 4,773 MPS ninth graders at risk of graduating in 2009 had graduated at the MPCP rate of 81.9%, we would have observed 3,908 MPS graduates in that year an increase of 588 over what we actually observed. Table 2 shows that a cumulative total of 3,939 additional MPS students would have graduated under that scenario over the six years Total % Chg. Actual MPS Graduates 2,741 3,326 2,906 3,229 3,380 3,567 3,320 22,469 n/a MPS Graduates if at MPCP Rates 3,514 2,997 3,308 3,609 4,857 4,216 3,908 26,408 n/a Difference 773 (329) , , % CONCLUSION This report utilizes an intuitively simple technique for calculating high school graduation rates based on a comparison of the number of regular diploma recipients in one year to the number of ninth graders three academic years earlier. This study updates our prior comparisons of MPS and MPCP schools to include data for the graduating class of After making reasonable assumptions about grade retention and net migration rates in MPCP and MPS schools, we find that graduation rates are higher in MPCP schools than in MPS schools in six of the seven years we consider. By law, students who participate in the MPCP are from lowerincome families. 12 Students in MPS schools come from a much broader range of social and economic backgrounds. Given the well-documented relationship between socioeconomic background and high school completion rates, this fact suggests that we ought to observe lower high school completion rates among students in MPCP schools. On the other hand, families who are sufficiently motivated to make use of vouchers and to send their children to MPCP schools may be different from other families in such a way that would lead us to expect higher graduation rates among students in MPCP schools. Whether the higher MPCP graduation rates are causal in nature that is, whether these higher graduation rates are due to something real that is going on in MPCP schools is a question that can only be addressed using a stronger research design. In this regard, we eagerly await further results of the SCDP longitudinal study being conducted at the University of Arkansas. This project, which involves matched pairs of students in MPS and MPCP schools will help address this question of causality. 12 In , income eligibility to enter the MPCP was $37,439 for a family of four. PAGE 7

9 APPENDIX The basic initial findings hold even under reasonable assumptions about grade retention and net migration. METHODOLOGY As in our earlier reports, this study began by producing graduation rate estimates for students in MPS and those participating in the MPCP by using the simple comparison of the number of high school graduates in the spring of calendar year X to the number of enrolled ninth graders in the fall of calendar year X-4. Cognizant of the potential biases inherent in these measures, however, this study then examined detailed evidence about the likely direction and magnitude of the grade retention and net migration biases for our major findings. 13 As shown below, the basic initial findings hold even under reasonable assumptions about grade retention and net migration. Information on MPS enrollment counts and numbers of graduates comes from the MPS Official State Aids (School Enrollment) Report and the Wisconsin School Performance Report. 14 Similar information was obtained for private schools participating in the MPCP program from surveys of those schools. 15 Based on the resulting data, for each of seven consecutive academic years ( through ) this study estimates high school graduation rates by taking the sum of all graduates in each academic year and then dividing by the sum of all ninth grade enrollments three academic years earlier (in though ). In the analyses described in Appendix Table 1A, a school was only included if both ninth grade enrollment in Year X-3 and graduation data in Year X were available for that school. Across the seven years, data were available for between 27 and 50 public high schools, MPS charter schools, and partnership schools in Milwaukee. The partnership schools are schools that operate under contract with MPS and focus on at-risk students. Including charter and partnership schools is essential because they are all part of the public school system. Excluding them would artificially inflate the graduation rate by excluding schools that the public system creates or employs specifically to educate some of their students who are at greatest risk of failing to graduate. Data were also available for between 9 and 14 private schools in the MPCP program across the seven years, including virtually all participating schools with a substantial number of high school students. 13 Mortality rates are (fortunately) generally very low among high school aged people, and are also unlikely to differ much between students attending MPS and MPCP schools. Thus, we do not discuss the role of mortality in biasing our results. 14 The MPS Office of Student Services calculates the district s Official State Aids Report every year during the district s official third Friday in September enrollment count. Published by the Wisconsin Department of Public Instruction in May for the prior school year, the Wisconsin School Performance Report serves as the state s annual public school report card. See 15 Researcher Michael Ford of School Choice Wisconsin has conducted surveys of MPCP schools under my direction and review for this report. PAGE 8

10 Data for individual schools are not reported at the school level. This is because the characteristics of the student body at individual schools likely are not representative of overall MPS or MPCP enrollment. As a result, such raw data disaggregated at the school level would not speak either to how well an individual school is performing or to how the MPS or MPCP programs are performing. In the analyses described in Appendix Table 1B, in contrast, all graduates in Year X are counted in the numerator of our rate (regardless of whether we have information about the number of ninth graders in corresponding schools in Year X-3). Likewise, all ninth graders in Year X-3 are counted in the denominator (regardless of whether we have information about the number of graduates from corresponding schools in Year X). That is, Appendix Table 1A is restricted to students in schools for which we have complete information; Appendix Table 1B includes all ninth graders and graduates in each year, regardless of what else we know about their schools. Appendix Tables 1A and 1B report the number of ninth grade students in through and the number of high school graduates in through , separately for MPCP and MPS schools. Using the information in Appendix Table 1A for (the most recent year for which complete data are available), the estimated graduation rate in MPS schools is 4,342/7,732 = 56%. The estimated graduation rate for MPCP schools in that year equals 399/612 = 65%. In general, the estimates in Appendix Table 1A do not differ very much from those in Appendix Table 1B. APPENDIX TABLE 1A. Graduation Rates by Year (All available schools) MPCP SCHOOLS MILWAUKEE PUBLIC SCHOOLS Graduation Rate 64% 58% 53% 53% 71% 66% 65% 36% 52% 43% 47% 50% 54% 56% 9th Graders ,246 7,362 8,247 7,999 7,817 8,077 7,732 Graduates ,969 3,836 3,583 3,792 3,898 4,379 4,342 Number of Schools APPENDIX TABLE 1B. Graduation Rates by Year (All available students) MPCP SCHOOLS MILWAUKEE PUBLIC SCHOOLS Graduation Rate 64% 59% 53% 53% 78% 70% 67% 35% 51% 43% 45% 49% 56% 53% 9th Graders ,451 7,547 8,313 8,841 8,724 8,477 8,735 Graduates ,969 3,874 3,583 4,002 4,254 4,729 4,661 PAGE 9

11 Because the numbers of schools in our estimates differ across academic years, it would not be technically appropriate to compare graduation rates in Appendix Tables 1A and 1B across years (e.g., to compare the MPCP graduation rate in to the rate in ). To facilitate such a comparison, Appendix Table 2 restricts the schools in our analysis to those with complete information for all six academic years under consideration. This limits the scope to 23 MPS schools and 7 MPCP schools. Appendix Table 2 presents graduation rate estimates based on these subsets of schools. APPENDIX TABLE 2. Graduation Rates by Year (All available students) MPCP SCHOOLS MILWAUKEE PUBLIC SCHOOLS Graduation Rate 60% 58% 59% 61% 82% 73% 78% 37% 51% 41% 43% 45% 49% 52% 9th Graders ,455 6,570 7,110 7,491 7,484 7,282 6,364 Graduates ,741 3,326 2,906 3,229 3,380 3,567 3,320 Number of Schools PAGE 10

12 CORRECTING FOR POTENTIAL METHODOLOGICAL PROBLEMS The graduation rate estimates in Appendix Tables 1A, 1B, and 2 are potentially biased by (1) mortality; (2) grade retention; and (3) net migration. Below we discuss the possible magnitude of grade retention and net migration and assess the extent to which they may affect our conclusion that graduation rates are higher in MPCP schools than in Milwaukee public schools. GRADE RETENTION. Imagine that there are 100 ninth graders in a school in the fall of calendar year X-4 and that 90 students graduate from that school in the spring of calendar year X. We would conclude based on the methodology used above that the graduation rate for the school equals 90/100 or 90%. But what if 10 of the 100 ninth graders in the fall of calendar year X were repeating the ninth grade, such that only 90 of the 100 ninth graders were attending the ninth grade for the first time? These 10 repeat ninth graders would be counted in the denominator of our graduation rate calculation in two different years, but in the numerator a maximum of once. As a result, our estimated graduation rate 90/100 = 90% would be downwardly biased. In fact, the denominator should exclude the 10 students who are repeating the ninth grade, and the correct graduation rate is thus 90/90 = 100%. NET MIGRATION. Imagine again that there are 100 ninth graders in a school in the fall of calendar year X-4 and that 90 students graduate from that school in the spring of calendar year X. As before, using our methodology we would conclude that the graduation rate equals 90/100 or 90%. But what if 50 more students moved into the school than left the school (through death or migration) between the ninth and 12th grades? In this case, there would be 150 not 100 prospective graduates and an unbiased graduation rate would equal 90/150, or 60%. The lesson is that simple graduation rates like those in Appendix Tables 1A, 1B, and 2 are downwardly biased by grade retention, upwardly biased by net in-migration, and downwardly biased by net outmigration. 16 Given that these appendix tables do not account for grade retention or net migration, how likely is it that these biases seriously affect the major conclusions? The answer to that question is examined in Appendix Table 3, which illustrates, through nine scenarios, how graduation rates for MPCP and MPS for would change under a variety of assumptions about grade retention rates and net migration. Using the numbers of ninth graders in and graduates in as reported in Appendix Table 2, Appendix Table 3 reports graduation rates when grade retention rates vary between 5% and 25% and total net migration rates vary between -10% and +10%. Appendix Table 3 accomplishes this by adjusting the number of ninth graders in our graduation rate calculations from Appendix Table 2 to reflect the number of individuals who would actually be candidates for graduating from high school under these nine scenarios. 16 Here we can count death as a form of out-migration. PAGE 11

13 APPENDIX TABLE 3. Estimated Graduation Rates for under Different Assumptions About Grade Retention And Net Migration Rates MPCP SCHOOLS Ninth Grade Retention Rate MPS SCHOOLS Ninth Grade Retention Rate 5% 15% 25% 5% 15% 25% NET MIGRATION RATE -10% 91% 102% 115% 61% 68% 77% 0% 82% 92% 104% 55% 61% 70% +10% 74% 83% 94% 50% 56% 63% For both MPCP and MPS schools, the highest estimated graduation rates are observed if we assume net out-migration (-10%) and fairly high rates of ninth grade retention (25%). Conversely the lowest estimated graduation rates are observed if we assume net in-migration (+10%) and fairly low rates of ninth grade retention (5%). The most important point to be made about Appendix Table 3 concerns the range of possible graduation rates for MPCP and MPS schools. It is certainly true that measures like those presented in Appendix Tables 1 and 2 are subject to systematic bias due to their failure to account for grade retention, net migration, and mortality. But how large are those biases in this case? And how do they affect our comparison of MPCP and MPS school graduation rates? For MPCP schools, the unbiased graduation rate for is almost certainly at least 74% (assuming that net migration falls within the range of +/- 10%). Any estimate lower than 74% would have to assume even more extreme net migration rates than those included in Appendix Table 3. In contrast, for MPS schools the unbiased graduation rate for almost certainly falls somewhere between 50% and 77%. An important lesson from Appendix Table 3 is that the true difference in graduation rates between MPCP and MPS schools depends on their respective ninth grade retention and PAGE 12

14 net migration rates. However, the only scenario in which we might conclude that MPS graduation rates are higher than MPCP graduate rates is one in which we make the most extreme of assumptions about grade retention and net migration rates. Such a conclusion would have to be based on evidence that the simple MPS and MPCP graduation rates like those presented in Appendix Table 2 are both highly biased but in opposite directions. Although there is little direct evidence on this point, there is some reason to suspect that MPCP schools have much lower grade retention rates than MPS schools. Whereas the MPS schools official ninth grade retention rate has consistently been between about 20% and 25% in recent years, MPCP school administrators report that ninth grade retention is quite rare in their schools. The figures at our disposal are consistent with large differences between MPS and MPCP schools with respect to ninth grade retention rates. If we simply compute the ratio of the number of ninth graders in one year to the number of tenth graders in the following year a very crude technique for estimating the proportion of ninth graders in one year who go on to the tenth grade the following year we observe ratios of about 1.1 for most MPCP schools and about 1.5 for most MPS schools. 17 That is, in the schools included in our graduation rate calculation there were usually about 10% more ninth graders in 2005 than there were tenth graders in In contrast, in the MPS schools in our graduation rate calculation there were typically about 50% more ninth graders in 2005 than there were 10th graders in Returning to Table 3, this means that the most accurate graduation rate estimates for for MPCP schools are probably those that assume a 5% grade retention rate. The most accurate graduation rate estimates for for the MPS schools are probably those that assume a 25% grade retention rate. Even allowing for these large differences in ninth grade retention rates, we still observe that high school graduation rates in in MPCP schools are at least as high as those observed in MPS schools. In order to conclude that graduation rates are actually the same or lower in MPCP schools than in MPS schools we would have to assume that the two groups of schools experience very different rates of net migration. Specifically, we would have to assume that the graduation rate estimates for MPCP schools are upwardly biased by high net in-migration and/or that the graduation rate estimates for the Milwaukee public schools are downwardly biased by high net out-migration rates. While MPCP and Milwaukee public schools appear to experience very different ninth grade retention rates, we see no evidence that they experience markedly different net migration rates. In the end it seems that part but clearly not all of the differences between MPCP and MPS school graduation rates in Appendix Tables 1 and 2 can be accounted for by differences in ninth grade retention rates. 17 These ratios should not be taken as good estimates of rates of ninth grade retention. For one thing, they conflate ninth grade retention and high school dropout. We present them only to suggest that the data at our disposal are consistent with large disparities in ninth grade retention rates between MPCP schools and MPS schools. PAGE 13

15 Cutting the dropout rate in half would yield $45 billion annually in new federal tax revenues or cost savings. - Columbia University Teachers College. ABOUT THE AUTHOR John Robert Warren is Professor and Director of Undergraduate Studies in the Department of Sociology at the University of Minnesota Twin Cities. Dr. Warren received his doctorate in 1998 from the University of Wisconsin-Madison. His published scholarship reflects extensive examination of issues associated with the accurate measurement of high school graduation rates. In State-Level High School Completion Rates: Concepts, Measures, and Trends, he provides a comprehensive review of those issues and validates a rigorous new method for accurately calculating graduation rates (see Education Policy Analysis Archives, Vol. 13, No. 51, December 23, Dr. Warren participates with scholars at the University of Wisconsin-Madison in the Wisconsin Longitudinal Study (WLS), an extensive long-term research project involving graduates of Wisconsin high schools in 1957 (see He also is a recognized expert in the emerging issue of state high school exit examinations and their consequences. School Choice Wisconsin is a nonprofit organization that seeks to ensure an honest debate about school choice by providing accurate information on the impact of school choice on families, communities, and schools. Headquartered in Milwaukee, home of the nation s largest and oldest voucher program for low-income families, School Choice Wisconsin supports expanded educational options for parents through the use of school vouchers, charter schools, and innovative public-private partnerships. PAGE 14

16 Simply put, the world has changed and there is no work for high school dropouts. - Robert Balfanz, Ph.D., Johns Hopkins University 2025 N. Summit Avenue Milwaukee, WI (414)