Evaluating and Reforming Urban Schools

Transcription

1 Evaluating and Reforming Urban Schools Holger Sieg University of Pennsylvania November 2, 2015

2 The U.S. Education System The U.S. system of public education is generally perceived as in need of some serious reform by most measures. Real dollar spending per student for K-12 public education has risen from $6,650 (2007 dollars) in 1970 to $11,286 in Student achievement performance in reading and math by age 17 has remained essentially flat since that date. International comparison of test performances in mathematics and science shows that U.S. students typically rank in the bottom quartile of test performance. The U.S. is spending more resources; getting nothing more in learning for the effort; and falling behind the rest of world in math and science. Not an enviable record. The purpose of this lecture is to discuss how we can empirical methods to determine which reform options are promising.

3 Example: Pittsburgh Public Schools Allegheny county has a population of 1,281,669 persons in The City of Pittsburgh has a population of 334,563. The racial make-up was 67 percent White and 27 percent African American. The county has 130 municipalities and 42 school districts, each governing itself. In 2007 Pittsburgh Public Schools operated 65 schools with 5,180 full-time employees and serves 29,445 students with an operating budget of $533.6 million.

4 Enrollment: PIT vs PHL

5 Expenditures: PIT vs PHL

6 Student Teacher Ratio: PIT vs PHL

7 Grade 11 Reading: Increased 2.4 points (4.8%) from 2009 and decreased 0.6 points (1.1%) since Advanced: Increased 3.2 points (13.8%) from 2009 and 5.9 points (28.7%) since

8 Grade 11 Mathematics Proficiency: Decreased 0.3 points (0.7%) from 2009 and 1.4 points (3.2%) since Advanced: Increased 6.1 points (36.3%) from 2009 and 4.3 points (23.2%) since

9 Policy Reform Options Early childhood education programs. Competition within the public school system: magnet schools. Competition among public schools: charter schools. Private school competition and school choice: vouchers. Gifted Education

10 Early Childhood Education The HighScope Perry Preschool Project provided early childhood education for disadvantaged children in In contrast to existing nursery school programs, the Perry program focused on a child s intellectual maturation rather than a child s social and emotional advances. It was evaluated in a randomized controlled trial of 123 children (58 were randomly assigned to a treatment group). The Rate of Return to the Highscope Perry Preschool Program, by James J. Heckman, Seong Hyeok Moon, Rodrigo Pinto, Peter A. Savelyev and Adam Yavitz; Journal of Public Economics, They provide empirical evidence which suggests that educational, employment and other outcomes for the children receiving the program were slightly better to outcomes for the children not receiving the program. The biggest social returns come from differences in crime (small sample).

11 Magnet Schools Magnet schools are public schools with specialized courses or curricula. Magnet refers to how the schools draw students from across the normal school zones that feed into regular public schools. Magnet schools emerged in the United States in the 1960 s as one means of remedying racial segregation in public schools. Some magnet schools have a competitive entrance process, requiring an entrance examination, interview, or audition. Other magnet schools select all students who apply or use a lottery system, or a system combining some elements of competitive entrance and a lottery. Most magnet schools concentrate on a particular discipline or area of study, while others such as International Baccalaureate schools have a more general focus.

12 Oversubscribed Magnet Programs Many of the most popular magnet programs are oversubscribed. School district runs a variety of lotteries to determine access. A lottery can be viewed as randomized experiment, where randomization occurs at the admission stage, and not at the participation stage. The lottery is binding in the sense that students with lower numbers are accepted, higher numbered students are rejected

13 Lotteries as a Quasi-Experimental Design If the lotteries are fair and balanced, the randomization should create treatment and control groups that have similar observed and unobserved characteristics. Suppose that individuals comply with the intended treatment: all students that win the lottery attend the magnet school and all students that lose the lottery do not attend a magnet school. From the lottery winners, we obtain: E[Y D = 1, W = 1] = E[Y 1 D = 1] (1) From he lottery losers, we obtain E[Y D = 1, W = 0] = E[Y 0 D = 1] (2) Hence lotteries give rise to a quasi-experimental design with proper randomization.

14 Effectiveness of Oversubscribed Magnet Schools Engberg, J., Epple, D., Imbrogno, J., Sieg, H. and R. Zimmer (2013), Evaluating Education Programs that have Lotteried Admission and Selective Attrition. Journal of Labor Economics. They study the effectiveness of magnet programs in Pittsburgh. They find that magnet programs are effective tools to attract and retain middle class households in urban districts. However, the differential retention rates create a differential attrition problem which makes it more difficult to evaluate the impact of the program on other outcomes such as student achievement. Overall the effect on achievement seem to be small.

15 Effectiveness of Oversubscribed Charter Schools Charter Schools are part of the public education system and are not allowed to charge tuition. In exchange these schools are accountable for student achievement. But they are not subject to some of the rules, regulations, and statutes that apply to other public schools. There are currently 80 charter schools in Philadelphia. A. Abdulkadiroglu, J. Angrist, S. Dynarski, T.J. Kane, and P.A. Pathak (2011), Accountability and Flexibility of Public Schools: Evidence from Boston s Charters and Pilots. Quarterly Journal of Economics. Lottery estimates show large and significant score gains for charter students in middle and high school. Charter schools with binding assignment lotteries appear to generate larger gains than other charters.

16 The Effectiveness of Vouchers Educational vouchers provide a fixed amount of money given by the government to families with school-age children, who can spend it at any type of school, public or private. Vouchers put private schools and public schools on equal footing. Rouse (1998) studies the Milwaukee voucher program. She focuses on oversubscribed schools that randomly admit students. Vouchers increased test scores by 1-2 percentage points. Recent evidence suggests that vouchers have a more substantial effect on high school graduation rates. Angrist et al. (2002) focus on Columbian vouchers distributed by lottery. Voucher winners 10 percent more likely to finish 8th grade, scored higher on standardized tests.

17 Gifted Programs Gifted programs have the dual objective of engaging and challenging gifted students to reach advanced levels of achievement as well as attracting and retaining students who might otherwise leave for suburban or private schools. Specialized lab schools, such as Masterman in Philadelphia, are also gifted programs. Gifted programs often employ IQ thresholds for admission, with those above the threshold being admitted. Assignment based on thresholds is not random, nevertheless we can use these thresholds to evaluate the program. The intuition is that participants slightly above the threshold and non-participants slightly below the threshold are likely to be similar.

18 Sample District (K-12) Tested Admitted Male African American Subsidized Lunch Income (11153) (13967) (16058) IQ (12.2) (7.7) Math (221) (177) (182) Reading (217) (163) (150) Offenses (1.884) (0.950) (0.521) 1 year retention year retention Standard deviations in parentheses.

19 Figure: Proportion Gifted by Max(FSIQ,GAI)

20 The Regression Discontinuity Design This is another quasi-experimental design that is popular to evaluate educational interventions. Let Z denote the observed IQ score. The probability to participate in the program is defined as: E[D Z] = Pr{D = 1 Z} (3) The RD design is then based on the assumption that Pr{D = 1 Z} is known to be discontinuous at Z 0. The basic idea is then to compare outcomes of individuals who are just below and just above the cut-off value.

21 Potential Outcome Recall that: then we obtain Y = Y 0 + (Y 1 Y 0 ) D = Y 0 + D (4) E[Y ] = E[Y 0 ] + E[ D] (5) If is constant among individuals that receive the treatment, then we have: E[Y ] = E[Y 0 ] + E[D] (6)

22 Local Treatment Effects Let e > 0 denote an arbitrarily small positive number. Then we have in the constant treatment case: E[Y Z = Z 0 + e] E[Y Z = Z 0 e] = (E[D Z = Z 0 + e] E[D Z = Z 0 e]) + E[Y 0 Z = Z 0 + e] E[Y 0 Z = Z 0 e] If we assume that E[Y 0 Z] is continuous at Z 0, the second term in the equation above vanishes in the limit. As a consequence, the treatment effect is identified from the following equation: = lim e 0 E[Y Z = Z 0 + e] E[Y Z = Z 0 e] E[D Z = Z 0 + e] E[D Z = Z 0 e] (7)

23 From a Broader Perspective The important question is What is good for students? not What is good for school districts?. Movement of more affluent students up the district income hierarchy may be beneficial if such relocations provide access to better educational opportunities. If peer effects are important, the departure of high SES or high ability students from urban schools adversely affect those who remain. This is the classic peer effect dilemma. The current decentralization of districts provides no mechanism for assessing or addressing such tradeoffs. As a consequence, the U.S. primary and secondary education system is highly segregated based on SES status, race, and ethnicity, which results in unequal access to economic opportunities.