Li Feng Texas State University-San Marcos. David Figlio Northwestern University and NBER. Tim Sass Georgia State University.

Transcription

1 SCHOOL ACCOUNTABILITY AND TEACHER MOBILITY* Li Feng Texas State University-San Marcos David Figlio Northwestern University and NBER Tim Sass Georgia State University April 12, 2013 Abstract We exploit a 2002 change in Florida's school accountability system to measure the effects of accountability pressure on teacher mobility in Florida. We find strong evidence that teachers are more likely to leave schools that have received a failing grade. Receipt of an "F" grade has larger effects on teacher mobility in middle and high school than in elementary school. The effects are also more pronounced for teachers of core academic subjects than for PE, music and art teachers. Schools designated as failing are more likely to lose their highest quality and middling teachers than their lowest performing teachers. *This is a considerably updated version of a paper with the same name that first appeared as NBER working paper We are grateful to the U.S. Department of Education (via the National Center for the Analysis of Longitudinal Data in Education Research), National Science Foundation and National Institutes of Child Health and Human Development for research support and to the Florida Department of Education for providing the data for this analysis. We also wish to thank seminar participants at Indiana, Northwestern, Oregon, Wisconsin, and the Swedish Research Institute for Industrial Economics, as well as conference participants at the American Education Finance Association, Association for Public Policy Analysis and Management, Society for Research on Educational Effectiveness, and Southern Economic Association meetings, for helpful comments. We alone are responsible for any errors in analysis or interpretation. The results reported herein do not necessarily reflect the views of the Florida Department of Education or of our funders. 1

2 I. Introduction School accountability -- the process of evaluating schools based on performance of their students and holding the schools responsible for student outcomes -- is becoming increasingly prevalent around the world. Accountability systems typically provide direct incentives in the form of explicit rewards or sanctions associated with student performance. In addition, accountability systems may engender social pressure, since a school s constituents have both educational and financial 1 reasons to influence low-performing schools to improve. There exists considerable evidence that schools are changing as a result of accountability, but the evidence regarding the effects on teachers the people charged with carrying out school policies and practices -- is extremely limited. This paper makes use of detailed individual teacher-level data from Florida to gauge the degree to which these direct and indirect forms of accountability pressure affect the occupational choices of teachers. There is strong reason to believe that accountability pressure influences the ways in which educators carry out their jobs. The weight of the evidence suggests that accountability systems tend to improve the outcomes of low-performing students (see, e.g., Ballou and Springer, 2008; Carnoy and Loeb, 2002; Chakrabarti, 2007; Chiang, 2009; Dee and Jacob, 2011; Figlio and Rouse, 2006; Hanushek and Raymond, 2004; Ladd and Lauen, 2009; Reback, Rockoff and Schwartz, 2011; Rockoff and Turner, 2010; Rouse et al., 2013; West and Peterson, 2006; Wong, Cook and Steiner, 2010), implying that these systems are changing the ways in which schools do business. 2 Rouse et al. (2013) document a number of the ways in which 1 School accountability ratings are capitalized into housing prices (Figlio and Lucas, 2004), which in turn affect the property tax base for schools, and they affect a school s ability to raise voluntary contributions (Figlio and Kenny, 2009). 2 Craig, Imberman, and Perdue (2013) identify some ways in which school accountability systems influence school resource allocations. 2

3 accountability pressure has changed school instructional policies and practices in Florida s lowperforming schools, and relate these instructional policy and practice changes to increased student performance. The same pressures to improve efficiency also may lead to other changes in the school environment; Booher-Jennings (2005), Krieg (2008), Neal and Schanzenbach (2010), Ozek (2010) and Reback (2008) show that schools subject to accountability pressure tend to concentrate attention on some students at the apparent expense of others. Some schools have responded by differentially reclassifying low-achieving students as learning disabled so that their scores will not count against the school in accountability systems (see, e.g., Cullen and Reback, 2007; Figlio and Getzler, 2007; Jacob, 2005). 3 Figlio and Winicki (2005) suggest that Virginia schools facing accountability pressures altered their school nutrition programs on testing days to increase the likelihood that students will do well on the exams, and Figlio (2006) indicates that schools differentially suspend students at different points in the testing cycle in an apparent attempt to alter the composition of the testing pool. Jacob and Levitt (2003) find that teachers are more likely to cheat when faced with increased accountability pressure. With school accountability changing the ways in which schools are operating, it seems natural to believe that these systems would influence the teacher labor market. School accountability systems may influence the desirability of certain teaching jobs, and may also affect the willingness of schools to retain certain teachers. From a theoretical perspective, the effects of accountability pressures on the teacher labor market are ambiguous. On the demand side, in order to avoid sanctions and/or the stigma associated with being designated as a failing school, schools could increase their efforts to identify low performing teachers and remove them from their classrooms. On the supply side, accountability pressure and associated changes in 3 Chakrabarti (2007), however, does not find that schools respond in this way. 3

4 school policies could lower the net benefit of teaching in a school by reducing teacher discretion over curriculum or teaching methods. Likewise, the potential stigma from teaching in a failing school could lead some teachers to seek employment at other schools. On the other hand, the resources that often accompany sanctions (e.g. reading coaches, enhanced training for teachers, etc.) could reduce the non-monetary costs associated with working in low-performing schools and actually increase teacher retention. A number of recent papers have analyzed the determinants of teacher mobility and attrition (Boyd et al., 2005, 2006; Feng, 2009; Hanushek et al., 2004; Imazeki, 2005; Jackson, 2012, forthcoming; Krieg, 2006; Podgursky et al., 2004; Scafidi et al. 2007). However, the literature relating recent public policy changes regarding teachers, such as accountability pressures, to teachers labor market decisions has been much spottier. Boyd et al. (2008) explore the responses of teachers to the introduction of mandated state testing in New York State. They find that teacher turnover in fourth grade, the critical accountability year in New York, decreased following the introduction of testing, and that entrants into fourth grade were more likely to be experienced teachers than had previously been the case. Clotfelter et al. (2004) evaluate how North Carolina s accountability system influenced the ability of schools serving low-performing students to attract and retain high-quality teachers. They find that the introduction of the accountability system exacerbated teacher turnover in these schools, though it is less evident that accountability led to lower qualifications of the teachers serving low-performing students. Both of these papers carefully describe the accountability systems in their states, but because they evaluate accountability systems that affected all schools within a state, it is difficult to derive causal inference from their analyses. 4

5 In this paper, we exploit a major rule change in Florida s school accountability system that took place in the summer of 2002 to identify the effects of changing school accountability pressures on teacher mobility between schools and occupations. Florida had graded every school in the state on a scale from A to F since the summer of 1999, based on proficiency rates in reading, writing and mathematics. Florida s system of school accountability with a series of rewards and sanctions for high-performing and low-performing schools, called the A+ Plan for Education, has become a model for the rest of the United States, with a number of states and localities, ranging from Arizona to Indiana to North Carolina to New York City 4, adopting accountability systems that mirror many key features of the policy. In 2002, the state dramatically changed its grading system to both recalibrate the acceptable student proficiency levels for the purposes of school accountability and to introduce student-level changes in test scores as an important determinant of school grades. Using student-level micro-data to calculate the school grades that would have occurred absent this change, we demonstrate that over half of all schools in the state experienced an accountability shock due to this grading change, with some schools receiving a higher grade than they would have otherwise received and other schools receiving a lower grade than would have otherwise occurred. Furthermore, some schools were shocked downward to receive a grade of F, which no school in the state had received in the prior year of grading. These grading shocks provide the vehicle for identification of accountability effects in this paper. Given that school grades were based on explicit thresholds in the underlying student performance metrics, we are able to employ regression discontinuity techniques in order to 4 In addition to Arizona, Indiana, North Carolina, and New York City, other states recently adopting school grading systems modeled after Florida s include Alabama, Louisiana, New Mexico, Ohio, South Carolina, and Utah. 5

6 determine the effects of accountability shocks on teacher mobility. We find that schools which just fell into the F category under the revised school grading scheme experienced a discrete jump of 5 to 14 percentage points in the probability of teacher turnover relative to schools that just missed being branded as an F school. Teacher departures are roughly evenly split between transfers to other schools within the same district and exits from public school teaching. However, the impacts of accountability pressure on teacher movement are more pronounced among teachers in core academic subjects than in music, art and physical education. The general finding that receipt of an F school grade significantly boosts teacher mobility is robust to a variety of functional forms and estimation techniques. Inclusion of school fixed effects as well as a variety of controls for observed teacher, classroom, school, and district characteristics yields even higher estimated impacts on the rate of teacher mobility. Since Florida has had statewide achievement testing in all grades 3-10 since we are also able to compute value-added measures of teacher quality and determine whether receipt of an F tends to increase or decrease the mobility of high quality teachers at a school. We find that receipt of an F grade translates into differentially higher turnover for the best teachers at a school (measured by their contribution to student test scores). Given the important role of teacher quality in determining student achievement, our findings suggest that school accountability can have very consequential effects for both teachers and their students. II. The Florida School Accountability Program Florida s A+ Plan education reform called for annual curriculum-based testing of all students in grades three through ten, and annual grading of all traditional public and charter schools based on aggregate test performance. As noted above, the Florida accountability system 6

7 assigns letter grades ( A, B, etc.) to each school based on students achievement (measured in several ways). High-performing and improving schools receive rewards while low-performing schools receive sanctions as well as additional assistance, through Florida s Assistance Plus program. School grading began in May 1999, immediately following passage into law of the A+ Plan. In each year from 1999 through 2001, a school would earn a grade of F if fewer than 60 percent of students scored at level 2 (out of 5) or above in reading, fewer than 60 percent of students scored at level 2 (out of 5) or above in mathematics, and fewer than 50 percent of students scored at level 3 (out of 6) or above on the Florida Writes! writing evaluation, known from 2001 onward as the FCAT Writing examination. A school could avoid the F label by meeting any one of these three standards in 1999; the same was true in 2000 and 2001 provided that at least 90 percent of the test-eligible students took the examination (or that the school could provide the state with a reasonable explanation for why fewer than 90 percent of students took the test.) All schools in the distribution were subject to accountability pressure, and according to Goldhaber and Hannaway (2004), schools throughout the distribution apparently felt pressure to perform in measurable ways. Thus, between 1999 and the summer of 2001, schools were assessed primarily on the basis of aggregate test score levels (and also some additional non-test factors, such as attendance and suspension rates, for the higher grade levels) and only in the grades with existing statewide curriculum-based assessments, 5 rather than on progress schools make toward higher levels of student achievement. Starting in summer 2002, however, school grades began to incorporate test score data from all grades from three through ten, and for the first time depended on year-to-year 5 Students were tested in grade 4 in reading and writing, in grade 5 in mathematics, in grade 8 in reading, writing and math, and in 10 in reading, writing and math. 7

8 progress of individual students and not just on the level of student test performance. In our analysis, we take advantage of the fact that during the school year teachers would not have been able to anticipate their school grade in summer 2002 because of the changes in the formula and because the changes were not decided until the last minute. By the beginning of the school year several things were known about the school grades that were to be assigned in summer First, the school grades were to have been based on the test scores of all students in grades three through ten in reading and mathematics, and in the fourth, eighth and tenth grades in writing. Second, the standards for acceptable performance by students were to be raised from level 2 to level 3 in reading and mathematics. Third, some notion of a student-growth system was to be established, though little was known about the specific nature of this system except that it would augment the levels-based grading system and would focus principally on the lowest-performing students. These elements would be combined to give each school a total number of grade points. The school s grade would be determined by the number of points. However, the specifics of the formula that would put these components together to form the school grades were not announced until March 5, 2002, meaning that for most of the school year teachers had virtually no information with which to anticipate their school s exact grade. Table 1 shows the distribution of schools across the five performance grades for the first six rounds of school grading, for all graded schools in Florida. As is apparent from the variation across years in the number of schools that fall into each performance category, there are considerable grade changes that have taken place since the accountability system was adopted. Most notable is the fact that while 70 schools received an F grade in the first year ( ) only 4 did so the subsequent year and none did by the summer of At the same time, an 8

9 increasing number of schools were receiving grades of A or B. This is partly due to the fact that schools had learned their way around the system: A school had to fail to meet proficiency targets in all three subjects to earn an F grade so as long as students did well enough in at least one subject the school would escape the worst stigma. Hannaway and Goldhaber (2004) and Chakrabarti (2007) find evidence that students in failing schools made the biggest gains in writing, which is viewed as one of the easier subjects in which to improve quickly. When the rules of the game changed, so did the number of schools caught by surprise. For instance, 60 schools earned an "F" grade in the summer of The number of schools that received an A grade also increased, due in large measure to the shift to the grade points system of school grading, which allows schools that miss performance goals in one area to compensate with higher performance in another area. Finally, note that as schools have adapted to the new grading system, the number of failing schools has decreased over time. 6 In this paper, we seek to exploit the degree to which schools and teachers were surprised by the change in school grading. Using an approach to identification introduced by Figlio and Kenny (2009) and Rouse et al. (2013), we measure the accountability shock to schools and teachers by comparing the grades that schools actually received in the summer of 2002 with the grade that they would have been predicted to receive in 2002 based on the old grading system (that in place in 2001). We have programmed both the old and new accountability formulas and, using the full set of student administrative test scores provided us 6 Note that in Table 1 there are 68 elementary schools that received a grade of N in These were new schools in that year. As such, they were not given a formal grade although the state did calculate their accountability points. Rouse et al. (2013) experimented with imputing what their grades would have been, and found that there would have been an additional 9 "F" graded schools and 10 additional "D" graded schools, for instance, had the state graded these schools in Rouse et al. (2013) found that their results regarding test scores were completely insensitive to the treatment of these schools. For the purposes of the present analysis, we exclude them from the analysis of teacher job changes. 9

10 by the Florida Department of Education, we have calculated both the actual school grade that schools received in 2002 with the grade that they would have received given their realized student test scores had the grading system remained unchanged. It is essential that we make this specific comparison, rather than simply comparing year-to-year changes in school grades, because year-to-year changes in school grades could reflect not just accountability shocks, but also changes in student demographics, changes in school policies and practices, changes in school staffing, and other changes in school quality. Given that understanding school staffing is the point of this paper, it is clearly inappropriate to compare grade changes per se. Table 2 compares realized school grades to predicted school grades (based on the old grading system but the new student test scores) for the set of graded schools in the state of Florida. 7 We demonstrate that 51 percent of schools experienced a change in their school grade based on the changing parameters of the grading system itself. Most of these schools (42 percent) experienced an upward shock in their school grades, while 9 percent of all schools experienced a downward shock in their school grades, receiving a lower grade than they would have expected had the grading system remained unchanged. Twenty percent of schools that might have expected to receive a D under the old system received an F under the new one, while 38 percent of these schools received a grade of C or better. Meanwhile, 59 percent of schools that might have expected to receive a B under the old system received an A under the new one, while 9 percent of these schools received a grade of C or worse. It is clear that 7 The number of observations in Table 2 does not exactly match that in Table 1 because we rely on administrative data on students provided by the Florida Department of Education to simulate each school s grade in This administrative dataset does not include some students in charter schools, alternative schools, and schools that do not have any students in the accountability grades. 10

11 the grading system change led to major changes in the accountability environment, and provides fertile ground for identification. III. Data We are interested in modeling the effects of school accountability shocks on teacher mobility. The most natural way to estimate this relationship is to consider year-to-year changes in teacher employment at a school. Thus our primary dependent variable is the likelihood that a teacher in year t leaves his or her school before year t+1. We also consider specific types of job changes: movement from one public school to another in the same district, movement to a public school in another district and exit from public school teaching. Our key independent variables are indicators for whether the school was upward or downward shocked in the 2002 school grading regime -- a change in accountability pressure that is exogenous to the school and its teachers. We also wish to control for school and student body characteristics that can affect the occupational choices of teachers. Thus we require data that track teachers over time and link teachers to the students they instruct and the schools in which they work. To fulfill the data requirements we rely on data from the Florida Department of Education's K-20 Education Data Warehouse (FL-EDW), an integrated longitudinal database covering all Florida public school students and school employees from pre-school through college. The FL-EDW contains a rich set of information on both individual students and their teachers which is linked through time. Statewide data, as opposed to data from an individual school district, are particularly useful for studying teacher labor markets since we can follow teachers who move from one district to another within Florida. We cannot, however, track teachers who move to another state. Two factors minimize the problem of inter-state mobility in 11

12 Florida, however. First, Florida is surrounded on three sides by water and population density is low in the areas just outside Florida s northern border. Second, due to population growth and a constitutionally mandated maximum class size, Florida was a net importer of teachers until very recently, and Florida was booming economically during the period in question for this research. Thus, unlike many Northern states where the school-age population is shrinking, there is relatively little outflow of teachers from Florida during the period under study. 8 The FL-EDW contains data from the school year forward, though our analyses are based primarily on data through , the first year following the school accountability shock. Teachers mobility patterns are determined by the identity of the school they are teaching in each year. In some of our specifications we condition on student test performance (or measure teachers' value added based on student performance); student test score records for all grades 3-10 are only available from the school year forward. We also, in some specifications, control for classroom, school and district-level average student characteristics that contribute to a teacher s working conditions and thus may influence their decision to change schools or occupations. These student characteristics include achievement scores, disciplinary incidents, race/ethnicity, and economic status (participation in free or reduced-price lunch program). In addition, we control for teacher characteristics such as a teacher s age, gender, certification status, education level and salary. IV. Methods and Results A. Descriptive Evidence 8 Using national data from the Schools and Staffing Survey and associated Teacher Follow-Up Survey (SASS/TFS) which track teachers across state lines, Feng (2010) finds that there are relatively fewer teachers moving into or out of state of Florida compared to other Southern states, such as North Carolina and Georgia. 12

13 We begin by investigating the number of relevant teachers who faced different accountability conditions during the accountability shock of summer Table 3 presents a descriptive summary of teacher job change before and after the accountability shock, broken down by the type of shock a school received. 9 Independent of the type of accountability shock, teacher mobility was increasing over the time period. This general increase in teacher job change could be due to the general expansion in the number of teachers employed; relatively inexperienced teachers tend to be more mobile than veteran teachers. 10 Prior to the accountability shock, schools that would become downward shocked had the highest teacher departure rate (18.2 percent), with slightly lower rates in no-shock schools (16.6 percent) and upward-shocked schools (15.9 percent). There is little difference in the inter-temporal change in teacher job change between upward-shock and no-shock schools. The fraction of teachers leaving no-shock schools was 0.7 percentage points higher in the post-accountability-shock period and for upward-shock schools it was 1.0 percentage point higher. However, there is a relatively large and statistically significant difference in the change in teacher departures between no-shock and downward-shock schools. Teacher job change increased by 2.8 percentage points in schools that received lower grades under the new accountability regime, whereas schools that did not experience a change in their grade as a result of the change in the accountability system had only a one percentage point increase in teacher mobility. This Since 2002 school grades were announced in mid-june 2002, too late for most teachers to make a job change before the start of school in August, we define the pre-shock period as job changes that occurred between the three school-year pairs, to , to and to The post-shock period is the transition between school years and The general trend toward increased mobility could also be due to the accountability system in place since 1999, though we have no way of gauging the contribution of accountability to generalized mobility. 13

14 percentage point difference in the rate of teacher departure is statistically significant at better than a 99.9 percent confidence level. The simple descriptive evidence suggests that schools experiencing downward shocks see more of their teachers depart after the implementation of the accountability formula change than do schools that did not experience an accountability shock. In contrast, there is no significant difference in teacher departure from schools that receive higher grades under the new accountability system relative to schools that experience no change in their grade as a result of the change in the accountability formula. It is possible that any improvement in teachers perceptions of their work environment engendered by an unexpected increase in the school grade is offset by increased pressure to maintain a higher grade. Of course these descriptive results do not directly account for other time-varying factors that may influence teacher mobility and which could be correlated with the type of shock a school receives. B. Regression Discontinuity Regression discontinuity (RD) techniques offer estimates of the causal effects of school accountability on teacher labor market decisions. The essence of the RD design is that there is a discrete threshold that determines treatment and subjects just below or just above the cutoff are essentially identical in both observed and unobserved dimensions, save for a random error. In the present context we can implement RD by exploiting the discrete cutoff for school grades in Florida. Because there are five distinct letter grades, there are four cutoffs -- F 280, 320, 380, and 410 grade points -- for assigning the letter grades to schools. Schools earning fewer than 280 grade points received a grade of F under the new system. Due to the uneven spacing of the cutoff points, schools with two neighboring letter grades are analyzed together in our results. For example, we analyze the subsample of schools with a D or F designation 14

15 together. All together we have four analysis samples D and F schools, C and D schools, B and C schools, and A and B schools. In the following discussion we focus on the subsample of D and F schools, though the same logic applies to the other three subsamples. Because the state strictly adhered to its grading system 11, the school grading system in Florida provides for a sharp discontinuity design (Imbens and Lemieux (2008)). The sharp RD design identifies the causal effects of being an F school by comparing the teacher turnover outcomes of the treatment and control groups with similar school grade points. A discrete jump in the relationship between school grade points and teacher turnover in the neighborhood of the cutoff is evidence of a causal impact of accountability pressure on teacher mobility. Formally, the outcome variable of interest is the probability of teacher turnover in the year immediately following the policy change ( Y,. The assignment variable D i,t is a deterministic function of the forcing variable: D, 1 x, 0 x, (1a) (1b) where D indicates the treatment status of being an F school and indicates the known cutoff of 280 grade points. If a school falls below the cutoff point of 280 in year 2002, the treatment status D i takes a value of one. 11 A local mean smoothing plot of the probability of F grade receipt in 2002 and total school grading points earned in 2002 reveals a discrete jump from zero to one at the discontinuity of school grade points of 280. There was only one school with reported school grade points of 276 that was classified as "D" grade school. Only 55 teachers were affected by this potential misclassification. When we generated this school's grade using currently-available data, we calculated that the school actually should have received 282 points, suggesting that the initiallyreported school data were in error and later updated. (The state offered the opportunity for schools to challenge school grades.) We rely on the state's official grade points reports from 2002 for our analysis. 15

16 The RD design mimics random assignment if teachers near the cutoff point have nearly an identical chance of being treated, with just a random error determining whether at teacher s school falls above or below the school-grade threshold. The conditions for identification appear to hold in our case. As mentioned above, the details of the new accountability rules were not announced until late in spring 2002 and the likelihood of teachers endogenously sorting across the cutoff points is thus minimized. We also check the identification assumptions by examining the turnover rate in the years prior to the policy change. Turnover in years prior to the policy change should be unrelated to the cutoff points, i.e., we should not see a discrete jump in turnover. Following the specification check suggested by Lee (2008), we present graphical evidence of the probability of turnover as a function of school grade points for both the period prior to the policy change and after the policy shock. Figure 1 presents local mean smoothing plots of the probability of turnover against the school grade points received in year 2002, along with 95 percent confidence bands. The left panel of Figure 1 depicts the relationship between teacher turnover and school grade points in the period prior to the policy shock while the right panel illustrates the relationship in the post-shock period. Consistent with the unanticipated nature of the accountability shock, there is no discrete change in mobility at the future grade cutoff and the overlapping confidence bands indicate that the likelihood of teacher job change is not significantly different for teachers in schools just above or just below the school grade cutoff. In contrast, in the post-shock period the effect of F receipt is associated with a noticeable jump in teacher turnover of about 5-8 percentage points. The difference is significant, as the confidence bands on the left of the cutoff are at least 3 percentage points above the confidence bands on the right of cutoff. In the appendix we also present a set of graphs with the pre-shock period broken down by year to investigate whether 16

17 there was any preexisting trend for schools that were assigned an F grade in an appendix (see Appendix Figure A1). We did not find any pre-existing trend, which helps to confirm the validity of our identification strategy. In our RD analysis we employ both non-parametric and parametric methods. The flexibility of non-parametric methods allows them to accommodate a variety of non-linear relationships. However, they require large numbers of observations near the cut point and are sensitive to the choice of the range of values above/below the cutpoint that are included in the analysis or bandwidth. Estimates from parametric methods, on the other hand, can be sensitive to the selection of a functional form. Thus no approach is necessarily superior and the two should be viewed as complements (Lee and Lemieux, 2010). Our non-parametric RD analysis employs local linear regression (Hahn, Todd, and van der Klaauw, 2001; Porter, 2003; Ludwig and Miller, 2007). This approach fits a kernel weighted linear regression for any teacher observations that fall into the interval around the cutoff with the radius being the bandwidth h. Specifically, we estimate the following: Y, α τ D, β x, c β D, x, c ϵ where c h x, c h (2) The difference between the left and right limits of this regression at the cutoff of 280 will be the estimated impact of being rated as an F school in The coefficient τ in equation (2) measures the impact on teacher turnover of being rated an F school in Taking advantage of the panel nature of our dataset, we also estimate the same local linear regression using data prior to Estimates from this pre-policy shock period serve as a specification check on our identification near the cutoff. We do not expect to see any discontinuity during the pre-shock period between future F schools and future D schools. 17

18 For such a non-parametric analysis, the choice of bandwidth is important. The bandwidth defines the weight assigned to each observation. As the bandwidth gets smaller, the observations close to the cutoff receive more weight in the estimation. Following Imbens and Lemieux, we estimate the model with a range of bandwidths, varying from just around the cutoff to 30 or 40 points away from the cutoff depending on the segments of the subsample. 12 Using the approach of Calonico, Cattaneo and Titiunik (2012a, 2012b) to select an optimal bandwidth, we present RD results using the optimal bandwidth and associated half-length and double-length bandwidths in the first three columns of Table 4. A full set of results using all bandwidth choices is available in the appendix (see Appendix Table A1). We also compared our results using the CCT approach with results obtained from employing the method of Porter (2003) and find the conclusion does not change whether we use one method or the other. We focus on the results based on the optimal bandwidth choice, which are presented in the second column of Table 4. Receiving an F grade instead of a D results in an increase in teacher turnover of 10.5 percentage points, roughly a 50 percent increase in the departure rate for teachers. 13 At the other end of the grading scale we observe the just the opposite effect; schools that unexpectedly receive a B grade rather than an A experience a 2.8 percentage point reduction in teacher turnover. This might be due to the lower stress level experienced by teachers in schools rated as B (see Rouse et al. (2013) for more explanation on the mechanism). Effects of changes in the middle of the school grade distribution are mixed. Receiving a C grade instead of a B results in a 4.5 percentage 12 The largest bandwidth we examined is 40 points away from the cutoff for the D-versus-F and C-versus-D subsamples. The largest bandwidth is 30 points away from the cutoff for the B- versus-c and A-versus-B subsamples. 13 It is difficult to be precise, since the turnover rate depends on the bandwidth. If we include data points 10 points to the left and right of the D/F cutoff, the post-shock turnover rates are 28 percent below the cutoff and 21 above the cutoff. 18

19 point higher rate of teacher turnover. In contrast, receiving a lower grade of D instead of a C grade leads to a 6.4 percentage point lower rate of teacher turnover. Although the point estimates of the impact of a school grade reduction on teacher turnover vary along the school grade distribution, only the results for shifting from a D to an F are robust to increases in bandwidth. For all other sub-samples, varying the bandwidth leads to a loss of statistical significance. Though not presented here, Appendix Table A1 shows that the most robust results are for schools being shocked from a D to an F grade. Even with a bandwidth of 40 points, the point estimate of the impact of moving from a D to an F is still a seven percentage point increase in the teacher turnover rate. In addition to computing estimates based on the stand-alone bandwidth selector recommended by Calonico et al. (2012a), we also calculated the conventional and bias-corrected point estimator, robust standard-errors and robust confidence intervals; these are presented in Appendix Table A2. Results from the robust estimators are all consistent with the estimates presented in Table 3 and fall below the five percent statistical significance level, confirming our main findings. RD designs can also be implemented in a parametric framework. However, as Lee and Lemieux (2010) emphasize, selection of the functional form is critically important. The simplest specification is a linear relationship between the probability of turnover and school grade points: Y, α τ D, β x, c ϵ (3) Subscript t indicates the policy shock year of Subscript t+1 indicates the academic year , immediately following the policy shock of The coefficient τ measures the causal impact of being designated a lower grade school, say an F school in year 2002, on the turnover rate of teachers between academic years and

20 To account for possible nonlinearities, we also estimate parametric specifications using polynomial quadratic and cubic terms. In addition, we present results using flexible quadratic and flexible cubic terms to allow the slope of these polynomial functions to change on both sides of cutoff. For example, the flexible cubic functional form yields the following equation. Y, α τ D, β x, c β D, x, c β x, c β x, c β D, x, c β D, x, c ϵ (4) The coefficient estimate of τ using the post policy shock cross-sectional data provides the parametric estimate of the regression discontinuity effect. 14 In addition to the non-parametric results, Table 4 also presents the estimated effect of receiving a lower grade in a neighboring-letter-grade subsample for the post-policy shock period only. To facilitate comparison with the non-parametric specification, we do not employ any additional controls. Only linear, quadratic, cubic terms or interaction terms between the forcing variable and the treatment variable are included in the model. Echoing the non-parametric results, we find that receiving an F grade leads to a five percentage point increase in turnover in the most restrictive (linear) specification and a 14 percentage point increase in the least restrictive (flexible cubic) specification. Estimates from subsamples other than D/F do not yield any consistent pattern. In addition to the traditional RD estimates using the cross-sectional post-policy-shock datasets (τ ), we also estimate a variant of equation (4) in the panel data setting, incorporating a difference-in-differences strategy. Essentially, we include a series of fixed effects for the treatment indicator variables and the school grade points in Including these fixed effects achieves two purposes. One is to control for any time-invariant trend in teacher turnover for 14 We also use three pre-policy intervention years to provide evidence on the identification. We do not detect any discontinuity when we use three pre-policy intervention years. 20

21 schools that received an F grade in The other is to capture any pre-existing differences between hypothetical F schools and hypothetical D schools prior to the policy shock. Implicit in the RD design is smoothness in all other covariates around the cutoff points. Lee and Lemieux (2010) propose two ways of incorporating covariates in RD estimation. One method is to conduct an RD analysis on the residual after accounting for the baseline covariates; another is to simply add these covariates into the parametric regression. We adopt the second method by including additional controls for observable factors that may influence teacher occupational outcomes. To further control for the possibility that teachers on either side of the cutoff points might be different and for the possibility that schools with a different letter grade might have differing faculty characteristics, we also include teachers age, race, gender, salaries, education, experience level, certification areas, and subject-area specialty. We also control for teachers working conditions by including average classroom, school and district-level student demographic and socio-economic characteristics such as average student achievement scores on the math portion of the FCAT, average number of disciplinary incidents per student, portion of African American students, portion of Hispanic students, portion of students receiving free or reduced-price lunches and school-level average FCAT math score gains. For each of the five parametric specifications, we estimate a pooled regression including three pre-policy periods ( through and one post-policy period ( ) with the aforementioned controls. Each regression is estimated with two different samples: the subsample including the neighboring grade, say D and F schools, and the subsample including the predicted higher grade, for example predicted D schools. Estimates from the various parametric models over the two samples are presented in the top and bottom panels of Table 5. The results in Table 5 confirm the earlier finding that the most robust results are for the 21

22 D-to-F schools. Unlike the non-parametric and parametric results using only post-shock-period data (Table 4), the panel RD estimates of the impact of being treated to F school status on teacher turnover are all statistically significant at the five percent level. Being assigned an F grade translates into an increase in teacher turnover of 7 to 17 percentage points, depending on the functional form. There is also some, though less robust, evidence that a reduction in the school grade lessens turnover at the upper end of the school grading scale. In the specifications without interaction terms, schools receiving a B grade have turnover rates about two percentage points lower than A schools, suggesting that there is on-the-job stress related to living up to the A standard. The bottom panel of Table 5 provides evidence on the teachers responses to deviations from the rating that their school would have received absent a change in the grading system. Estimates are produced from sub-samples in which only schools that would have received an equal grade or one letter grade higher (based on the old grading system) than they actually received (based on the new grading system) are included in the analysis. This, for example, the Predicted D subsample includes schools that would have received a D under the old system but earned an F or D under the new system; schools predicted to receive a C grade or F grade under the old grading system are excluded. Teachers and administrators in some of these predicted to be D schools were potentially in for a considerable surprise in Summer 2002 when they learned that instead of a D grade, their school will joining the ranks of F schools a designation that no schools in the state of Florida received in the preceding year. At the same time, there are other predicted to be D schools which just missed the receipt of an F grade. This near miss group should be nearly identical to the shocked-to-f grade schools. The estimated 22

23 impact of moving from a D to an F on teacher turnover from this closely matched subsample ranges from 6 to 18 percentage points. Prior research on general teacher mobility distinguishes between moves between schools within a district, moves between districts and exit from public school teaching and finds that different factors can affect the type of move (c.f. Hanushek et al., 2004; Boyd, et al., 2005a; Scafidi, et al., 2007; Feng, 2009). These studies show that working conditions have a large effect on teacher mobility within a district, but inter-district moves and exits are more likely to be affected by factors other than working conditions, such as salary differentials and geographic preferences. The accountability pressure under study here can be considered as a characteristic of teachers working conditions and thus we would expect there be differential effects of accountability pressure on the type of move. Distinguishing different types of moves is important for policy since intra-district and inter-district moves can alter the distribution of teachers across schools but do not affect the average quality of teachers in the public school system whereas exits could potentially have a net negative impact on student achievement in public schools. Table 6 presents estimates from parametric RD models of the impact of receiving a school grade of F on the three types of teacher occupational changes. We find robust evidence across the various specifications and two sub-samples that receipt of an F grade leads to a three to nine percentage point increase in the probability of exit from the Florida public school system. In contrast, we find no evidence that accountability pressure affects teacher mobility across school districts in Florida. 15 Being designated a failing school may make it more likely teachers 15 Florida s countywide school districts are geographically large, so teaching in a new district generally involves a long-distance move. Results could differ in states with more compact school districts. 23

24 in the schools will move to another school in the same district, though this result is not consistently statistically significant. In the linear specifications receipt of an F grade is estimated to increase the proportion of teachers moving to another school in the district by six to seven percentage points. 16 C. Heterogeneity Results for Different School Levels and Subject Areas Since teachers at different school levels might face different opportunity wages, and because accountability may influence teachers in large schools differently from small schools, it is conceivable that accountability pressure may have differential effects for elementary, middle and high school teachers. 17 We present evidence of differential turnover rates at different school levels in Table 6. Panel A presents the estimated impact of receiving an F grade instead of a D grade while Panel B presents the estimated effect of being shocked down to an F grade. The most robust results are for high school teachers, where receiving an F grade or being shocked down to an F leads to a statistically significant increase in turnover of 9 to 25 percentage points in all specifications. The point estimate of the effect of receiving or being shocked to an F on turnover of middle school teachers is also positive in all cases, ranging in 16 Appendix Figure A2, which illustrates the discontinuity in teacher turnover using the flexible quadratic specification, provides visual evidence of the causal effect of receiving an F grade. During the pre-shock period of , we do not observe any difference in teacher turnover rates near the cutoff. The average turnover rate pre-shock is between 15 percent and 20 percent. The estimated discontinuity is not statistically significant different from zero. After the policy change, schools to the left and right of cutoff experience higher turnover rates. However, schools to the left of the cutoff experience a percent turnover rate whereas schools to the right of the cutoff only experience a percent turnover rate. 17 We classified school levels by the actual grade taught using the job classification code in their actual job assignment. Elementary school teachers refer to grade 3-6 teachers teaching in selfcontained classrooms. Middle school teachers refer to any secondary classroom teachers teaching in grades 7-8. High school teachers refer to any secondary classroom teachers teaching grade

25 magnitude from 5 to 21 percentage points. However, it is only statistically significant in six of ten cases. We observe no effects of F grades on teacher turnover in elementary schools. We also examine the differential effects of accountability pressure on teachers of seven major subject areas. 18 Two of the subject areas--mathematics and English/Language Arts (ELA), are tested subject areas and are thus under much more scrutiny in the accountability system. We also examine the effect of accountability on two untested core academic subject areas, science and social studies, 19 as well as three non-core academic areas, Art, Music, and Physical Education. It is possible that there will be some effects of accountability pressure on science and social studies teachers, whose work is closely related to the tested subjects of math and reading. Further, it is not uncommon for teachers to teach both tested and non-tested subjects (e.g. math and science). In contrast, we would expect little accountability pressure on teachers in subjects outside the core curriculum, like physical education, music and art. Further, it seems likely that teachers in the core academic areas would have greater opportunity costs, on average, than instructors in physical education, music and the arts. Panel A of Table 8 displays estimates from the regression discontinuity analysis using the D-and-F grade sample while Panel B presents estimates of the effects of being shocked down to an F grade, each broken down by subject area. Across both samples, we find no statistically significant effects of F grades on the mobility of teachers in non-core-academic subjects. In contrast, we find robust evidence that teachers of core academic subjects, whether directly tested or not, are more likely to make a job change when their school is designated as failing. In 22 of 18 Subject areas are based on the primary job classification. For example, physical education teachers will include anyone classified as a physical education teacher, irrespective of the grade level. Though it is relatively rare, there are some elementary school teachers teaching in departmentalized math or ELA classes. We include these teachers in our analysis. 19 Science scores were later included in the determination of school grades but were not part of the school grading system in the period under study. 25