Study on the Distribution of. Teacher Quality. in Texas Schools. Presented by

Transcription

1 Study on the Distribution of Teacher Quality Fall in Texas Schools 2010 No.1 No.3 Figure 1 No.2 No.4 Presented by the Association of Texas Professional Educators E d F u l l e r, P h. D., University Council for Educational Administration

2 Study on the Distribution of Teacher Quality in Texas Schools Presented by The Association of Texas Professional Educators Ed Fuller, Ph.D. University Council for Educational Administration fall 2010

3 CONTENTS Executive Summary Major Findings...4 Policy Recommendations Introduction Purpose of the Study...8 Documenting the Distribution of Teacher Quality....8 exploring the Creation of a Composite Measure of Teacher Quality establishing a Relationship Between Teacher Quality and Student Achievement Review of Literature...9 Teacher Quality....9 teacher Characteristics Associated with Gains in Student Achievement Four Areas of Teacher Quality Teacher Experience Teacher Certification Status teacher Preparation Program Quality and Certification Scores Teacher Stability Data and Methodology Constructing and Selecting a TQI Rating Past Teacher Quality Indices New Texas Teacher Quality Index Methodology One Methodology Two Components of TQI Ratings Teacher Experience Teacher Certification Status teacher Preparation Program and Certification Examination Scores Teacher Retention and Employment Sample of Schools table 1A: Number of Schools Included in the Final Sample table 1B: Percentage of Schools Included in the Final Sample Findings Findings Section I: associations between TQI Ratings and Student Achievement Findings Section II: TQI and Measures of Teacher Quality Interpreting the TQI Ratings Figure 1: Standard Normal Distribution Elementary Schools table 2: Average Teacher Qualification Measures Across TQI Rating Groups for Elementary Schools Table 3: Percentage of Novice Teachers by TQI Rating. 23 table 4: Percentage of Teachers Employed at Least Three of the Past Five Years by TQI Middle Schools table 5: Average Teacher Qualification Measures Across TQI Rating Groups (Middle Schools) High Schools table 6: Average Teacher Qualification Measures Across TQI Rating Groups (High Schools) Findings Section III: distribution of TAKS acheivement by TQI Ratings Elementary Schools table 7: Average TAKS Z-Scores by TQI Rating for Elementary Schools table 8A: Percentage of Hispanic and Economically Disadvantaged Students by Octile of TQI Ratings table 8B: Percentage of Hispanic and Bilingual Students in Predominantly Hispanic Schools by Octile of TQI Ratings table 8C: Average TAKS Z-Scores for Predominantly Hispanic Elementary Schools by TQI Rating Middle Schools Table 9: Average TAKS Z-Scores by TQI Rating for Middle Schools High Schools table 10: Average TAKS Z-Scores by TQI Rating for High Schools Findings Section IV: Distribution of TQI Ratings by School Accountability Rating and Student Demographics Elementary Schools Accountability Ratings table 11: Elementary School TQI by School Accountability Rating Economically Disadvantaged Students table 12: Elementary School TQI by the Percentage of Economically Disadvantaged Students table 13: Elementary School TQI by the Percentage of Economically Disadvantaged Students for Selected Region ESCs Minority Students table 14: Elementary School TQI by the Percentage of Minority Students table 15: Elementary School TQI by the Percentage of Minority Students for Selected Region ESCs Study on the Distribution of Teacher Quality in Texas Schools fall 2010

4 table 16: Percentage of Novice Teachers and Teachers Employed Greater than Two of the Last Five Academic Years Middle Schools and the Distribution of TQI Ratings Accountability Ratings table 17: Middle School TQI Ratings by School Accountability Ratings figure 2: Selected Teacher Qualifications for Academically Unacceptable and Recognized Middle Schools Economically Disadvantaged Students table 18: Middle School TQI by the Percentage of Economically Disadvantaged Students figure 3: Percentage of Teacher FTEs Assigned Out-of-Field By the Percentage of Economically Disadvantaged Students in the School Minority Students table 19: Middle School TQI by the Percentage of Minority Students High Schools and the Distribution of TQI Ratings Accountability Ratings table 20: High School TQI Ratings by School Accountability Ratings figure 4: Percentage of Teacher FTEs with Selected Characteristics For the Lowest- and Highest-Performing High Schools Economically Disadvantaged Students table 21: High School TQI Ratings by the Percentage of Economically Disadvantaged Students Minority Students table 22: High School TQI Ratings by the Percentage of Minority Students Teacher Quality Index by Region Education Service Center Area Figure 5: TEA Region Education Service Centers Map.. 38 table 23: Region Education Service Center Headquarter Locations Elementary Schools table 24: Elementary School TQI Ratings and Difference in TQI Ratings between Low- and High- Poverty Schools by Region Education Service Center Middle Schools table 25: Middle School TQI Ratings and Difference in TQI Ratings between Low- and High-Poverty Schools by Region Education Service Center High Schools table 26: High School TQI Ratings and Difference in TQI Ratings between Low- and High- Poverty Schools by Region Education Service Center Teacher Quality Index by Metro Area and School District table 27: TQI Ratings for Secondary Schools by School Poverty for Five Major Metro Areas table 28: Average TQI and Student Demographics for Low- Poverty and High- Poverty Middle Schools in Selected Major Urban Districts table 29: Average TQI and Student Demographics for Low- Poverty and High- Poverty High Schools in Selected Major Urban Districts Conclusions table 30: High School TQI Ratings by the Number of Times Rated Academically Unacceptable ( through ) Discussion Policy Recommendations State-Level Policy District-Level Policy Recommendations Appendix A Ordinary Least Squares Regression Results table A-1: Standardized Coefficients and Statistical Significance For Ordinary Least Squares Regression.. 51 Elements of the TQI Ratings by School Level Elementary Schools table A-2a: Components of Elementary School TQI Ratings table A-2b: Proportion of Student Achievement Variance Explained by Elementary School TQI Ratings Middle Schools table A-3a: Components of Middle School TQI Ratings.. 53 table A-3b: Proportion of Student Achievement Variance Explained by Middle School TQI Ratings High Schools Table A-4a: Components of High School TQI Ratings. 55 table A-4b: Proportion of Student Achievement Variance Explained by High School TQI Ratings Appendix B table B-1: Distribution of TQI Ratings for Elementary Schools Table B-2: Distribution of TQI Ratings for Middle Schools Table B-3: Distribution of TQI Ratings for High Schools.. 58 References fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 3

5 EXECUTIVE SUMMARY.Improving teacher quality has long been a focus of policymakers. In recent years, most reform efforts include strategies to improve the quality of teachers overall and/or create a more equitable distribution of teachers within schools, districts, and states. Indeed, the federal No Child Left Behind Act of 2001 mandated that all states assess the distribution of teacher quality, develop and implement plans to improve overall teacher quality and create a more equitable distribution of teacher quality. Yet, as noted by The Education Trust (2006), few states have taken this mandate seriously, with most states not even assessing the distribution of teacher quality, much less implementing plans to address the issues associated with teacher quality..this study examined the distribution of teacher quality in Texas public schools by creating several Teacher Quality Indices based on different measures of teacher quality. Each Teacher Quality Index (TQI) provides a single measure of teacher quality based on a larger number of teacher quality measures such as teacher experience, certification status, academic ability, or even stability at a school. Although there is ongoing debate about whether objective measures of teacher quality (such as experience, certification status, quality of the preparation, and teacher stability) accurately identify teacher effectiveness in improving student achievement, recent research has found statistically significant relationships between these objective measures of quality and gains in student achievement. Importantly, this study focuses on those measures found in previous research to be associated with improvements in student achievement and how these measures are distributed across Texas public schools. Major Findings The major results of this study are as follows: Even after controlling for prior achievement, student demographics, and geographic location, teacher quality at the school level is associated with student achievement especially at the secondary level. At the elementary-school level, teacher quality appears to be more equitably distributed than at the secondary-school level, but this is more likely a result of the combination of a lack of detailed data and the greater supply of elementary teachers than the result of any state or district policies to equalize teacher quality across schools. Students in lower-performing schools have substantially less access to teacher quality than students in higher-performing schools. At all school levels, but particularly at the middle- and high-school levels, students in high-poverty and predominantly minority schools have far less access to teacher quality than students in low-poverty and predominantly White schools. More generally, the results of this study unambiguously reveal a substantial inequitable distribution of teacher quality across the state at the middle- and high-school level. Clearly, students in low-performing schools as well as in high-poverty and predominantly minority schools have far less access to the same levels of teacher quality as students in high-performing, low-poverty, and predominantly White schools. Moreover, this finding holds true for schools that are within driving distance of one another both within the same district and across district lines between contiguous districts. Policy Recommendations The differences in teacher quality across schools and districts in Texas are substantial. If Texas is going to increase overall achievement and prepare a greater percentage of students to graduate from high school and be well-prepared for life after high school, we must address the inequitable distribution of teacher quality. To do so, Texas policymakers and educators should explore a number of recommendations. State policy recommendations are described below as well as at the end of the report. In addition, district-level policy recommendations are included after the state-policy recommendations at the end of the report. The following recommendations are targeted to state policymakers in the Governor s Office, Legislature, State Board of Education, State Board for Educator Certification, the Texas Education Agency, and the Texas Higher Education Coordinating Board. 4 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

6 Fund and support the gathering of input from teachers and administrators on improving teacher quality and more equitably distributing teacher quality..the state should hire a group of education experts (some of whom should have experience as teachers, principals, and central office administrators in Texas public schools) to travel the state and convene groups of educators in order to gather their input on how to best improve teacher quality and lessen the inequitable distribution of teacher quality. The results of this effort should drive state policy. An excellent model for such an effort is the Teacher Leaders Network organized by Dr. Barnett Berry of the Center for Teaching Quality ( This effort brings together teacher leaders from across the nation in a virtual network to share best practices, provide support, and push for policy changes that support teachers. Create an annual statewide report that analyzes the aggregate TQI and the individual TQI components, and provide the overall results to the public and the individual school reports to district personnel..unfortunately, the state has not highlighted the distribution of teacher quality and the trends in teacher quality over time. Unless the state publicly raises the issue, the issue will remain low on the priority list of state and district policymakers. One important step that state leaders could take is to start a conversation with district administrators about the difference between highly qualified teachers and teacher quality. According to TEA (2010), a highly qualified teacher is a teacher that meets the following requirements: Has obtained full Texas teacher certification, including appropriate special education certification for special education teachers; Holds a minimum of a bachelor s degree; and Has demonstrated subject matter competency in each of the academic subjects in which the teacher teaches. In this definition, full state certification does not mean that a teacher is fully certified in the sense that the teacher holds a standard certificate but rather any type of certificate other than a permit that is granted by the State Board for Educator Certification. The analysis in this study relied on a full standard certificate. In fact, when serving as the Co-Directors of Research at the State Board for Educator Certification, Alexander and Fuller (2004) found that Texas middle school mathematics teachers who had obtained a full standard certificate in mathematics were more effective at increasing student achievement than teachers with other full state certificates. Further, this definition requires teachers to demonstrate subject-matter competency through either a major in the field of study or a passing score on a Texas state certification examination for a particular content area. This leaves open the possibility that a person could demonstrate competency by correctly answering 70% of items correct on a certification examination that arguably could be passed by an honors Algebra II student in a high-performing high school in Texas. In addition, alternative certification teachers are only required by the state to have 12 hours of undergraduate coursework in a content area to be considered highly qualified, and the counting of these hours is left to the discretion of those working in alternative certification programs. Because these requirements are so lax, almost every teacher in Texas (and the nation) is considered highly qualified, but this measure is not based on any empirical evidence related to student achievement. Unfortunately, district leaders rarely acknowledge that they have a problem with their distribution of teacher quality because all schools have nearly 100% highly qualified teachers. The state should impress upon district leaders the need to focus on the measures in this report rather than highly qualified teacher status. Provide monetary incentives for districts to address TQI inequities across their respective schools, and increase the flexibility districts have in addressing their unique needs..this effort holds great promise if districts design and implement programs in thoughtful ways that address the issues brought forth by this study. TEA should provide incentives for a district to specifically address inequities in teacher quality across its schools. Further, TEA should allow districts greater flexibility in the spending of money to improve the distribution of teacher quality across campuses. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 5

7 This could be accomplished by restructuring the District Awards for Teacher Excellence (DATE) grant incentives or creating an alternative program that exists alongside the DATE grant program. As currently configured, the DATE program could allow districts to create an incentive program that actually exacerbates rather than ameliorates inequities in the distribution of teacher quality. Adopt and fund a new cost-of education index..the cost-of-education index (CEI) was created in the early 1990s to provide funding to districts that had difficulty in hiring well-qualified teachers due to factors outside the control of district leaders, such as the percentage of economically disadvantaged students. The CEI has never been updated despite repeated efforts by some policymakers and researchers to do so. Thus, some districts receive far more money than they should while other districts do not receive nearly the amount they deserve based on an updated CEI. The primary hurdle to updating the CEI is cost. The state should construct a new school finance system with an updated CEI and implement the changes over time so that no district loses a substantial amount of money in a short period of time. Support the creation of urban teacher academies in the 10 largest metropolitan areas across the state..urban teacher academies provide opportunities for newly certified teachers to learn how to be effective teachers of lowperforming and economically disadvantaged students under the guidance of master teachers. These programs have been shown to be quite effective in building on the training provided by high-quality teacher preparation programs. For more information, see Support the creation of urban leader academies in the 10 largest metropolitan areas across the state..urban leader academies would be similar to urban teacher academies but focus on preparing newly certified principals to be effective leaders attuned to increasing and equalizing teacher quality within schools. The University of Texas at Austin, through the University of Texas Collaborative Urban Leadership Program (UTCULP), has initiated a pre-service program to accomplish this goal, but the effort is limited to three districts and focuses only on pre-service components of training. Create an incentive program for preparation programs to produce teachers that meet the demand for teachers in their local labor market. Currently, there is no incentive for programs to produce a high school mathematics teacher as opposed to an elementary teacher, even though there is a shortage of mathematics teachers and a surplus of elementary teachers. Increase the requirements to enter teacher preparation programs in Texas, especially alternative certification programs that tend to have lower entrance requirements than traditional university-based or post-baccalaureate programs..although recent additions to the accountability system for educator preparation have dramatically improved the measures used to identify effective preparation programs, there is still room for improvement. For example, the entrance requirements for many alternative certification programs are still abysmally low. Further, some individuals can enter and complete an alternative certification and become employed as a middle or high school teacher with as little as 12 undergraduate credit hours in the subject area in which they obtained certification. Yet, individuals from traditional certification programs must complete a major in the subject area in which they obtain certification. Perhaps the state should require a minimum of 24 hours and allow programs to decide on additional content requirements. Finally, the state should require a closely supervised field experience for all teachers, even those from alternative certification programs. Fund a statewide working conditions study, and encourage all schools to participate in the study. Recent research has found that teacher working conditions have a significant impact on teacher effectiveness as measured by gains in student achievement and are the primary factor in improving teacher retention. Without addressing the issue of teacher working conditions, all other efforts to increase the degree of equity in the distribution of teachers will surely fail. Thus, the state needs to fund a high-quality working conditions survey from an organization that can ensure valid and reliable results for schools, districts, and the state. The data should be provided back to schools and districts and training be provided on the use of such tools to improve working conditions and the equitable distribution of teacher quality. Improve the training of school and district leaders..research on the relationship between working conditions and teacher turnover has consistently found that school leadership behaviors are the primary factor affecting teachers decisions to stay at or leave a particular school. These behaviors are 6 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

8 the underlying force behind the powerful effect that working conditions have on teacher retention. In fact, leadership behavior is a stronger predictor of teacher retention than either student demographics or student achievement. Better training of school leaders concomitant with other changes can increase the likelihood that more schools will have the type of school leader that attracts well-qualified and effective teachers regardless of the school characteristics..because district personnel have a great deal of influence on the hiring and distribution of teachers, superintendent programs should be required to teach prospective superintendents about the distribution of teacher quality and strategies to equalize teacher quality. Improve data collection and dissemination efforts..although Texas used to be recognized for having one of the best education data systems in the country, Texas has fallen behind a number of states. The state has invested additional money and effort into upgrading data systems, including matching students to teachers in every school in the state, yet TEA has not addressed some substantial issues regarding existing and missing data related to teacher quality. The state should bring researchers and data system experts together to identify weaknesses and potential solutions for improving the current data on educators. Researchers need better data and greater access to data on the background characteristics of all teachers employed, such as undergraduate institution, grade point average, SAT/ACT scores, certification scores, type of master s degree, and the major/minor for the undergraduate degree. Although the state should facilitate the use of such data by researchers, the state also needs to ensure that such data is not available to the general public in order to protect the confidentiality of individual teachers. In sum, the data should be made available to researchers to guide policymakers but still be protected to ensure its confidentiality and the privacy of individual teachers. Further, the state needs to invest in creating more accurate data on teacher experience and certification status. Some of this data is currently available through the Education Research Centers at the University of Texas at Austin, the University of Texas at Dallas, and Texas A&M University. However, the state could create more efficient and effective ways to make this data accessible without violating the confidentiality of individuals. Provide school-level value-added data..the general consensus of researchers on the accuracy of teacher-level value-added efforts is that such systems are generally not accurate and stable enough for use when making high-stakes decisions about teachers. However, the state should provide useful school-level value-added information for each grade level and subject area for which information is available so that school and district personnel can identify areas of strengths and weaknesses. Moreover, the state should model appropriate use of this data and provide training on correct interpretation and appropriate use of such information. If the state does begin to provide such data, an oversight team of researchers and practitioners should be created to ensure that the data is used and denominated in ways that meet the best practices as established by experts in the field. Improve the school accountability system..currently, the state s school accountability system provides a disincentive for well-qualified and effective teachers to move to low-performing schools. The primary driver of this disincentive is the absence of an accurate barometer and recognition of student growth. Teachers are often fearful of seeking employment in low-performing schools because of the increased scrutiny, the added pressure, and the belief that they will be punished for low levels of achievement even if their students make large gains. Although the state has implemented Required Improvement and the Texas Projection Measure in an effort to reward student growth, both measures have serious methodological flaws and are clearly inferior to having an actual measure of student growth as one component of the school accountability system. Develop a statewide campaign designed to increase the prestige of the teaching profession. Currently, many prospective teachers do not view the teaching profession as a prestigious one. A statewide campaign coupled with more stringent entrance requirements for preparation programs that increase the overall quality of teachers can increase the prestige of the profession and increase the supply of better-qualified entrants into the profession. Importantly, no one strategy will be sufficient in ameliorating the inequitable distribution. Indeed, a multi-pronged, multiyear strategy at the state, district, and school levels is necessary to ensure that all students have access to a well-qualified teacher. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 7

9 INTRODUCTION Currently, there is widespread consensus among researchers and policymakers that teacher quality is the most important school factor 1 affecting student achievement. Indeed, nearly every major policy report on education highlights the impact teacher quality has on student achievement and calls for providing a high-quality teacher for every student. Concomitant to this research have been examinations of the distribution of teacher quality across schools. Although the first line of research has shown that teacher quality matters tremendously to student achievement, the second line of research has consistently found that children in lower-performing schools and schools serving high proportions of poor and minority students have far lower teacher quality than higher-performing schools and schools serving low proportions of poor and minority students (Peske & Haycock, 2006). This is not to say that there are not highly effective and well-qualified teachers in every school, but rather to say that the proportion of well-qualified teachers in the lower-performing, high-poverty, and predominantly minority schools is lower than in higher-performing, low-poverty, and predominantly White schools. Because of this inequitable distribution of teacher quality, the federal No Child Left Behind Act included a provision that required states to assess the inequity in certain teacher qualifications and implement plans to close the teacher quality gap (The Education Trust, 2006). Unfortunately, as noted by The Education Trust (2006), few states have even conducted an analysis of distribution of teacher quality, and almost none have adopted and implemented serious plans to address the issue. 2 According to the report, Texas reported the percentage of highly qualified teachers in classrooms and the percentage of highly qualified teachers in high- and low-poverty schools as well as in high- and low-minority schools. Yet, despite possessing the data to make such reports available, the Texas Education Agency (TEA) failed to report on the percentage of inexperienced teachers across the state and in high- and low-poverty schools as well as in high- and low-minority schools. Indeed, TEA has still not complied with this mandate (see Yet simply focusing on the percentage of classes taught by highly qualified teachers misses the mark (The Education Trust, 2006, p. 2). Indeed, the authors note (2006, p.3) that this narrow focus ignores the distribution of inexperienced teachers and that It is possible for every teacher in a school to be highly qualified and still have classes taught by out-offield teachers. Indeed, relying on highly qualified teacher status as a measure of the distribution of teacher quality greatly obscures the reality of the extent to which students in high-poverty, high-minority, and/or low-performing schools have dramatically less access to well-qualified teachers (Fuller & Carpenter, 2009, 2008; Fuller, 2009, 2008, 2007, 2003, 2002; The Education Trust, 2006). Purpose of the Study There are three purposes of this study. The first purpose is to document the distribution of teacher quality across schools using more than just the percentage of highly qualified teachers and classrooms. The second purpose is to explore different strategies to create a composite measure of teacher quality. The third purpose is to establish a statistically significant relationship between composite measures of teacher quality and student achievement. Without establishing such a statistically significant relationship, the rest of the study is largely irrelevant. Documenting the Distribution of Teacher Quality With respect to documenting the distribution of teacher quality in Texas, Fuller and his colleagues (Fuller & Alexander, 2003; Fuller & Carpenter, 2009, 2008; Fuller & Brewer, 2005; Fuller, 2009, 2008, 2007, 2003, 2002) have repeatedly documented the inequitable distribution of teachers in Texas in a number of studies that relied on various measures of teacher quality. However, these studies described how individual measures of teacher quality varied across schools. 1 Teacher quality clearly exerts an extremely strong influence on student achievement and must be addressed in order to improve overall student achievement and close achievement gaps. It is important to note, however, that the combined effect of factors outside of the school, such as parental level of education, family income, and the number of books in a household, exert a stronger influence on student achievement than any school factors. Many policymakers and education writers now claim that teacher quality is the single most important factor influencing student achievement, but research does not validate such a claim. In terms of education policy, teacher quality is certainly where efforts need to be focused, but efforts that address issues of poverty should also be addressed by policymakers. 2 This is consistent with the trend during the past decades to largely ignore the inputs of education, such as per-pupil expenditures and teacher quality, and focus solely on outputs, such as student achievement and graduation rates. Recent research would suggest that we need to attend to both inputs and outputs to create a well-functioning system. 8 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

10 A more concise manner for documenting the distribution of teacher quality is to create a Teacher Quality Index, or TQI. A TQI is a composite indicator of teacher quality at a school or district that is created by combining a number of individual teacher quality measures in some manner. Fuller (2008a) created a TQI for Texas schools in 2008 that documented the inequitable distribution of teachers across schools. Yet that study had two drawbacks. First, the TQI was created using a fairly simplistic methodology. Following the lead of Futernick (n.d.) in his creation of a TQI in California, Fuller (2008a) simply ranked schools on individual measures of teacher quality and then averaged those rankings. Second, the study did not include a statistical analysis of the association between the TQI and student achievement. Researchers from the Illinois Education Research Council (DeAngelis, Presley, & White, 2005; Pressley, White, & Gong, 2005) employed a more sophisticated methodology called principal components analysis (PCA). Yet, their study had shortcomings as well. The authors assumed that certain measures identified in the literature as being associated with student achievement were included in the TQI without assessing whether each measure was, in fact, actually associated with student achievement in Illinois. Rather, the authors used PCA to weight each teacher-quality measure and then used the resulting weighted teacher-quality measure to create a TQI. Using regression analysis, the authors did find that the TQI created through PCA was statistically significantly associated with student achievement. Exploring the Creation of a Composite Measure of Teacher Quality Because there was no consensus on how to construct a TQI, this study explored multiple strategies to create a TQI and reported on three different TQIs one based on regression results (Regress TQI), one based on an average of teacherquality measures but excluding measures of teacher stability (Avg1 TQI), and one based on an average of teacher-quality measures and including measures of teacher stability (Avg 2 TQI). Each TQI was a single numeric indicator of a number of different teacher-quality measures. The three different TQIs will be described in subsequent sections of this study. Establishing a Relationship Between Teacher Quality and Student Achievement Finally, in order to establish a statistically significant relationship between the different TQIs and student achievement, I used ordinary least-squares regression analysis to examine whether each TQI was statistically significantly related to student achievement after controlling for prior achievement, percentage of economically disadvantaged students, percentage of African American students, percentage of bilingual/english as a Second Language students, percentage of mobile students, school size, and school geographic location. The results of this analysis are discussed in Appendix A. The remainder of this report reviews the literature on teacher quality and its relationship to student achievement, provides a description of the teacher quality measures and construction of three TQI ratings, and then examines the TQI ratings across Texas public schools. After providing summary conclusions, the report proffers some policy recommendations for states and districts to address the inequitable distribution of teacher quality in Texas. Review of Literature Teacher Quality Over the past decades, researchers have examined the relationship between certain observable characteristics of teachers (such as years of experience, certification status, teachers own test scores, etc.) and changes in student achievement. Traditionally, the examinations of teacher characteristics have focused on qualifications such as the undergraduate college attended, certification status, advanced degrees, and years of teaching experience (Fabiano, 1999; Rowan, Chiang, & Miller, 1997). With the advent of certification tests for teachers in the 1990s, researchers have also examined the relationship between teacher test scores and student achievement (Rice, 2003). Most recently, researchers have examined the impact of teacher instability and turnover on student achievement (Ingersoll & May, 2010). Rather than focusing on the characteristics and qualifications of teachers, economists have turned to assessing teacher quality by directly assessing the achievement gains of the students taught by the teacher. Indeed, in just the past five years, 3 PCA is a statistical procedure that examines a relatively large number of measures related to one another and that are, perhaps, even redundant such as teacher quality measures. PCA then attempts to identify a smaller number of variables than the original number of variables that still explain the underlying concept in this case, teacher quality. PCA can be used to create a single index for teacher quality based on how well each individual measure of teacher quality explains the overall concept of teacher quality. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 9

11 a number of researchers have measured teacher quality by associating student value-added achievement gains with specific teachers over time and assigning teacher-quality ratings as a function of student outcomes (c.f., Goldhaber et al., 2007; Hanushek, Kain, O Brien, Rivken, 2005; Sanders & Rivers, 1996). Such analyses, however, are problematic for a number of reasons. First, as Sass (2008) concluded, researchers simply have not developed a value-added system that accurately identifies effective and ineffective teachers despite large investments of time, money, and effort. Most problematic with current value-added assessments is the relatively high instability of teacher effectiveness across years. A number of teachers, in fact, move from being effective in raising student scores to being ineffective in raising student scores and vice versa over just one year of time. Similarly, a recent Institute of Education Sciences report found that value-added systems had relatively large error rates that led to a substantial percentage of teachers being incorrectly identified as high- or low-performing (Schochet, P.Z. & Hanley, 2010). In fact, the researchers found there was about a 25% chance, if using three years of data, or a 35% chance, if using one year of data, that a teacher who was average would be identified as significantly worse than average. Second, such analyses are rather difficult to accomplish given current limitations of school and district information management systems, which often do not integrate student achievement and personnel data. Third, recent evidence has found that a teacher s peers in the school and the school context significantly affect a teacher s value-added assessment. In other words, where you teach and with whom you teach impacts your effectiveness as a teacher (Jackson & Bruegmann, 2009). Finally, even if such assessments were accurate, they provide no useful information to districts when hiring beginning teachers or the roughly 70% of teachers who teach in subject areas and grade levels without any formal assessment of student achievement. Most research on teacher quality falls somewhere between the direct assessment of student achievement gains as a function of teacher quality and more subjective observational data of teacher qualities such as caring about students and working hard. Indeed, large-scale education policy studies tend to focus on the relationships between easily classifiable teacher qualifications and student outcomes. In addition to the criteria of college degree, certification status, experience, and student achievement, researchers have considered content area and educational preparation, teachers own test scores, and the quality of undergraduate and graduate institutions attended (Angrist & Guryan, 2003; Coleman, 1966; Ehrenberg & Brewer, 1995; Figlio, 2002; Kersting, 2008; Lankford, Loeb, & Wyckoff, 2002). Teacher Characteristics Associated with Gains in Student Achievement Researchers have consistently found that particular characteristics of teachers are positively associated with student gains in achievement. The two most consistent findings are (a) teachers own prior test scores, especially those related to verbal ability (Coleman, 1966; Ehrenberg & Brewer, 1995), and (b) the selectivity or competitiveness of the undergraduate institution attended by the teacher (Angrist & Guryan, 2003; Ehrenberg & Brewer, 1994; Figlio, 2002; Lankford et al., 2002; Reback, 2002; Rice, 2003). Indeed, one might assume some overlap between these two characteristics teachers from selective institutions tend to do well on tests of verbal ability. A third characteristic associated with student achievement is a teacher s years of experience in education. Despite some disagreement about the relationship between teacher experience and student achievement, there is a growing consensus that novice teachers (those with three or fewer years of experience) and especially beginning teachers (those in their first year of teaching) are less effective than more experienced teachers in increasing student achievement (Ehrenberg & Brewer, 1994; Ferguson, 1991; Boyd, Grossman, Hamilton, & Wyckoff, 2007; Goldhaber & Brewer, 1996; Murnane & Phillips, 1981; Rice, 2003; Ladd, Clotfelter, & Vigdor, 2007, 2010). Similarly, studies on teacher subject-matter knowledge indicate that while there is a positive relationship, the relationship is not as strong as one might suspect (Wilson, Floden & Ferrini-Mundy, 2001); it is most important in mathematics and science (Goldhaber & Brewer, 2000; Rowan et al., 1997; Wenglinsky, 2002; Ladd, Clotfelter, & Vigdor, 2007, 2010), and the effect is more pronounced for upper than lower grades (Rice, 2003; Ladd, Clotfelter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009). A fifth teacher characteristic related to student achievement is certification in the area in which one teaches. A surprisingly small amount of research has examined this relationship (Darling-Hammond, Berry, & Thoreson, 2001), although more recent research from North Carolina has addressed this issue (see Ladd, Clotfelter, & Vigdor, 2007, 2010). A handful of studies have found that students taught by certified teachers have greater levels and gains in achievement than their peers taught by uncertified teachers (typically those who have not yet obtained full state certification), particularly in mathematics, science, and reading (Alexander & Fuller, 2004; Boyd, et al., 2007; Laczko-Kerr & Berliner, 2002; Ladd, Clotfelter, & Vig- 10 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

12 dor, 2007, 2010). Surprisingly, there are even fewer studies on the relationship between teachers being assigned in-field and student achievement gains. The available research has suggested that in-field status is more closely associated with student achievement gains than certification status (Wilson, 2008; Ladd, Clotfelter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009). For example, Alexander and Fuller (2004) found that Texas middle school teachers assigned in-field evidenced greater gains in student achievement than teachers assigned out-of-field. Moreover, the large effect size of their findings suggested that in-field status has a fairly substantial effect on student achievement..although not a teacher characteristic, recent research has also found that school-level employment stability of teachers is related to gains in student achievement (Ingersoll & May, 2010). This area of research is somewhat controversial, as the effect of turnover on student achievement depends not just on the absolute level of turnover but also on the qualifications of those who leave the schools and of those who remain. If the least qualified and least effective are those who leave the school, then turnover could actually improve student achievement. However, if those who leave are the most qualified and most effective, then turnover could certainly negatively impact student achievement. In their review of the business and education literature, Ingersoll and May (2010) found that some degree of turnover is preferable for high performance, but too much turnover has a negative impact on the performance of those remaining in the organization. Recently, several papers have emerged that examine the relationship between teacher turnover and student achievement (Levy, Ellis, Joy, Jablonski, & Karelitz, 2010; Meier & Hicklin 2007; Keesler 2010). In general, research in this area finds that teacher turnover tends to have a negative effect on student achievement, especially in schools where turnover is consistently high. Not all research, however, finds that every one of the above characteristics is related to improvement in student achievement. For example, Buddin and Zamarro (2010) found that none of the observable characteristics of teachers (experience, certification status, or certification test scores) were associated with gains in student achievement. Some, in fact, argue that measures of teacher quality such as certification status and licensure scores are so ineffective at predicting performance that all barriers to becoming a teacher should be removed and teachers should be hired and fired solely on their effectiveness in the classroom (c.f., Walsh, 2001; Rockoff, 2004; Kane, Rockoff, & Staiger, 2006)..However, even though the research is somewhat mixed, the preponderance of the available evidence suggests that measures of teacher quality are, in fact, associated with gains in student achievement. This is especially true at the secondary levels in the areas of mathematics and science. Four Areas of Teacher Quality Ideally, this study of teacher quality would examine the qualifications of teachers, accurate value-added assessments of teachers, and principals assessments of teachers. Yet, there is no value-added assessment of teachers in Texas, except in a few selected districts. Even if such data were available, the general consensus of researchers is that there is no value-added methodology currently available that provides accurate and stable estimates of teacher effectiveness (Sass, 2008). Further, the current assessment of teachers by principals is confidential information and should remain confidential because the information is related to employment decisions. The availability of summary information about the quality and effectiveness of teachers as perceived by principals information that did not violate the privacy and confidentiality of individual teachers would certainly inform this study. Because the only data available on teacher quality in Texas is the objective characteristics of teachers, this study focuses on four broad areas of teacher characteristics that are related to student achievement in the literature: 1. Teacher experience; 2. Teacher certification status; 3..Teacher preparation program quality and certification scores; and, 4. Teacher stability. Teacher Experience Teacher experience includes two measures of teacher experience: the percentage of novice teachers and percentage of beginning teachers. Both measures have been found to have a negative effect on student achievement. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 11

13 Teacher Certification Status Teacher certification status includes two primary measures: the percentage of teachers fully certified/not fully certified and the percentage of teachers assigned in-field/out-of-field. Although the research on these measures is weaker than that of teacher experience, the existing research suggests that fully certified teachers particularly those assigned in-field are associated with gains in student achievement. However, these effects are more pronounced at the secondary levels and in the subject areas of mathematics and science. Teacher Preparation Program Quality and Certification Scores Teacher preparation program quality and certification scores include four measures, all based to a varying degree on teachers scores on the Pedagogy and Professional Responsibilities examinations administered by the state. Certification score data for the years since 2007 is no longer publicly available. Further, certification score data prior to 1992 is incomplete. However, more than 50% of all teachers included in the study had pedagogy scores. While this introduces a problem with missing data, the percentage of teachers with missing certification score data did not vary across school demographics. Because the missing data was equitably distributed across schools with different student demographics, the impact of the missing data is reduced. As noted above, recent research has found that teachers who score at the top end of the distribution on such certification tests and tests of general ability tend to be more effective in increasing student achievement. At the other end of the distribution, those scoring substantially lower than their peers on such tests tend to be less effective in increasing student achievement than other teachers. Further, these measures serve as a proxy for teacher verbal ability because a high correlation has been found between the pedagogy tests and tests of general verbal ability. At the preparation program level, graduates from highly selective undergraduate programs tend to have greater verbal ability and, hence, greater test scores than their peers from less selective undergraduate programs. Thus, graduating from a selective undergraduate institution serves as a proxy for teacher verbal ability. Because the majority of newly certified Texas teachers are from alternative certification programs rather than traditional undergraduate preparation programs, information on the selectivity of the undergraduate program is not available for each teacher. Indeed, the state simply does not collect information on the undergraduate program attended by alternative certification teachers. Thus, to identify highly selective programs, this study takes a different approach than that typically taken by researchers. First, I identified all teachers as being from a highly selective institution if they had obtained an undergraduate degree from a Texas public institution of higher education that was rated as a Doctoral I or II institution or a Research I or II instruction by the Carnegie Classification system. Second, for those teachers with missing data on undergraduate institution, I used the overall pedagogy scores for that institution to identify highly selective institutions. Specifically, any institution with an average pedagogy score.33 standard deviations or greater than average was identified as a highly selective institution. For those programs starting after 2007, the initial certification score passing rates from the TEA website were used to identify highly selective programs. Thus, highly selective programs were identified based largely on the aggregate certification scores of the graduates of the programs. The underlying assumption is that programs with high scores must be highly selective in selecting individuals to enter into the program. In the analysis, these programs are not identified as highly selective because it is not a pure measure of undergraduate institution selectivity. Rather, I identify such preparation programs as high-performing. Teacher Stability Finally, teacher stability includes two measures of teacher retention: a measure of the percentage of teachers who have taught in a Texas public school at least three of the past five years, and a measure of the percentage of teachers who have taught at least three out of the last five years at the same schools. The first teacher stability measure was used to identify the percentage of teachers who have been in classrooms teaching the Texas Essential Knowledge and Skills (TEKS) statewide curriculum. The idea behind this measure is that teachers who are recently familiar with teaching the current TEKS might be more effective in improving achievement than teachers who have either never taught, never taught in Texas, or have been out of the classroom for an extended period of time and thus are not as familiar with the current TEKS as those teachers who have been in the classroom in recent years. A recent study suggests that this measure has a research foundation. Specifically, Ost (2009) found that teachers who consistently teach the same subject at the same grade level are more effective than those who do not because these teachers are intimately familiar 12 Study on the Distribution of Teacher Quality in Texas Schools fall 2010