Study on the Distribution of. Teacher Quality. in Texas Schools. Presented by

Transcription

1 Study on the Distribution of Teacher Quality Fall in Texas Schools 2010 No.1 No.3 Figure 1 No.2 No.4 Presented by the Association of Texas Professional Educators E d F u l l e r, P h. D., University Council for Educational Administration

2 Study on the Distribution of Teacher Quality in Texas Schools Presented by The Association of Texas Professional Educators Ed Fuller, Ph.D. University Council for Educational Administration fall 2010

3 CONTENTS Executive Summary Major Findings...4 Policy Recommendations Introduction Purpose of the Study...8 Documenting the Distribution of Teacher Quality....8 exploring the Creation of a Composite Measure of Teacher Quality establishing a Relationship Between Teacher Quality and Student Achievement Review of Literature...9 Teacher Quality....9 teacher Characteristics Associated with Gains in Student Achievement Four Areas of Teacher Quality Teacher Experience Teacher Certification Status teacher Preparation Program Quality and Certification Scores Teacher Stability Data and Methodology Constructing and Selecting a TQI Rating Past Teacher Quality Indices New Texas Teacher Quality Index Methodology One Methodology Two Components of TQI Ratings Teacher Experience Teacher Certification Status teacher Preparation Program and Certification Examination Scores Teacher Retention and Employment Sample of Schools table 1A: Number of Schools Included in the Final Sample table 1B: Percentage of Schools Included in the Final Sample Findings Findings Section I: associations between TQI Ratings and Student Achievement Findings Section II: TQI and Measures of Teacher Quality Interpreting the TQI Ratings Figure 1: Standard Normal Distribution Elementary Schools table 2: Average Teacher Qualification Measures Across TQI Rating Groups for Elementary Schools Table 3: Percentage of Novice Teachers by TQI Rating. 23 table 4: Percentage of Teachers Employed at Least Three of the Past Five Years by TQI Middle Schools table 5: Average Teacher Qualification Measures Across TQI Rating Groups (Middle Schools) High Schools table 6: Average Teacher Qualification Measures Across TQI Rating Groups (High Schools) Findings Section III: distribution of TAKS acheivement by TQI Ratings Elementary Schools table 7: Average TAKS Z-Scores by TQI Rating for Elementary Schools table 8A: Percentage of Hispanic and Economically Disadvantaged Students by Octile of TQI Ratings table 8B: Percentage of Hispanic and Bilingual Students in Predominantly Hispanic Schools by Octile of TQI Ratings table 8C: Average TAKS Z-Scores for Predominantly Hispanic Elementary Schools by TQI Rating Middle Schools Table 9: Average TAKS Z-Scores by TQI Rating for Middle Schools High Schools table 10: Average TAKS Z-Scores by TQI Rating for High Schools Findings Section IV: Distribution of TQI Ratings by School Accountability Rating and Student Demographics Elementary Schools Accountability Ratings table 11: Elementary School TQI by School Accountability Rating Economically Disadvantaged Students table 12: Elementary School TQI by the Percentage of Economically Disadvantaged Students table 13: Elementary School TQI by the Percentage of Economically Disadvantaged Students for Selected Region ESCs Minority Students table 14: Elementary School TQI by the Percentage of Minority Students table 15: Elementary School TQI by the Percentage of Minority Students for Selected Region ESCs Study on the Distribution of Teacher Quality in Texas Schools fall 2010

4 table 16: Percentage of Novice Teachers and Teachers Employed Greater than Two of the Last Five Academic Years Middle Schools and the Distribution of TQI Ratings Accountability Ratings table 17: Middle School TQI Ratings by School Accountability Ratings figure 2: Selected Teacher Qualifications for Academically Unacceptable and Recognized Middle Schools Economically Disadvantaged Students table 18: Middle School TQI by the Percentage of Economically Disadvantaged Students figure 3: Percentage of Teacher FTEs Assigned Out-of-Field By the Percentage of Economically Disadvantaged Students in the School Minority Students table 19: Middle School TQI by the Percentage of Minority Students High Schools and the Distribution of TQI Ratings Accountability Ratings table 20: High School TQI Ratings by School Accountability Ratings figure 4: Percentage of Teacher FTEs with Selected Characteristics For the Lowest- and Highest-Performing High Schools Economically Disadvantaged Students table 21: High School TQI Ratings by the Percentage of Economically Disadvantaged Students Minority Students table 22: High School TQI Ratings by the Percentage of Minority Students Teacher Quality Index by Region Education Service Center Area Figure 5: TEA Region Education Service Centers Map.. 38 table 23: Region Education Service Center Headquarter Locations Elementary Schools table 24: Elementary School TQI Ratings and Difference in TQI Ratings between Low- and High- Poverty Schools by Region Education Service Center Middle Schools table 25: Middle School TQI Ratings and Difference in TQI Ratings between Low- and High-Poverty Schools by Region Education Service Center High Schools table 26: High School TQI Ratings and Difference in TQI Ratings between Low- and High- Poverty Schools by Region Education Service Center Teacher Quality Index by Metro Area and School District table 27: TQI Ratings for Secondary Schools by School Poverty for Five Major Metro Areas table 28: Average TQI and Student Demographics for Low- Poverty and High- Poverty Middle Schools in Selected Major Urban Districts table 29: Average TQI and Student Demographics for Low- Poverty and High- Poverty High Schools in Selected Major Urban Districts Conclusions table 30: High School TQI Ratings by the Number of Times Rated Academically Unacceptable ( through ) Discussion Policy Recommendations State-Level Policy District-Level Policy Recommendations Appendix A Ordinary Least Squares Regression Results table A-1: Standardized Coefficients and Statistical Significance For Ordinary Least Squares Regression.. 51 Elements of the TQI Ratings by School Level Elementary Schools table A-2a: Components of Elementary School TQI Ratings table A-2b: Proportion of Student Achievement Variance Explained by Elementary School TQI Ratings Middle Schools table A-3a: Components of Middle School TQI Ratings.. 53 table A-3b: Proportion of Student Achievement Variance Explained by Middle School TQI Ratings High Schools Table A-4a: Components of High School TQI Ratings. 55 table A-4b: Proportion of Student Achievement Variance Explained by High School TQI Ratings Appendix B table B-1: Distribution of TQI Ratings for Elementary Schools Table B-2: Distribution of TQI Ratings for Middle Schools Table B-3: Distribution of TQI Ratings for High Schools.. 58 References fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 3

5 EXECUTIVE SUMMARY.Improving teacher quality has long been a focus of policymakers. In recent years, most reform efforts include strategies to improve the quality of teachers overall and/or create a more equitable distribution of teachers within schools, districts, and states. Indeed, the federal No Child Left Behind Act of 2001 mandated that all states assess the distribution of teacher quality, develop and implement plans to improve overall teacher quality and create a more equitable distribution of teacher quality. Yet, as noted by The Education Trust (2006), few states have taken this mandate seriously, with most states not even assessing the distribution of teacher quality, much less implementing plans to address the issues associated with teacher quality..this study examined the distribution of teacher quality in Texas public schools by creating several Teacher Quality Indices based on different measures of teacher quality. Each Teacher Quality Index (TQI) provides a single measure of teacher quality based on a larger number of teacher quality measures such as teacher experience, certification status, academic ability, or even stability at a school. Although there is ongoing debate about whether objective measures of teacher quality (such as experience, certification status, quality of the preparation, and teacher stability) accurately identify teacher effectiveness in improving student achievement, recent research has found statistically significant relationships between these objective measures of quality and gains in student achievement. Importantly, this study focuses on those measures found in previous research to be associated with improvements in student achievement and how these measures are distributed across Texas public schools. Major Findings The major results of this study are as follows: Even after controlling for prior achievement, student demographics, and geographic location, teacher quality at the school level is associated with student achievement especially at the secondary level. At the elementary-school level, teacher quality appears to be more equitably distributed than at the secondary-school level, but this is more likely a result of the combination of a lack of detailed data and the greater supply of elementary teachers than the result of any state or district policies to equalize teacher quality across schools. Students in lower-performing schools have substantially less access to teacher quality than students in higher-performing schools. At all school levels, but particularly at the middle- and high-school levels, students in high-poverty and predominantly minority schools have far less access to teacher quality than students in low-poverty and predominantly White schools. More generally, the results of this study unambiguously reveal a substantial inequitable distribution of teacher quality across the state at the middle- and high-school level. Clearly, students in low-performing schools as well as in high-poverty and predominantly minority schools have far less access to the same levels of teacher quality as students in high-performing, low-poverty, and predominantly White schools. Moreover, this finding holds true for schools that are within driving distance of one another both within the same district and across district lines between contiguous districts. Policy Recommendations The differences in teacher quality across schools and districts in Texas are substantial. If Texas is going to increase overall achievement and prepare a greater percentage of students to graduate from high school and be well-prepared for life after high school, we must address the inequitable distribution of teacher quality. To do so, Texas policymakers and educators should explore a number of recommendations. State policy recommendations are described below as well as at the end of the report. In addition, district-level policy recommendations are included after the state-policy recommendations at the end of the report. The following recommendations are targeted to state policymakers in the Governor s Office, Legislature, State Board of Education, State Board for Educator Certification, the Texas Education Agency, and the Texas Higher Education Coordinating Board. 4 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

6 Fund and support the gathering of input from teachers and administrators on improving teacher quality and more equitably distributing teacher quality..the state should hire a group of education experts (some of whom should have experience as teachers, principals, and central office administrators in Texas public schools) to travel the state and convene groups of educators in order to gather their input on how to best improve teacher quality and lessen the inequitable distribution of teacher quality. The results of this effort should drive state policy. An excellent model for such an effort is the Teacher Leaders Network organized by Dr. Barnett Berry of the Center for Teaching Quality ( This effort brings together teacher leaders from across the nation in a virtual network to share best practices, provide support, and push for policy changes that support teachers. Create an annual statewide report that analyzes the aggregate TQI and the individual TQI components, and provide the overall results to the public and the individual school reports to district personnel..unfortunately, the state has not highlighted the distribution of teacher quality and the trends in teacher quality over time. Unless the state publicly raises the issue, the issue will remain low on the priority list of state and district policymakers. One important step that state leaders could take is to start a conversation with district administrators about the difference between highly qualified teachers and teacher quality. According to TEA (2010), a highly qualified teacher is a teacher that meets the following requirements: Has obtained full Texas teacher certification, including appropriate special education certification for special education teachers; Holds a minimum of a bachelor s degree; and Has demonstrated subject matter competency in each of the academic subjects in which the teacher teaches. In this definition, full state certification does not mean that a teacher is fully certified in the sense that the teacher holds a standard certificate but rather any type of certificate other than a permit that is granted by the State Board for Educator Certification. The analysis in this study relied on a full standard certificate. In fact, when serving as the Co-Directors of Research at the State Board for Educator Certification, Alexander and Fuller (2004) found that Texas middle school mathematics teachers who had obtained a full standard certificate in mathematics were more effective at increasing student achievement than teachers with other full state certificates. Further, this definition requires teachers to demonstrate subject-matter competency through either a major in the field of study or a passing score on a Texas state certification examination for a particular content area. This leaves open the possibility that a person could demonstrate competency by correctly answering 70% of items correct on a certification examination that arguably could be passed by an honors Algebra II student in a high-performing high school in Texas. In addition, alternative certification teachers are only required by the state to have 12 hours of undergraduate coursework in a content area to be considered highly qualified, and the counting of these hours is left to the discretion of those working in alternative certification programs. Because these requirements are so lax, almost every teacher in Texas (and the nation) is considered highly qualified, but this measure is not based on any empirical evidence related to student achievement. Unfortunately, district leaders rarely acknowledge that they have a problem with their distribution of teacher quality because all schools have nearly 100% highly qualified teachers. The state should impress upon district leaders the need to focus on the measures in this report rather than highly qualified teacher status. Provide monetary incentives for districts to address TQI inequities across their respective schools, and increase the flexibility districts have in addressing their unique needs..this effort holds great promise if districts design and implement programs in thoughtful ways that address the issues brought forth by this study. TEA should provide incentives for a district to specifically address inequities in teacher quality across its schools. Further, TEA should allow districts greater flexibility in the spending of money to improve the distribution of teacher quality across campuses. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 5

7 This could be accomplished by restructuring the District Awards for Teacher Excellence (DATE) grant incentives or creating an alternative program that exists alongside the DATE grant program. As currently configured, the DATE program could allow districts to create an incentive program that actually exacerbates rather than ameliorates inequities in the distribution of teacher quality. Adopt and fund a new cost-of education index..the cost-of-education index (CEI) was created in the early 1990s to provide funding to districts that had difficulty in hiring well-qualified teachers due to factors outside the control of district leaders, such as the percentage of economically disadvantaged students. The CEI has never been updated despite repeated efforts by some policymakers and researchers to do so. Thus, some districts receive far more money than they should while other districts do not receive nearly the amount they deserve based on an updated CEI. The primary hurdle to updating the CEI is cost. The state should construct a new school finance system with an updated CEI and implement the changes over time so that no district loses a substantial amount of money in a short period of time. Support the creation of urban teacher academies in the 10 largest metropolitan areas across the state..urban teacher academies provide opportunities for newly certified teachers to learn how to be effective teachers of lowperforming and economically disadvantaged students under the guidance of master teachers. These programs have been shown to be quite effective in building on the training provided by high-quality teacher preparation programs. For more information, see Support the creation of urban leader academies in the 10 largest metropolitan areas across the state..urban leader academies would be similar to urban teacher academies but focus on preparing newly certified principals to be effective leaders attuned to increasing and equalizing teacher quality within schools. The University of Texas at Austin, through the University of Texas Collaborative Urban Leadership Program (UTCULP), has initiated a pre-service program to accomplish this goal, but the effort is limited to three districts and focuses only on pre-service components of training. Create an incentive program for preparation programs to produce teachers that meet the demand for teachers in their local labor market. Currently, there is no incentive for programs to produce a high school mathematics teacher as opposed to an elementary teacher, even though there is a shortage of mathematics teachers and a surplus of elementary teachers. Increase the requirements to enter teacher preparation programs in Texas, especially alternative certification programs that tend to have lower entrance requirements than traditional university-based or post-baccalaureate programs..although recent additions to the accountability system for educator preparation have dramatically improved the measures used to identify effective preparation programs, there is still room for improvement. For example, the entrance requirements for many alternative certification programs are still abysmally low. Further, some individuals can enter and complete an alternative certification and become employed as a middle or high school teacher with as little as 12 undergraduate credit hours in the subject area in which they obtained certification. Yet, individuals from traditional certification programs must complete a major in the subject area in which they obtain certification. Perhaps the state should require a minimum of 24 hours and allow programs to decide on additional content requirements. Finally, the state should require a closely supervised field experience for all teachers, even those from alternative certification programs. Fund a statewide working conditions study, and encourage all schools to participate in the study. Recent research has found that teacher working conditions have a significant impact on teacher effectiveness as measured by gains in student achievement and are the primary factor in improving teacher retention. Without addressing the issue of teacher working conditions, all other efforts to increase the degree of equity in the distribution of teachers will surely fail. Thus, the state needs to fund a high-quality working conditions survey from an organization that can ensure valid and reliable results for schools, districts, and the state. The data should be provided back to schools and districts and training be provided on the use of such tools to improve working conditions and the equitable distribution of teacher quality. Improve the training of school and district leaders..research on the relationship between working conditions and teacher turnover has consistently found that school leadership behaviors are the primary factor affecting teachers decisions to stay at or leave a particular school. These behaviors are 6 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

8 the underlying force behind the powerful effect that working conditions have on teacher retention. In fact, leadership behavior is a stronger predictor of teacher retention than either student demographics or student achievement. Better training of school leaders concomitant with other changes can increase the likelihood that more schools will have the type of school leader that attracts well-qualified and effective teachers regardless of the school characteristics..because district personnel have a great deal of influence on the hiring and distribution of teachers, superintendent programs should be required to teach prospective superintendents about the distribution of teacher quality and strategies to equalize teacher quality. Improve data collection and dissemination efforts..although Texas used to be recognized for having one of the best education data systems in the country, Texas has fallen behind a number of states. The state has invested additional money and effort into upgrading data systems, including matching students to teachers in every school in the state, yet TEA has not addressed some substantial issues regarding existing and missing data related to teacher quality. The state should bring researchers and data system experts together to identify weaknesses and potential solutions for improving the current data on educators. Researchers need better data and greater access to data on the background characteristics of all teachers employed, such as undergraduate institution, grade point average, SAT/ACT scores, certification scores, type of master s degree, and the major/minor for the undergraduate degree. Although the state should facilitate the use of such data by researchers, the state also needs to ensure that such data is not available to the general public in order to protect the confidentiality of individual teachers. In sum, the data should be made available to researchers to guide policymakers but still be protected to ensure its confidentiality and the privacy of individual teachers. Further, the state needs to invest in creating more accurate data on teacher experience and certification status. Some of this data is currently available through the Education Research Centers at the University of Texas at Austin, the University of Texas at Dallas, and Texas A&M University. However, the state could create more efficient and effective ways to make this data accessible without violating the confidentiality of individuals. Provide school-level value-added data..the general consensus of researchers on the accuracy of teacher-level value-added efforts is that such systems are generally not accurate and stable enough for use when making high-stakes decisions about teachers. However, the state should provide useful school-level value-added information for each grade level and subject area for which information is available so that school and district personnel can identify areas of strengths and weaknesses. Moreover, the state should model appropriate use of this data and provide training on correct interpretation and appropriate use of such information. If the state does begin to provide such data, an oversight team of researchers and practitioners should be created to ensure that the data is used and denominated in ways that meet the best practices as established by experts in the field. Improve the school accountability system..currently, the state s school accountability system provides a disincentive for well-qualified and effective teachers to move to low-performing schools. The primary driver of this disincentive is the absence of an accurate barometer and recognition of student growth. Teachers are often fearful of seeking employment in low-performing schools because of the increased scrutiny, the added pressure, and the belief that they will be punished for low levels of achievement even if their students make large gains. Although the state has implemented Required Improvement and the Texas Projection Measure in an effort to reward student growth, both measures have serious methodological flaws and are clearly inferior to having an actual measure of student growth as one component of the school accountability system. Develop a statewide campaign designed to increase the prestige of the teaching profession. Currently, many prospective teachers do not view the teaching profession as a prestigious one. A statewide campaign coupled with more stringent entrance requirements for preparation programs that increase the overall quality of teachers can increase the prestige of the profession and increase the supply of better-qualified entrants into the profession. Importantly, no one strategy will be sufficient in ameliorating the inequitable distribution. Indeed, a multi-pronged, multiyear strategy at the state, district, and school levels is necessary to ensure that all students have access to a well-qualified teacher. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 7

9 INTRODUCTION Currently, there is widespread consensus among researchers and policymakers that teacher quality is the most important school factor 1 affecting student achievement. Indeed, nearly every major policy report on education highlights the impact teacher quality has on student achievement and calls for providing a high-quality teacher for every student. Concomitant to this research have been examinations of the distribution of teacher quality across schools. Although the first line of research has shown that teacher quality matters tremendously to student achievement, the second line of research has consistently found that children in lower-performing schools and schools serving high proportions of poor and minority students have far lower teacher quality than higher-performing schools and schools serving low proportions of poor and minority students (Peske & Haycock, 2006). This is not to say that there are not highly effective and well-qualified teachers in every school, but rather to say that the proportion of well-qualified teachers in the lower-performing, high-poverty, and predominantly minority schools is lower than in higher-performing, low-poverty, and predominantly White schools. Because of this inequitable distribution of teacher quality, the federal No Child Left Behind Act included a provision that required states to assess the inequity in certain teacher qualifications and implement plans to close the teacher quality gap (The Education Trust, 2006). Unfortunately, as noted by The Education Trust (2006), few states have even conducted an analysis of distribution of teacher quality, and almost none have adopted and implemented serious plans to address the issue. 2 According to the report, Texas reported the percentage of highly qualified teachers in classrooms and the percentage of highly qualified teachers in high- and low-poverty schools as well as in high- and low-minority schools. Yet, despite possessing the data to make such reports available, the Texas Education Agency (TEA) failed to report on the percentage of inexperienced teachers across the state and in high- and low-poverty schools as well as in high- and low-minority schools. Indeed, TEA has still not complied with this mandate (see Yet simply focusing on the percentage of classes taught by highly qualified teachers misses the mark (The Education Trust, 2006, p. 2). Indeed, the authors note (2006, p.3) that this narrow focus ignores the distribution of inexperienced teachers and that It is possible for every teacher in a school to be highly qualified and still have classes taught by out-offield teachers. Indeed, relying on highly qualified teacher status as a measure of the distribution of teacher quality greatly obscures the reality of the extent to which students in high-poverty, high-minority, and/or low-performing schools have dramatically less access to well-qualified teachers (Fuller & Carpenter, 2009, 2008; Fuller, 2009, 2008, 2007, 2003, 2002; The Education Trust, 2006). Purpose of the Study There are three purposes of this study. The first purpose is to document the distribution of teacher quality across schools using more than just the percentage of highly qualified teachers and classrooms. The second purpose is to explore different strategies to create a composite measure of teacher quality. The third purpose is to establish a statistically significant relationship between composite measures of teacher quality and student achievement. Without establishing such a statistically significant relationship, the rest of the study is largely irrelevant. Documenting the Distribution of Teacher Quality With respect to documenting the distribution of teacher quality in Texas, Fuller and his colleagues (Fuller & Alexander, 2003; Fuller & Carpenter, 2009, 2008; Fuller & Brewer, 2005; Fuller, 2009, 2008, 2007, 2003, 2002) have repeatedly documented the inequitable distribution of teachers in Texas in a number of studies that relied on various measures of teacher quality. However, these studies described how individual measures of teacher quality varied across schools. 1 Teacher quality clearly exerts an extremely strong influence on student achievement and must be addressed in order to improve overall student achievement and close achievement gaps. It is important to note, however, that the combined effect of factors outside of the school, such as parental level of education, family income, and the number of books in a household, exert a stronger influence on student achievement than any school factors. Many policymakers and education writers now claim that teacher quality is the single most important factor influencing student achievement, but research does not validate such a claim. In terms of education policy, teacher quality is certainly where efforts need to be focused, but efforts that address issues of poverty should also be addressed by policymakers. 2 This is consistent with the trend during the past decades to largely ignore the inputs of education, such as per-pupil expenditures and teacher quality, and focus solely on outputs, such as student achievement and graduation rates. Recent research would suggest that we need to attend to both inputs and outputs to create a well-functioning system. 8 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

10 A more concise manner for documenting the distribution of teacher quality is to create a Teacher Quality Index, or TQI. A TQI is a composite indicator of teacher quality at a school or district that is created by combining a number of individual teacher quality measures in some manner. Fuller (2008a) created a TQI for Texas schools in 2008 that documented the inequitable distribution of teachers across schools. Yet that study had two drawbacks. First, the TQI was created using a fairly simplistic methodology. Following the lead of Futernick (n.d.) in his creation of a TQI in California, Fuller (2008a) simply ranked schools on individual measures of teacher quality and then averaged those rankings. Second, the study did not include a statistical analysis of the association between the TQI and student achievement. Researchers from the Illinois Education Research Council (DeAngelis, Presley, & White, 2005; Pressley, White, & Gong, 2005) employed a more sophisticated methodology called principal components analysis (PCA). Yet, their study had shortcomings as well. The authors assumed that certain measures identified in the literature as being associated with student achievement were included in the TQI without assessing whether each measure was, in fact, actually associated with student achievement in Illinois. Rather, the authors used PCA to weight each teacher-quality measure and then used the resulting weighted teacher-quality measure to create a TQI. Using regression analysis, the authors did find that the TQI created through PCA was statistically significantly associated with student achievement. Exploring the Creation of a Composite Measure of Teacher Quality Because there was no consensus on how to construct a TQI, this study explored multiple strategies to create a TQI and reported on three different TQIs one based on regression results (Regress TQI), one based on an average of teacherquality measures but excluding measures of teacher stability (Avg1 TQI), and one based on an average of teacher-quality measures and including measures of teacher stability (Avg 2 TQI). Each TQI was a single numeric indicator of a number of different teacher-quality measures. The three different TQIs will be described in subsequent sections of this study. Establishing a Relationship Between Teacher Quality and Student Achievement Finally, in order to establish a statistically significant relationship between the different TQIs and student achievement, I used ordinary least-squares regression analysis to examine whether each TQI was statistically significantly related to student achievement after controlling for prior achievement, percentage of economically disadvantaged students, percentage of African American students, percentage of bilingual/english as a Second Language students, percentage of mobile students, school size, and school geographic location. The results of this analysis are discussed in Appendix A. The remainder of this report reviews the literature on teacher quality and its relationship to student achievement, provides a description of the teacher quality measures and construction of three TQI ratings, and then examines the TQI ratings across Texas public schools. After providing summary conclusions, the report proffers some policy recommendations for states and districts to address the inequitable distribution of teacher quality in Texas. Review of Literature Teacher Quality Over the past decades, researchers have examined the relationship between certain observable characteristics of teachers (such as years of experience, certification status, teachers own test scores, etc.) and changes in student achievement. Traditionally, the examinations of teacher characteristics have focused on qualifications such as the undergraduate college attended, certification status, advanced degrees, and years of teaching experience (Fabiano, 1999; Rowan, Chiang, & Miller, 1997). With the advent of certification tests for teachers in the 1990s, researchers have also examined the relationship between teacher test scores and student achievement (Rice, 2003). Most recently, researchers have examined the impact of teacher instability and turnover on student achievement (Ingersoll & May, 2010). Rather than focusing on the characteristics and qualifications of teachers, economists have turned to assessing teacher quality by directly assessing the achievement gains of the students taught by the teacher. Indeed, in just the past five years, 3 PCA is a statistical procedure that examines a relatively large number of measures related to one another and that are, perhaps, even redundant such as teacher quality measures. PCA then attempts to identify a smaller number of variables than the original number of variables that still explain the underlying concept in this case, teacher quality. PCA can be used to create a single index for teacher quality based on how well each individual measure of teacher quality explains the overall concept of teacher quality. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 9

11 a number of researchers have measured teacher quality by associating student value-added achievement gains with specific teachers over time and assigning teacher-quality ratings as a function of student outcomes (c.f., Goldhaber et al., 2007; Hanushek, Kain, O Brien, Rivken, 2005; Sanders & Rivers, 1996). Such analyses, however, are problematic for a number of reasons. First, as Sass (2008) concluded, researchers simply have not developed a value-added system that accurately identifies effective and ineffective teachers despite large investments of time, money, and effort. Most problematic with current value-added assessments is the relatively high instability of teacher effectiveness across years. A number of teachers, in fact, move from being effective in raising student scores to being ineffective in raising student scores and vice versa over just one year of time. Similarly, a recent Institute of Education Sciences report found that value-added systems had relatively large error rates that led to a substantial percentage of teachers being incorrectly identified as high- or low-performing (Schochet, P.Z. & Hanley, 2010). In fact, the researchers found there was about a 25% chance, if using three years of data, or a 35% chance, if using one year of data, that a teacher who was average would be identified as significantly worse than average. Second, such analyses are rather difficult to accomplish given current limitations of school and district information management systems, which often do not integrate student achievement and personnel data. Third, recent evidence has found that a teacher s peers in the school and the school context significantly affect a teacher s value-added assessment. In other words, where you teach and with whom you teach impacts your effectiveness as a teacher (Jackson & Bruegmann, 2009). Finally, even if such assessments were accurate, they provide no useful information to districts when hiring beginning teachers or the roughly 70% of teachers who teach in subject areas and grade levels without any formal assessment of student achievement. Most research on teacher quality falls somewhere between the direct assessment of student achievement gains as a function of teacher quality and more subjective observational data of teacher qualities such as caring about students and working hard. Indeed, large-scale education policy studies tend to focus on the relationships between easily classifiable teacher qualifications and student outcomes. In addition to the criteria of college degree, certification status, experience, and student achievement, researchers have considered content area and educational preparation, teachers own test scores, and the quality of undergraduate and graduate institutions attended (Angrist & Guryan, 2003; Coleman, 1966; Ehrenberg & Brewer, 1995; Figlio, 2002; Kersting, 2008; Lankford, Loeb, & Wyckoff, 2002). Teacher Characteristics Associated with Gains in Student Achievement Researchers have consistently found that particular characteristics of teachers are positively associated with student gains in achievement. The two most consistent findings are (a) teachers own prior test scores, especially those related to verbal ability (Coleman, 1966; Ehrenberg & Brewer, 1995), and (b) the selectivity or competitiveness of the undergraduate institution attended by the teacher (Angrist & Guryan, 2003; Ehrenberg & Brewer, 1994; Figlio, 2002; Lankford et al., 2002; Reback, 2002; Rice, 2003). Indeed, one might assume some overlap between these two characteristics teachers from selective institutions tend to do well on tests of verbal ability. A third characteristic associated with student achievement is a teacher s years of experience in education. Despite some disagreement about the relationship between teacher experience and student achievement, there is a growing consensus that novice teachers (those with three or fewer years of experience) and especially beginning teachers (those in their first year of teaching) are less effective than more experienced teachers in increasing student achievement (Ehrenberg & Brewer, 1994; Ferguson, 1991; Boyd, Grossman, Hamilton, & Wyckoff, 2007; Goldhaber & Brewer, 1996; Murnane & Phillips, 1981; Rice, 2003; Ladd, Clotfelter, & Vigdor, 2007, 2010). Similarly, studies on teacher subject-matter knowledge indicate that while there is a positive relationship, the relationship is not as strong as one might suspect (Wilson, Floden & Ferrini-Mundy, 2001); it is most important in mathematics and science (Goldhaber & Brewer, 2000; Rowan et al., 1997; Wenglinsky, 2002; Ladd, Clotfelter, & Vigdor, 2007, 2010), and the effect is more pronounced for upper than lower grades (Rice, 2003; Ladd, Clotfelter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009). A fifth teacher characteristic related to student achievement is certification in the area in which one teaches. A surprisingly small amount of research has examined this relationship (Darling-Hammond, Berry, & Thoreson, 2001), although more recent research from North Carolina has addressed this issue (see Ladd, Clotfelter, & Vigdor, 2007, 2010). A handful of studies have found that students taught by certified teachers have greater levels and gains in achievement than their peers taught by uncertified teachers (typically those who have not yet obtained full state certification), particularly in mathematics, science, and reading (Alexander & Fuller, 2004; Boyd, et al., 2007; Laczko-Kerr & Berliner, 2002; Ladd, Clotfelter, & Vig- 10 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

12 dor, 2007, 2010). Surprisingly, there are even fewer studies on the relationship between teachers being assigned in-field and student achievement gains. The available research has suggested that in-field status is more closely associated with student achievement gains than certification status (Wilson, 2008; Ladd, Clotfelter, & Vigdor, 2007, 2010; Wayne & Youngs, 2009). For example, Alexander and Fuller (2004) found that Texas middle school teachers assigned in-field evidenced greater gains in student achievement than teachers assigned out-of-field. Moreover, the large effect size of their findings suggested that in-field status has a fairly substantial effect on student achievement..although not a teacher characteristic, recent research has also found that school-level employment stability of teachers is related to gains in student achievement (Ingersoll & May, 2010). This area of research is somewhat controversial, as the effect of turnover on student achievement depends not just on the absolute level of turnover but also on the qualifications of those who leave the schools and of those who remain. If the least qualified and least effective are those who leave the school, then turnover could actually improve student achievement. However, if those who leave are the most qualified and most effective, then turnover could certainly negatively impact student achievement. In their review of the business and education literature, Ingersoll and May (2010) found that some degree of turnover is preferable for high performance, but too much turnover has a negative impact on the performance of those remaining in the organization. Recently, several papers have emerged that examine the relationship between teacher turnover and student achievement (Levy, Ellis, Joy, Jablonski, & Karelitz, 2010; Meier & Hicklin 2007; Keesler 2010). In general, research in this area finds that teacher turnover tends to have a negative effect on student achievement, especially in schools where turnover is consistently high. Not all research, however, finds that every one of the above characteristics is related to improvement in student achievement. For example, Buddin and Zamarro (2010) found that none of the observable characteristics of teachers (experience, certification status, or certification test scores) were associated with gains in student achievement. Some, in fact, argue that measures of teacher quality such as certification status and licensure scores are so ineffective at predicting performance that all barriers to becoming a teacher should be removed and teachers should be hired and fired solely on their effectiveness in the classroom (c.f., Walsh, 2001; Rockoff, 2004; Kane, Rockoff, & Staiger, 2006)..However, even though the research is somewhat mixed, the preponderance of the available evidence suggests that measures of teacher quality are, in fact, associated with gains in student achievement. This is especially true at the secondary levels in the areas of mathematics and science. Four Areas of Teacher Quality Ideally, this study of teacher quality would examine the qualifications of teachers, accurate value-added assessments of teachers, and principals assessments of teachers. Yet, there is no value-added assessment of teachers in Texas, except in a few selected districts. Even if such data were available, the general consensus of researchers is that there is no value-added methodology currently available that provides accurate and stable estimates of teacher effectiveness (Sass, 2008). Further, the current assessment of teachers by principals is confidential information and should remain confidential because the information is related to employment decisions. The availability of summary information about the quality and effectiveness of teachers as perceived by principals information that did not violate the privacy and confidentiality of individual teachers would certainly inform this study. Because the only data available on teacher quality in Texas is the objective characteristics of teachers, this study focuses on four broad areas of teacher characteristics that are related to student achievement in the literature: 1. Teacher experience; 2. Teacher certification status; 3..Teacher preparation program quality and certification scores; and, 4. Teacher stability. Teacher Experience Teacher experience includes two measures of teacher experience: the percentage of novice teachers and percentage of beginning teachers. Both measures have been found to have a negative effect on student achievement. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 11

13 Teacher Certification Status Teacher certification status includes two primary measures: the percentage of teachers fully certified/not fully certified and the percentage of teachers assigned in-field/out-of-field. Although the research on these measures is weaker than that of teacher experience, the existing research suggests that fully certified teachers particularly those assigned in-field are associated with gains in student achievement. However, these effects are more pronounced at the secondary levels and in the subject areas of mathematics and science. Teacher Preparation Program Quality and Certification Scores Teacher preparation program quality and certification scores include four measures, all based to a varying degree on teachers scores on the Pedagogy and Professional Responsibilities examinations administered by the state. Certification score data for the years since 2007 is no longer publicly available. Further, certification score data prior to 1992 is incomplete. However, more than 50% of all teachers included in the study had pedagogy scores. While this introduces a problem with missing data, the percentage of teachers with missing certification score data did not vary across school demographics. Because the missing data was equitably distributed across schools with different student demographics, the impact of the missing data is reduced. As noted above, recent research has found that teachers who score at the top end of the distribution on such certification tests and tests of general ability tend to be more effective in increasing student achievement. At the other end of the distribution, those scoring substantially lower than their peers on such tests tend to be less effective in increasing student achievement than other teachers. Further, these measures serve as a proxy for teacher verbal ability because a high correlation has been found between the pedagogy tests and tests of general verbal ability. At the preparation program level, graduates from highly selective undergraduate programs tend to have greater verbal ability and, hence, greater test scores than their peers from less selective undergraduate programs. Thus, graduating from a selective undergraduate institution serves as a proxy for teacher verbal ability. Because the majority of newly certified Texas teachers are from alternative certification programs rather than traditional undergraduate preparation programs, information on the selectivity of the undergraduate program is not available for each teacher. Indeed, the state simply does not collect information on the undergraduate program attended by alternative certification teachers. Thus, to identify highly selective programs, this study takes a different approach than that typically taken by researchers. First, I identified all teachers as being from a highly selective institution if they had obtained an undergraduate degree from a Texas public institution of higher education that was rated as a Doctoral I or II institution or a Research I or II instruction by the Carnegie Classification system. Second, for those teachers with missing data on undergraduate institution, I used the overall pedagogy scores for that institution to identify highly selective institutions. Specifically, any institution with an average pedagogy score.33 standard deviations or greater than average was identified as a highly selective institution. For those programs starting after 2007, the initial certification score passing rates from the TEA website were used to identify highly selective programs. Thus, highly selective programs were identified based largely on the aggregate certification scores of the graduates of the programs. The underlying assumption is that programs with high scores must be highly selective in selecting individuals to enter into the program. In the analysis, these programs are not identified as highly selective because it is not a pure measure of undergraduate institution selectivity. Rather, I identify such preparation programs as high-performing. Teacher Stability Finally, teacher stability includes two measures of teacher retention: a measure of the percentage of teachers who have taught in a Texas public school at least three of the past five years, and a measure of the percentage of teachers who have taught at least three out of the last five years at the same schools. The first teacher stability measure was used to identify the percentage of teachers who have been in classrooms teaching the Texas Essential Knowledge and Skills (TEKS) statewide curriculum. The idea behind this measure is that teachers who are recently familiar with teaching the current TEKS might be more effective in improving achievement than teachers who have either never taught, never taught in Texas, or have been out of the classroom for an extended period of time and thus are not as familiar with the current TEKS as those teachers who have been in the classroom in recent years. A recent study suggests that this measure has a research foundation. Specifically, Ost (2009) found that teachers who consistently teach the same subject at the same grade level are more effective than those who do not because these teachers are intimately familiar 12 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

14 with the content and can focus on improving instruction rather than spending time learning the variations in curriculum from one grade level to another or from one subject area to another. I make the same argument with respect to the TEKS that those teachers familiar with teaching the TEKS in recent years will be more effective in improving achievement than those who are unfamiliar with the TEKS for whatever reason. For example, while an out-of-state teacher might have years of teaching experience, she or he will not be as effective in teaching Texas students as someone with the same set of skills and experience who has been teaching in Texas because the out-of-state teacher is simply not as familiar with teaching the TEKS as the Texas teacher. As mentioned above, recent research has found that greater teacher retention is associated with increased student achievement, independent of the qualifications or effectiveness of the teachers who stayed at or left the school. Indeed, in their review of the literature on teacher turnover, Ingersoll and May (2010) concluded that a general consensus has emerged that teacher turnover is a perennial problem for a large number of schools and that such turnover has a negative effect on student achievement, especially when turnover rates are consistently high. The individual measures within these four areas are described in greater detail in the next section of the report. Moreover, the following section of the report describes the relationship between each of the measures under the four broad areas and student achievement on the TAKS mathematics and reading assessments. Data and Methodology.This section of the report describes how the three TQI were constructed as well as the data and methodology employed in the construction of each of the three TQI. All the data are from the academic year unless otherwise noted. Constructing and Selecting a TQI Rating Past Teacher Quality Indices Theoretically, there are numerous methods one could use to construct a TQI. The most prominent TQIs have been created in California, Texas, and Illinois and relied on two different methodologies. The California and Texas methodologies were identical. The researchers ranked each teacher-quality measure included in the study (the variables used in California and Texas were similar, but not entirely identical) and placed schools into deciles based on the ranking for a measure. The deciles were averaged across all measures, and schools were then placed into final TQI deciles based on this average. Researchers from the Illinois Education Research Council (DeAngelis, Presley, & White, 2005; Presley, White, & Gong, 2005) employed a more sophisticated methodology called principal components analysis (PCA). PCA is a statistical procedure that examines a relatively large number of measures related to one another and that are, perhaps, even redundant such as teacher-quality measures. PCA then attempts to identify a smaller number of variables than the original number of variables that still explains the underlying concept in this case, teacher quality. So, for example, PCA could be used to create a single index for teacher quality based on how well each individual measure of teacher quality (certification status, novice teachers, teacher certification scores, etc.) explains the overall concept of teacher quality. Both methodologies have drawbacks. First, both methodologies did not rely on any statistical analysis of individual teacher-quality measures and student achievement. Rather, both methodologies relied entirely on a reading of the literature in the area to determine the basket of teacher-quality measures used in constructing a TQI. The problem with such an approach is twofold: First, measures are often defined differently in different states, especially with respect to certification status; second, a relationship that is found to be statistically significant in one state might not be statistically significant in another state. Second, the California and Texas TQI methodologies were overly simplistic and never validated as being statistically significantly related to student achievement. The Illinois PCA analysis was found to be statistically associated with student achievement in Illinois schools. However, it is unclear whether the study controlled for prior achievement. Third, the Illinois study did not create different TQIs for the different school levels. This could be problematic because teacher quality is profoundly affected by the supply of teachers, and there is generally a greater supply of elementary teach- fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 13

15 ers than secondary teachers. Specifically, the greater the supply of teachers, the more selective schools can be. The more selective schools can be, the greater the teacher quality of newly hired teachers and of the overall teaching force. One method not used to create a TQI would be to conduct a statistical analysis of the relationships between the various measures of teacher quality and student achievement, then use the results of such an analysis to weight the various measures in the construction of a single TQI. New Texas Teacher Quality Index This analysis explored the construction of a TQI using two different methods. These two methods were used to create three separate but related TQIs. Ultimately, all three TQIs were found to be fairly highly correlated, and all were statistically significantly related to student achievement, even after controlling for prior levels of achievement, student demographics, and geographic location. Methodology One The first methodology employed regression analysis to identify the individual teacher quality measures statistically significantly associated with student achievement after controlling for prior achievement, student demographics, and geographic location. This methodology had not previously been employed in creating a TQI, but it was suggested as the best methodology by several researchers from across the country. The TQI created by this methodology was called the Regress TQI. Methodology Two The second methodology was similar to the California and Texas methodology described above. This methodology identified a basket of teacher-quality measures and simply averaged the measures. Variables that the literature base indicated would have a negative relationship with student achievement were multiplied by negative one. Two TQIs were created using this methodology Avg 1 TQI and Avg 2 TQI. The difference in the two TQIs was the variables included in each TQI. In short, the Avg 1 TQI included variables in the first three teacher-quality areas (teacher experience, teacher certification status, and teacher preparation program quality/certification scores), while the Avg 2 TQI included variables in all four teacher-quality areas (teacher experience, teacher certification status, teacher preparation program quality/certification scores, and teacher stability). Components of TQI Ratings.This section of the report describes and defines the various measures of teacher qualifications that were explored in this study. The relationship between each of the following measures and student achievement were examined through a literature review and through ordinary least-squares regression analysis using Texas-specific data. Not every measure, however, was included in a final TQI. As shown in Appendix A, the variables included in the Regress TQI were selected based entirely on the results of the regression analysis for middle schools and high schools. For elementary schools, as explained above, only two of the four statistically significant variables were included in the analysis. Three of the four statistically significant variables were related to teacher stability the area of teacher quality with the smallest and least definitive research base. The decision was made to use only the teacher-stability variable with the strongest relationship to student achievement so that the elementary school TQI was not just a reflection of teacher stability. However, even if all four variables had been included, the resulting TQI would have been nearly identical to the final Regress TQI. Indeed, the correlation between the two-variable and fourvariable models was.907. For the other two TQI (Avg 1 TQI and Avg 2 TQI), the decision to include or exclude a measure was based primarily on the results of the regression analysis. For elementary schools and middle schools, however, there was not always a statistically significant result in each of the four areas, despite findings in the literature that would suggest such variables to be statistically significant. 14 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

16 Specifically, neither school level had a statistically significant result in the area of teacher preparation program quality and certification scores. Although the lack of statistical significance might indicate no relationship between the area of teacher quality and student achievement in Texas, the data available might simply not be precise enough for the analysis to identify a relationship that actually exists. There was missing data for all four of the individual measures in this area, and, moreover, the available data were simply proxies of data used in other studies. For example, Texas does not collect or identify the undergraduate institution of all teachers, particularly those from alternative certification and out-of-state programs. Thus, rather than using actual undergraduate programs to identify selective institutions, the study relies on a composite of undergraduate institution data and pedagogy data to identify high-performing preparation programs. For elementary schools, there was also no statistically significant result in the area of certification status. Again, while this might signal there was no statistically significant relationship between certification status and student achievement, the lack of precise data might also be the reason for the lack of a statistically significant relationship. Indeed, because elementary teachers receive a composite certificate that is based primarily on the scores from the reading sub-test, we don t have any idea of the distribution of teachers who did not pass the mathematics, science, social studies, or reading sub-tests. So, for example, a teacher could be considered fully certified to teach all elementary subjects, but the same teacher could have incorrectly answered all of the mathematics sub-test items. Such a teacher should be considered not fully certified in mathematics. The availability of sub-score data may, in fact, lead to a statistically significant relationship between certification status and student achievement. In the Avg 1 TQI, the measures found to be statistically significant in the regression analysis were included in the TQI but only if they did not pertain to teacher stability. If there was no statistically significant measure within each of the three broad areas of teacher quality (experience, certification status, and program quality/certification scores), then the measure within each of the three areas that was closest to being statistically significant was included. Regardless of the level of statistical significance, none of the variables related to teacher stability were included in the Avg 1 TQI. These related measures were not included because the literature base on the relationship between teacher stability and student achievement is relatively small and not as consistent as for the other three areas of teacher quality. Further, some researchers argue that teacher turnover can often improve student achievement and that the turnover results in the less effective teachers leaving the campus..finally, in the Avg 2 TQI, each of the measures found to be statistically significant in the regression analysis were included in the TQI. In addition, each measure selected for the Avg 1 TQI was also included in the Avg 2 TQI. Finally, any measures in the teacher stability area that were found to be statistically significantly related to student achievement in the regression analysis were also included in this TQI..Ultimately, as discussed in greater detail below, each of the three TQIs were found to be statistically significantly related to student achievement, even after controlling for prior achievement, student demographics, and geographic location. Thus, even though Avg 1 TQI and Avg 2 TQI were not based solely on regression results, the composite indicators were statistically significantly associated with student achievement. Further, even though some would argue against the inclusion of teacher stability variables, the exclusion or inclusion of such variables had little effect on the relationship between the TQI and student achievement..the details of the construction of each TQI are included in Appendix A. Each of the variables that were identified in the literature as being related to student achievement and that were available through existing data sources are described below. Again, each of these variables was entered into an ordinary least-squares regression analysis to determine if the measure had an independent association with student achievement after controlling for other factors. Teacher Experience Percentage of Novice Teachers: The percentage of a school s teachers with fewer than four years of teaching experience. The data was modified to remove errors and to improve its accuracy. To do so, I used a combination of employment history, initial certification year, and district-reported years of experience to create a more accurate indicator of a teacher s experience as an educator. Because I modified the original data, this measure cannot be replicated with state data. (Original Data Source: Educator responsibility data, TEA; Final Data Source: Teacher experience file created by Dr. Ed Fuller). fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 15

17 Percentage of Beginning Teachers: The percentage of a school s teachers in their first year of teaching. This data was modified in the same way as described for the percentage of novice teachers. (Original Data Source: Educator responsibility data, TEA; Final Data Source: Teacher experience file created by Dr. Ed Fuller). Teacher Certification Status Percentage of Teachers Fully Certified: The percentage of a school s teachers who have obtained full state certification. To obtain full state certification, a person must complete an approved preparation program, pass a professional responsibility (pedagogy) examination, pass a content examination, and clear a criminal background check. In most cases, the person must hold a bachelor s degree from an accredited undergraduate institution unless teaching in vocational education. A person employed on a one-year certificate for teachers from out-of-state and a person employed on a probationary certificate while enrolled in an alternative certification or post-baccalaureate program are not fully certified. It is important to note that fully certified in this analysis is not fully certified as construed by the No Child Left Behind Act. Under NCLB, teachers on probationary or out-of-state certificates may be considered fully certified if they meet certain requirements (Data Source: Who is Teaching in Texas, 2009; SBEC). Percentage of Teachers Out-of-Field: The percentage of a school s teachers assigned out-of-field. A teacher is considered assigned out-of-field if she or he does not hold a full state certification for the subject area she or he is assigned to teach. The match between the certificate and course and subject area taught is determined by rules generated by SBEC. (Data Source: Who is Teaching in Texas, 2009; SBEC). Percentage of Teachers Not Fully Certified: The percentage of a school s teachers who do not hold a full state certificate. Teachers not holding a full state certificate include persons employed on a one-year certificate for teachers from out-of-state and persons employed on a probationary certificate while enrolled in an alternative certification or post-baccalaureate program. A final category of teachers not fully certified are long-term substitute teachers. It is important to note that not fully certified in this analysis is different than the not fully certified as defined by the No Child Left Behind Act. Under NCLB, teachers on probationary or out-of-state certificates may be considered fully certified if they meet certain requirements, while teachers on probationary and out-of-state certificates are automatically considered not fully certified in this analysis. (Data Source: Who is Teaching in Texas, 2009; SBEC). Teacher Preparation Program and Certification Examination Scores Percentage of Teachers from High-Performing Educator Preparation Programs: The percentage of a school s teachers from an educator preparation program classified as high-performing. This variable is somewhat similar to the percentage of teachers from highly selective programs, but the percentage of teachers with data on undergraduate programs attended was too small to use it as a measure of teacher quality. Thus, the high-performing measure was created by combining three different types of data. First, teachers from undergraduate programs ranked as Doctoral I, Doctoral II, Research I, or Research II institutions under the Carnegie Classification system were identified as being from high-performing preparation programs. Second, teachers from preparation programs that had an average pedagogy certification score greater than.33 standard deviations above average were identified as being from high-performing preparation programs. Third, for teachers from preparation programs started after 2007, teachers from programs with average pedagogy certification scores in the top 10% of all preparation programs were identified as being from high-performing preparation programs. (Data Source: Teacher Certification Scores; SBEC; Higher Education Graduation Institutions and Degrees, Texas Higher Education Coordinating Board). School-Level Average Pedagogy Certification Scores: This measure is the average of the pedagogy scores of teachers employed in the school. Because pedagogy scores are from both the ExCET and TExES examinations, which have different scales, the scores were converted into z-scores. This measure had the most missing data of any measure in this study. 16 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

18 Teachers certified prior to 1987 did not take a certification examination, and those certified prior to 1993 often did not have a score reported even if a score existed. Further, out-of-state teachers may not have to take a certification exam under certain circumstances. Finally, scores were made confidential in 2008; thus, scores could not be obtained for any teachers certified after December Ultimately, about 55% of all teachers had pedagogy certification scores in the database. This percentage was consistent across school levels and across schools serving different percentages of economically disadvantaged students. Further, an analysis of the relationship between this measure and student achievement was conducted for all schools in the study as well as for a sub-sample of schools that had certification scores for at least 75% of teachers. With both sets of schools, the relationship between the aggregate pedagogy z-scores and student achievement was essentially the same. Because the missing data appeared to be spread relatively uniformly across schools, and because restricting the sample to schools with at least 75% of teachers with certification score data did not significantly alter the relationship between scores and achievement, this measure was included in the study. (Data Source: Teacher Certification Scores; SBEC). Percentage of Pedagogy Certification Scores Greater than One Standard Deviation Above Average: The percentage of a school s teachers who had pedagogy scores one standard deviation above the average pedagogy score. Again, as with the pedagogy scores described above, there was the problem of missing data. The missing data was spread relatively uniformly across schools, and restricting the sample of schools to those that had at least 75% of teachers with pedagogy scores did not alter the results. Thus, despite the missing data, the measure was included in the study. (Data Source: Teacher Certification Scores; SBEC). Percentage of Pedagogy Certification Scores Less than One Standard Deviation Below Average: The percentage of a school s teachers who had pedagogy scores one standard deviation below the average pedagogy score. Obviously, there was an issue of missing data again. But, the missing data was spread relatively uniformly across schools, and restricting the sample of schools to those that had at least 75% of teachers with pedagogy scores did not alter the results. Thus, despite the missing data, the measure was included in the study. (Data Source: Teacher Certification Scores; SBEC). Teacher Retention and Employment Percentage of Teachers Employed in a Texas Public School for at Least Three Years from to : The percentage of a school s teachers who were employed in a Texas public school at least three out of the five academic years spanning the academic years from to (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA). Average One-Year Teacher Retention Rate from to : The average teacher retention rate for a school for the following three groups of academic years: to ; to ; and to The retention rate was calculated for each pair of years, and then the average of the three retention rates was calculated. A three-year average was used because a one-year retention rate could reflect the impact of opening a new school in the area rather than teacher dissatisfaction with a school. (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA). Percentage of Teachers Staying at the Same School from to : The one-year teacher retention rate from the school year to the school year. Teachers were matched individually from one year to the next using educator responsibility data from TEA. This data includes the school in which each person is employed as well. Once the two files were matched, I determined whether a teacher had taught at a school for both years or for only the base year of Although, as noted above, a one-year retention rate does not always reflect teacher dissatisfaction, the simple loss of teachers and the institutional knowledge that they possess can often have deleterious effects on student achievement. (Data Source: Educator Responsibility data, 2008 and 2009, TEA). Percentage of Teachers Staying at the Same School: The percentage of a school s teachers employed in who had been employed at the same school at least three out of five academic years covering the span of years from to (Data Source: Educator Responsibility data, 2005, 2006, 2007, 2008, 2009, TEA). fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 17

19 Sample of Schools Not all Texas public schools were included in this study. First, only schools that appear in the Texas Education Agency s Academic Excellence Indicator System (AEIS: for the school year were included in the study. The numbers of schools by school level included in the AEIS files are represented in Column 1 in Tables 1A and 1B. Using this initial number of schools as a base, schools without regular accountability ratings were removed from the list of schools. The resulting number of schools is displayed in Column 2. These schools were removed because schools without regular accountability ratings typically serve small populations of special types of students, such as special education students or students removed from their regular schools because of disciplinary infractions. Because such schools are typically small, the teacher qualifications measures tend not to provide an accurate picture of the qualifications of the staff. Next, charter schools were removed from the sample because they are not required by statute to follow the same rules as regular public schools with respect to teacher qualifications. Further, many are very small and serve special populations of students, particularly students who have previously dropped out. The number of schools after the removal of charter schools is displayed in Column 3..Subsequently, schools that did not enroll students in both the and school years were removed from the sample. The resulting number of schools is shown in Column 4. Such schools were removed for a number of reasons, but primarily because one primary purpose of the study was to examine the relationship between teacher quality and student achievement. In doing so, controlling for prior levels of achievement is a critical component of the analysis. Thus, schools that did not exist in both years could not provide both years of TAKS scores that are necessary to complete the analysis..similarly, I then removed schools that did not enroll students in selected grade levels. Schools had to meet strict gradeenrollment criteria for two reasons. First, this study examined the relationship between measures of teacher qualifications and student achievement at the school level, and schools with different grade configurations can have different levels of student achievement. Indeed, the addition or elimination of one grade level can dramatically alter the overall achievement profile of a school. Thus, in order to make an apples-to-apples comparison on student achievement across schools, schools had to enroll students in the same grade levels. Second, teacher certification operates differently at the different school levels. For example, elementary teachers and some middle school teachers can obtain a generalist certificate that allows them to be considered fully certified for any core subject area at that school level, while most middle school teachers and all high school teachers obtain certificates specific to a particular content area, such as mathematics. Only by including schools with the same grade level configurations could I ensure that differences in teacher qualifications were based on supply, demand, and hiring factors rather than differences in grade configurations. The number of schools left in the sample after removal of schools not meeting the grade-level criteria is shown in Column 5. At the elementary-school level, schools had to enroll students in grades three, four, and five but not in any higher grades to be included in the sample. At the middle-school level, schools had to enroll students in grades six, seven, and eight and not in any other grade levels to be included in the analysis. At the high-school level, schools had to enroll students in grades nine, ten, and eleven and not in any grades lower than grade nine to be included in the analysis. Schools were also removed if they did not have a large enough number of students taking the TAKS test at each of the three grade levels. At each school level, a school had to have at least 30 students take the TAKS test at each grade level in order to be included in the analysis. Schools with smaller numbers of test-takers were excluded from the analysis because the measurement error on the test scores is simply too large to provide an accurate estimate of the test performance of the students. Finally, to be included, schools had to meet the TAKS testing accountability sub-population criteria of 30 students at each grade level in both the and school years. This criterion was employed because two years of test scores were needed to assess the relationship between measures of teacher qualifications and student achievement, and a sample of 30 is the minimum number of students necessary to obtain a relatively accurate assessment of a school s achievement. Schools not meeting the TAKS sample-size criteria were excluded from the analysis. The final sample of schools is denoted in Column Study on the Distribution of Teacher Quality in Texas Schools fall 2010

20 Findings The findings are separated into four different sections. Section I describes the findings about the association between each TQI and student achievement. More specifically, this section discusses whether a TQI was statistically significantly associated with student achievement on TAKS. Section II describes the relationship between the three TQI measures and the individual measures of teacher quality. Section III describes how TAKS achievement was distributed across schools with different TQI ratings. Section IV examines the distribution of TQI ratings by school accountability rating and student demographics. Findings Section I: Association Between TQI Ratings and Student Achievement The first finding and one that is important to note before proceeding with an analysis of the distribution of teacher quality across schools was that each of the three TQI ratings (Regress TQI, Avg 1 TQI, and Avg 2 TQI) were found to be statistically significantly associated with student achievement in Importantly, this statistically significant association held even after controlling for the effects of prior achievement, school demographics, and geographic location of the school on student achievement in Because these other factors were controlled for in the analysis, we know that the association between each TQI and student achievement did not exist simply because a TQI was related to student demographics that were, in turn, related to student achievement. Rather, we know that the association between a TQI and student achievement was a separate association apart from any association between student demographics and either a TQI fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 19

21 or student achievement..further, as shown in Appendix A in Tables A-2b, A-3b, and A-4b, each TQI explained a fair proportion of the variance in achievement when entered into the regression equations before other variables. At the elementary-school level, the TQI ratings explained between 3% and 13% of the variance in student achievement. At the middle-school level, the three TQI ratings explained between 10% and 24% of the variance in student achievement. At the high-school level, the three TQI ratings explained between 29% and 34% of the variance in TAKS scores. In other words, teacher quality explains between 4% and 33% of the variability or fluctuation in student achievement on TAKS. Importantly, at the high school level, teacher quality explains about 30% of the variability or fluctuation in student achievement on TAKS. When the TQI ratings were entered after all other control variables (student demographics and geographic location) but before the prior achievement variable, the ratings explained less than 3% of the variance in scores. The difference in the proportion of variance explained when entering the TQI ratings first and later in the equations is due to the shared explanatory power between the TQI ratings and student demographics. In other words, the TQI ratings and student demographics share some ability to explain student achievement. If the TQI ratings are entered first, then the shared explanatory power is attributed to the TQI ratings. If student demographics are entered first, then the shared explanatory power is attributed to the student demographics. Regardless, the results showed that TQI ratings were statistically significantly associated with student achievement. Further, the association appeared to be non-trivial, even after controlling for prior achievement, student demographics, and geographic location. Findings Section II: TQI and Measures of Teacher Quality In this section and in subsequent sections, three different TQI ratings are presented. The first is the Regress TQI that was created by weighting measures of teacher quality based on results of OLS regression analyses as described in the methodology section. The second is the Avg 1 TQI that was constructed by simply averaging a series of selected measures of teacher quality directly. The third rating the Avg 2 TQI included the measures employed in the Avg 1 TQI as well as measures of teacher retention and stability at the school..this section of the report describes the distribution of teacher qualification measures across the TQI ratings groupings for each of the three school levels. The distributions for each of the three TQI ratings (Regress, Avg 1, and Avg 2) are presented. The purpose of this section is to assist the reader in comprehending how the TQI ratings reflect the various measures of teacher qualifications..before reporting on the relationship between the TQI ratings and individual measures of teacher quality, a guide to understanding the TQI numbers is provided below. Interpreting the TQI Ratings.Before reporting the findings, this section describes how to interpret a TQI rating. All three TQI ratings have been converted into standard normal scores, or z-scores. The benefit of this conversion is that each TQI rating has a mean, or average, of 0 and a standard deviation of 1.0. This is shown below in Figure Study on the Distribution of Teacher Quality in Texas Schools fall 2010

22 The standardized normal scores along the bottom of the distribution are precisely where the TQI ratings are as well. As shown in the figure, a TQI rating of less than 0 is lower than the average TQI rating of 0, and a TQI rating greater than 0 is greater than the average TQI of 0. The further away the TQI rating is from 0, the further away the TQI rating is from the average TQI rating. Additionally, because the distribution of the TQI ratings is normally distributed, we know that 34.1% of the schools will have a TQI rating between 0 and 1, and another 34.1% of schools will have a TQI rating between 0 and Likewise, we know that 13.6% of schools will have TQI ratings between 1.0 and 2.0, while 13.6% will have a TQI rating between -1.0 and Elementary Schools As shown in Table 2, there was a direct relationship between the individual measures of teacher quality and both the Avg 1 and Avg 2 TQI ratings. Indeed, for every measure, as the TQI rating increased, so did the measure of teacher quality. For most individual teacher-quality measures, there was also a direct relationship between the measure and the Regress TQI. However, some of the relationships between individual measures of teacher quality and the Regress TQI ratings were rather weak. For example, the average teacher pedagogy exam z-scores and the Regress TQI were not directly related. Specifically, while one would expect the average teacher pedagogy z-score to increase as the Regress TQI rating increased, this was not actually the case. In fact, the average teacher pedagogy score increased as the TQI rating increased from < -1.5 TQI to 0.0 TQI, but then decreased from 0.0 TQI to > 1.5 TQI. The other ambiguous relationships also involved some use of pedagogy z-scores. There were several potential reasons as to why these relationships were not more direct. First, the relationship between an individual teacher-quality measure (such as pedagogy scores) and the Regress TQI rating might be weak because only two measures were included in the calculation of the Regress TQI at the elementary-school level and neither involved pedagogy scores. Second, pedagogy scores might simply not be reflective of teacher quality at the elementary school because of less variation in scores or less correlation with the elementary pedagogy examination scores and teacher verbal ability. What was abundantly clear from the table was that schools with low TQI ratings had far greater percentages of novice and beginning teachers than schools with high TQI ratings. Indeed, in schools with TQI ratings of less than -1.0, at least onethird of teachers had fewer than four years of teaching experience. Alternatively, schools with TQI ratings of +1.0 or greater had less than 8% novice teachers. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 21

23 Teachers Teachers Teachers Teachers years 22 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

24 The two primary components of the elementary TQI ratings were the percentage of novice teachers and the percentage of teachers employed at least three of the last five academic years. These measures were correlated (correlation coefficient =.742, p <.001), but not identical. Many teachers move into the state from other states, transfer from private schools, or have breaks in service. Thus, both measures were useful in explaining the overall teacher qualifications at the school. As shown in Table 3, schools with TQI ratings of less than -1.0 had substantially greater percentages of novice teachers than schools with TQI ratings greater than 1.0. Strikingly, at least one-third of teachers in schools with TQI ratings less than -1.0 had three or fewer years of teaching experience. As shown in Table 4, schools with TQI ratings less than -1.0 had far lower percentages of teachers employed three of the past five years in Texas than schools with TQI ratings greater than 1.0. Indeed, the differences are at least 23 percentage points. In schools with TQI ratings greater than 1.0, nearly all teachers had been employed in a Texas public school for at least three of the past five academic years. In addition, schools with low TQI ratings had lower percentages of teachers fully certified and of teachers remaining at the school for various measures of time as compared to schools with higher TQI ratings. For example, schools with the lowest TQI ratings had at least 14% teachers assigned out of-field as compared to less than 5% in schools with the highest TQI ratings. Middle Schools As shown in Table 5, there is a substantial difference in all of the teacher-quality measures between schools with the greatest TQI ratings (> +1.5) and those with the lowest TQI ratings (< -1.5). Although all of the differences are fairly substantial, one of the most remarkable is for the percentage of teachers assigned out-of-field. Although less than 11% of teachers in schools with the greatest TQI ratings were assigned out-of-field, greater than 40% of teachers in the schools with the lowest TQI ratings were assigned out-of-field. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 23

25 Teachers Teachers Teachers Teachers years 24 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

26 Teachers Teachers Teachers Teachers years fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 25

27 High Schools As shown in Table 6 on page 25, there were substantial differences in all teacher-quality measures between schools with low- and high-tqi ratings. Some of the most significant differences include: Schools with the highest TQI ratings had nearly 100% fully certified teachers, while schools with the lowest TQI ratings had only 75% fully certified teachers; In schools with the highest TQI ratings, at least 75% of teachers had been employed at the schools for at least three of the previous five academic years, as compared with 50% or less for schools with the lowest TQI ratings. Schools with the highest TQI ratings had teacher certification scores that were more than 0.5 standard deviations greater than schools with the lowest TQI ratings; and, Although the percentage of teachers from high-performing teacher preparation programs approached 60% in the schools with the highest TQI ratings, less than 18% of teachers in schools with the lowest TQI ratings were from highperforming programs. Findings Section III: Distribution of TAKS Achievement by TQI Ratings This section takes a detailed look at the relationship between student achievement on TAKS and the three TQI measures. The TAKS data used was downloaded from the TEA website and included the average scale scores for each school in Texas and for each subpopulation and grade-level in the school. Because the scale scores are not scaled the same way for each grade level, scale scores for different grade levels are not directly comparable. For example, a score of 2150 in grade 4 mathematics does not indicate the same level of achievement as a score of 2150 in grade 5 mathematics. In order to rectify this problem, the TAKS scale scores were converted into z-scores, and then the mathematics and reading/english Language Arts z-scores were averaged across all three grade levels for a particular school level. For elementary schools, z-scores were averaged across grades 3, 4, and 5. For middle schools, the z-scores were averaged across grades 6, 7, and 8. Finally, the high school z-scores were averaged across grades 9, 10, and 11. Elementary Schools In general, as shown below in Table 7, there was a direct relationship between TAKS z-scores and the TQI ratings as TQI ratings increased from the lowest octile to the highest octile, so did TAKS z-scores. For example, the average TAKS z-score for schools in the lowest octile of Avg 1 TQI was , while the average TAKS z-score for the highest octile for the Avg 1 TQI was So, the schools with the highest octile had TAKS z-scores that were standard deviations greater than schools with the lowest octile a truly substantial difference. The only exception to this generalization was that schools in the top octile of TQI ratings had slightly lower levels of achievement than octiles four through seven for the Regress TQI and less than octile seven for the other two TQI ratings. Even with the exception for octile eight, the difference in achievement between schools with the lowest TQI ratings and the greatest TQI ratings was substantial. Even the smallest difference was greater than 0.5 standard deviations, and the differences for the two average TQIs were greater than one standard deviation. The correlations between the average TQI ratings and student achievement were moderately high, but the correlation between the Regress TQI rating and student achievement was relatively small, though still statistically significant. 26 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

28 To understand this unexpected finding for the highest octile, I examined the schools in the highest octile and found that the schools were predominantly Hispanic and generally high-poverty. Indeed, the percentage of Hispanic and economically disadvantaged students in the highest octile schools was greater than any other octile for the Regress TQI. Further, I found that 40% of all of the highest octile schools were located in the Region I Education Service Center part of the area of the state commonly referred to as The Valley. This area has lower than predicted teacher attrition and turnover, thus slightly greater than predicted TQI ratings based on student demographics. Because so many of the highest octile schools were predominantly Hispanic, I then focused on predominantly Hispanic schools those with at least 50% Hispanic students enrolled. For the Regress TQI, the highest octile schools had the greatest percentage of Hispanic and bilingual education students. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 27

29 Bilingual Bilingual Bilingual.In analyzing the average TAKS z-scores by octiles of the TQI ratings for only predominantly Hispanic schools, the relationship between TAKS scores and TQI ratings is more direct. In others, in general, the greater the TQI ratings, the greater the TAKS z-scores. 28 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

30 Middle Schools As shown in Table 9, there is a strong relationship between each TQI rating and the average TAKS combined mathematics and reading z-scores for middle schools. The differences in achievement between the schools with the lowest and highest TQI ratings are quite substantial approximately 1.5 standard deviations. The correlations between the TQI ratings and TAKS achievement were moderate to high. High Schools As shown in Table 10, there is a strong relationship between each TQI rating and the average TAKS combined mathematics and reading z-scores for high schools. The differences in achievement between the schools with the lowest and highest TQI ratings are rather striking approximately 2.0 standard deviations. The correlations between the TQI ratings and TAKS achievement were quite high for education research greater than.500. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 29

31 Findings Section IV: Distribution of TQI Ratings by School Accountability Rating and Student Demographics.The purpose of this section was to examine if teacher quality was equitably distributed across schools with different accountability ratings and that serve different populations of economically disadvantaged students and minority students. To do so, I present a series of tables for each school level that examine the TQI rating across schools with different accountability ratings, percentages of economically disadvantaged students, and percentages of minority students. Before presenting the findings, I describe the data employed in this section. School accountability ratings were taken from the AEIS school reference data file downloaded from the TEA website. The criteria for determining school accountability ratings can be found at: The ratings were after Required Improvement, the Texas Projection Measure, and Exceptions were applied to the data by TEA. In this analysis, the purpose of the accountability ratings was not necessarily to identify low- and high-performing schools because the TAKS score analysis does that far more accurately than the accountability ratings. This was especially true given the concerns over the accuracy of the Texas Projection Measure. Rather, this study employs school accountability ratings because teachers perceive that schools with higher ratings are easier places to teach. Holding all other factors equal, most teachers would prefer to become employed in schools that are easier places to work. Now, teachers perceptions may change as more doubt is cast on the validity of the accountability ratings, but teachers will typically choose a school with a higher accountability rating if all other factors are held equal. The percentage of economically disadvantaged students was constructed using data from the AEIS data file on student demographics. A student was determined to be economically disadvantaged if he/she participated in the federal free- or reduced-priced lunch program or met other criteria as determined by TEA. At each school level, the total sample of schools was divided equally into quintiles with each quintile having roughly the same number of schools. The percentage of minority students was constructed using data from the AEIS data file on student demographics. The percentage of minority students was calculated as the sum of the percentage of Hispanic students and the percentage of African American students. The total sample of schools was divided equally into quintiles with each quintile having roughly the same number of schools. Elementary Schools Accountability Ratings As with the analysis by student achievement, schools with greater achievement as measured by accountability ratings had greater TQI ratings than schools with lower achievement as measured by accountability ratings. Specifically, the differences in TQI ratings between schools rated Exemplary and schools rated Academically Unacceptable were between 0.65 and 0.95 standard deviations. However, because the number of academically unacceptable schools was very small, the results of this analysis must be interpreted with caution. 30 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

32 Economically Disadvantaged Students.As shown in Table 12, the lowest-poverty schools (those with less than 33.3% students participating in the free- and reduced-price lunch program) had a slightly greater Regress TQI and much greater average TQI ratings than the highestpoverty schools (those with more than 91.3% students participating in the free- and reduced-price lunch program). For the Regress TQI, there was little variation across schools by poverty level. In contrast, there was almost a one standard deviation difference between the lowest- and highest-poverty schools in the average TQI ratings. Part of this small difference in the Regress TQI is due to the correlation between the percentage of economically disadvantaged students and the percentage of Hispanic students. Four region service centers ESC 1 (Edinburg), ESC 2 (Corpus Christi), ESC 19 (El Paso), and ESC 20 (San Antonio) enrolled 44% of all Hispanic students and employed 60% of all Hispanic teachers. While these regions have seen tremendous growth over the past decade, which would require the hiring of new teachers, Hispanic teachers have the lowest attrition rate of any teachers in Texas. Thus, many schools in these region ESCs have low teacher attrition rates and low percentages of novice teachers. Hence, such schools have relatively greater Regress TQI ratings than one would expect given the student population..as shown in Table 13, the difference in the Regress TQI between the lowest- and highest-poverty schools increases to.251 from.154 when the four aforementioned predominantly Hispanic region ESCs are removed from the analysis. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 31

33 Minority Students.In this analysis, the percentage of minority students is the combined percentage of African American and Hispanic students enrolled in a school. Again, as with the economically disadvantaged student analysis, the difference in the Regress TQI was relatively small while the differences in the average TQI ratings were much larger. Again, as with the previous analysis, the removal of the four predominantly Hispanic regions increases the Regress TQI from.119 standard deviations to.313 standard deviations..table 16 below presents by school achievement and student characteristics the percentage of novice teachers and teachers who had taught at least three of the last five years. Novice teachers were those who had taught three or fewer years. This experience could have been in Texas public or private schools or in a public or private school in another state. Teachers who have taught at least three out of the last five years were those who have been employed in Texas public schools at least three of the five years between and As shown, lower-performing, high-poverty schools, and schools with high percentages of minority students had greater percentages of novice teachers and lower percentages of teachers who had taught at least three of the past five academic years than higher-performing, low-poverty schools, and schools with low percentages of minority students. 32 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

34 - Middle Schools and the Distribution of TQI Ratings Accountability Ratings.Not surprisingly, there was a large difference in the TQI ratings between schools rated Exemplary and those rated Academically Unacceptable. The Regress TQI difference was.631 standard deviations while the differences in average TQI ratings were around 1.0 standard deviation. However, caution must be used in interpreting this analysis because only eight schools were rated Exemplary too small of a number to arrive at an accurate estimate of the average TQI for the group of schools. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 33

35 Without a doubt, the Academically Unacceptable schools had low TQI ratings generally around -.5 standard deviations below the average TQI. For example, as shown below in Figure 2, the Academically Unacceptable schools had lower percentages of teachers with more than one year of experience, who held a full state certificate, were assigned in-field, remained at the same school from to , and remained at the same school for at least three of the last five years Not Fully Beginning Certified In-Field Same Same School 1 School 3+ Yr AU Recognized Economically Disadvantaged Students.As shown in Table 18, there were substantial differences in the TQI ratings between low-poverty schools (less than 42.4% economically disadvantaged students) and high-poverty schools (those with greater than 89.4% economically disadvantaged students). The difference in the Regress TQI was about.5 standard deviations while the differences for the average TQI ratings were at least 1.0 standard deviation..the most pronounced differences between the groups of schools were in the percentage of teachers who were assigned out-of-field. The lowest-poverty schools had only 23% of teacher FTEs assigned out-of-field as compared to 31% in the highest-poverty schools. 34 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

36 Minority Students As shown in Table 19, the results across schools with different percentages of minority students were fairly similar to that for economically disadvantaged students. The difference in the Regress TQI between predominantly non-minority schools (those with less than 44.5% minority students) and predominantly minority schools (those with greater than 97.5% minority students) was almost.4 standard deviations. The differences for the average TQI ratings were greater around 1.0 standard deviations. High Schools and the Distribution of TQI Ratings Accountability Ratings Not surprisingly, low-performing schools identified as Academically Unacceptable by the state accountability rating have lower TQI ratings than schools rated Exemplary. The TQI ratings differences in this analysis are at least 1.5 standard deviations. Academically Unacceptable schools had TQI ratings of about -.8 standard deviations below average while Exemplary schools had TQI ratings of about.7 standard deviations. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 35

37 In terms of actual teacher characteristics, Academically Unacceptable schools had far greater percentages of beginning teachers, novice teachers, teachers assigned out-of-field, and teachers not fully certified. Nearly 30% of teachers in Academically Unacceptable schools were novice teachers and 25% were assigned out-of-field (did not have the proper full state certificate for their teaching assignment). In comparison, Exemplary schools had less than 15% novice teachers and fewer than 15% of the teachers were assigned out-of-field Beginning Novice Out-of-Field Not Fully Certified Acad Unacceptable Exemplary Economically Disadvantaged Students.As shown in Table 21, the poverty level of a school and the TQI rating of the school is inversely related the greater the poverty, the lower the rating. The highest poverty schools (those with greater than 89.4% of economically disadvantaged students) had TQI ratings at least 1.5 standard deviations lower than the TQI ratings for the lowest poverty schools (those with less than 42.4% economically disadvantaged schools). The lowest poverty schools have TQI ratings of about.6 standard deviations while the lowest-performing schools had TQI ratings between -.87 and -1.0 standard deviations below the average TQI ratings. The relationship between school poverty and TQI ratings should not be surprising since there is a strong correlation between the percentage of economically disadvantaged students enrolled in a school and the achievement level of the school. 36 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

38 For example, 80% of the lowest poverty high schools are in the high- and highest-performing groups of schools while 90% of the highest-poverty high schools are in the low- and lowest-performing groups of schools. In other words, schools with high percentages of economically disadvantaged students have low student achievement and low teacher quality while schools with low percentages of economically disadvantaged students have high student achievement and high teacher quality. Minority Students As shown in Table 22, schools with the lowest minority enrollment (less than 44.5% minority students) had TQI ratings at least 1.3 standard deviations greater than schools with the greatest minority enrollment (those with greater than 97.5% minority students). The schools with the lowest percentage of minority students had TQI ratings of about.45 standard deviations above average while schools with the greatest percentage of minority students had TQI ratings of less than -.8 standard deviations below average. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 37

39 Teacher Quality Index by Region Education Service Center Area.Texas is divided into 20 region education service centers, as shown below in Figure 5 (from the TEA Website). The locations of the region headquarters are shown in Table Study on the Distribution of Teacher Quality in Texas Schools fall 2010

40 Elementary Schools As shown below in Table 24, teacher quality varies across regions at all three levels of schooling. At the elementary level, regions 5, 8, 14, and 15 had the highest TQI ratings while regions 1 and 4 had the lowest TQI ratings. The regions with the highest ratings tended to be more rural in nature, have smaller districts and schools, and had stable or declining student enrollments, while regions 1 and 4 were much more urban, predominantly minority, and very fast-growing with large districts and schools. B The El Paso region (Region 19 ESC) had essentially no difference in the Avg 1 TQI ratings between low- and high-poverty schools. After closer inspection, the low Avg 1 TQI rating across the eight low-poverty schools was largely due to one school having a rating of almost -3.0 standard deviations. After removal of this one school, the difference in the Avg 1 TQI rating between low- and high-poverty elementary schools increased to approximately 0.26 standard deviations. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 39

41 Middle Schools At the middle school level, regions 14, 15, and 17 had higher TQI ratings while regions 1, 3, 4, 10, and 12 had lower TQI ratings. Again, the regions with the highest ratings tended to be more rural in nature, have smaller districts and schools, and had stable or declining student enrollments, while regions with the lowest TQI ratings were much more urban, enrolled more minority and poor students, and were faster growing than other regions. TABLE 25: Middle School TQI Ratings and Difference in TQI Ratings Between Low- and High-Poverty Schools by Region Education Service Center 40 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

42 High Schools.Finally, at the high school level, regions 11, 13, and 14 had higher TQI ratings while regions 1, 4, 5, 17, 18, and 19 had lower TQI ratings. Unlike at the lower levels of schooling, there is less commonality among the higher- and lower-performing regions. TABLE 26: High School TQI Ratings and Difference in TQI Ratings Between Low- and High-Poverty Schools by Region Education Service Center fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 41

43 Teacher Quality Index by Metro Area and School District.The analysis shown in Table 27 examined the distribution of teacher quality for secondary schools across five major metro areas. The metro areas include the major urban district as well as all districts contiguous to the major urban district of interest..in all five major metro areas, there was a substantial difference in the three TQI ratings between the lowest- and highestpoverty schools. Indeed, the smallest difference was standard deviations while the greatest difference was 1.9 standard deviations. Thus, there was a substantial degree of inequity in access to teacher quality across secondary schools in all five major metro areas. s 42 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

44 This analysis reports on the Avg 1 TQI ratings between high- and low-poverty schools in middle schools in selected large districts from around Texas. High- and low-poverty status is not a set standard but rather relative to the district. For example, the low-poverty schools in Aldine ISD had an average percentage of economically disadvantaged students greater than the high-poverty schools in Clear Creek ISD..Almost all districts had a fairly substantial difference in TQI ratings between low- and high-poverty schools. Three districts had fairly insignificant differences: Dallas ISD, Garland ISD, and Katy ISD. Interestingly, in Ysleta ISD, high-poverty schools had a greater TQI than low-poverty schools. Further investigation into this phenomenon should be undertaken. Despite these four outlier districts, most districts evidenced the same trend as across the entire state: students in highpoverty schools had less access to well-qualified teachers than students in low-poverty schools. TABLE 28: Average TQI and Student Demographics for Low-Poverty and High-Poverty Middle Schools in Selected Major Urban Districts As shown in Table 29, most districts had a fairly substantial difference in the average TQI between low- and high-poverty schools. Interestingly, two districts had greater TQI ratings for high-poverty schools than for low-poverty schools: Clear Creek ISD and Lewisville ISD. The difference in the percentage of economically disadvantaged students in Clear Creek between low- and high-poverty schools was only marginal, which may explain the difference in the TQI ratings. However, the difference in the percentage of economically disadvantaged students between low- and high-poverty schools for Lewisville ISD was relatively large, yet Lewisville ISD had provided greater teacher quality to the high-poverty schools. One possible explanation for these two districts is that the districts are fast-growth and have opened new schools. New schools often have lower teacher quality even while serving more affluent students. Further investigation into these two districts is warranted. Despite the findings for Clear Creek and Lewisville ISDs, almost three-quarters of all of the districts (15 of 21) had differences greater than 0.5 standard deviations. Thus, most districts had fairly inequitable distributions of teacher quality. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 43

45 TABLE 29: Average TQI and Student Demographics for Low-Poverty and High-Poverty High Schools in Selected Major Urban Districts 44 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

46 Conclusions This study reaches several important conclusions that should have implications for state-, district-, and school-level policy efforts to improve student achievement. First and foremost, this study strongly suggests that overall teacher quality measures such as a TQI are associated with student achievement at the secondary level, even after controlling for school demographics, geographic location, and prior student achievement. Further, there is some indication that teacher quality matters at the elementary-school level as well, but the available data on teachers at the elementary level is likely not detailed enough to identify the relationship between teacher quality and student achievement. Second, the relationship between school TQI ratings and student achievement increases in strength as the school level increases. Thus, the relationship is weakest for elementary schools and strongest for high schools. This is due, in part, to the oversupply of elementary teachers and the shortage of high school teachers, particularly those in mathematics and science..third, TQI ratings and student achievement on TAKS are positively associated as TQI ratings increase, school achievement increases. Likewise, as school achievement increases, so do TQI ratings. Unfortunately, this analysis cannot disentangle whether low TQI ratings cause low student achievement or schools with low student achievement have difficulty in recruiting high-quality teachers. The most likely scenario is that the measures are mutually reinforcing. As teacher quality decreases, student achievement decreases relative to other schools, which, in turn, makes recruiting and retaining well-qualified teachers at the school more difficult. Thus, a vicious cycle is created that is difficult to break without some rather dramatic policy interventions..fourth, schools rated Academically Unacceptable have substantially lower TQI ratings than schools rated Exemplary. Indeed, at the high school level, the difference in TQI ratings between schools rated Exemplary and schools rated Academically Unacceptable was a staggering 1.5 standard deviations. Further, the greater the number of times a school was rated Academically Unacceptable, the lower the TQI rating (as shown in Table 30)..Fifth, secondary school TQI ratings are negatively associated with the percentage of economically disadvantaged students in the school. In other words, as the percentage of economically disadvantaged students increases, the TQI ratings decrease. Sixth, secondary school TQI ratings are negatively associated with the percentage of minority students in the school. In other words, as the percentage of minority students increases, the TQI ratings decrease. TQI ratings are particularly low in schools serving high proportions of African American students. In fact, out of the 50 high schools with the largest percentages of African American students, only one had a TQI rating greater than average. Seventh, there are substantial differences in TQI ratings both between schools within urban districts and between schools within the larger metro areas. In other words, not only are there inequitable distributions within districts, there are also inequitable distributions between districts in the same local labor markets. Thus, policies must not focus specifically at either state- or district-level policies, but rather at both levels and at the school level. Finally, particular measures of teacher quality were more strongly associated with student achievement than others. In particular, the percentage of novice or beginning teachers was associated with achievement at all three levels. This was fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 45

47 consistent with other research from across the country. Further, the percentage of teachers assigned to teach out-of-field and the percentage of teachers staying at the same school were both statistically significantly related to student achievement at the middle- and high-school level. Again, these findings were consistent with findings from other research from across the country. Discussion The results above unambiguously reveal a substantial inequitable distribution of teacher quality across the state at the middle- and high-school level. Clearly, students in low-performing schools as well as in high-poverty and predominantly minority schools have far less access to the same levels of teacher quality as students in high-performing, low-poverty, and predominantly White schools. Moreover, this finding holds true for schools that are within driving distance of one another both within the same district and across district lines between contiguous districts. While this study did not find that the differences in access to teacher quality were as large at the elementary school level as at the secondary levels, more detailed data may show gaps as large as at the other school levels. In particular, if the state could produce school-level data on the verbal ability of teachers (SAT/ACT verbal scores) and certification scores of teachers by content area on the generalist test, an examination of the data could reveal very disparate levels of teacher quality across schools. Without such data, readers should be cautious in concluding there is an equitable distribution of teacher quality at the elementary-school level. The dramatic differences in access to teacher quality especially at the middle- and high-school levels surely help explain the continuing gaps in achievement, graduation, and college attendance between economically disadvantaged and non-economically disadvantaged students and between non-white and White students in Texas public schools. Unless the state addresses these long-standing disparities with courageous leadership, the state will find itself slowly slipping further and further into mediocrity both educationally and economically. Policy Recommendations This final section of the paper offers some state and district policy recommendations. Because research has shown inequities in teacher quality exist between districts, between schools within individual districts, and between student subpopulations within individual schools, policymakers and leaders at all three levels must take action. This effort to improve teacher quality and reduce the inequitable access to teacher quality should be led by those at the state level. As such, the governor, legislative leaders, State Board of Education members, State Board for Educator Certification members, and the commissioner of education should make the issue of teacher quality and distribution of teacher quality a high priority. Yet, this does not mean that the policies and strategies should be developed by state-level policymakers. Indeed, evaluations of reform efforts from the past 100 years come to the same conclusion: Reform efforts simply do not work unless they involve teachers and administrators in developing and implementing the reform effort. This bears repeating: Top-down reform efforts that exclude or ignore the ideas, perceptions, values, and beliefs of educators are doomed to fail. Thus, state leadership should play four crucial roles: Make the issue of teacher quality a high priority by highlighting the importance of teacher quality and documenting the inequitable access to teacher quality; Gather input from teachers, administrators, researchers, and policymakers from across the state by convening meetings across the state to gather input on this issue; Support the implementation of policies that address teacher quality issues through targeted funding and technical assistance; and, Evaluate state and district efforts to increase teacher quality. State leaders should be credited for several recent efforts in this area. First, the Legislature eliminated funding for the poorly designed Governor s Educator Excellence Grant/Texas Educator Excellence Grant and transferred that funding 46 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

48 into the District Awards for Teacher Excellence (DATE) grant program. If the requirements and/or incentives in the DATE grant were properly constructed, the fund could have the potential to improve the distribution of teachers. Indeed, districts such as Round Rock ISD have structured DATE grants to reward teachers for returning each year to harder-to-staff schools and to provide larger performance awards to the harder-to-staff schools than to other schools. However, TEA must be diligent in reviewing these programs because a DATE grant could be used to construct a program that actually exacerbates the inequitable distribution of teachers, thus making a bad situation even worse. Second, the Legislature passed a bill that dramatically expands and improves the teacher accountability system for teacher preparation programs. Most importantly, the bill increases the amount of information available on the quality of teacher preparation programs across the state. If all teachers prepared by preparation programs were of greater quality, districts would have a greater supply of high-quality teachers to hire..districts working on this issue should be commended as well. In particular, districts such as Round Rock ISD and Austin ISD that are using DATE funds to address the inequitable distribution of teachers should be recognized for their efforts. While we do not yet know whether these district efforts will equalize teacher quality and improve student achievement, the districts should be recognized for employing thoughtful efforts to address the issue. Yet, there is still far more work to do. The following recommendations are intended to move the conversations and efforts as we endeavor to provide every child a well-qualified and effective teacher. State-Level Policy Recommendations The following recommendations are targeted to state policymakers in the governor s office and Legislature and at the State Board of Education, State Board for Educator Certification, the Texas Education Agency, and the Texas Higher Education Coordinating Board. Fund and support the gathering of input from teachers and administrators about how to improve teacher quality and more equitably distribute teacher quality..the state should hire a group of education experts (some of whom should have experience as teachers, principals, and central office administrators in Texas public schools) to travel the state and convene groups of educators in order to gather their input on how to best improve teacher quality and lessen the inequitable distribution of teacher quality. The results of this effort should drive state policy. An excellent model for such an effort is the Teacher Leaders Network organized by Dr. Barnett Berry of the Center for Teaching Quality ( This effort brings together teacher leaders from across the nation in a virtual network to share best practices, provide support, and push for policy changes that support teachers. Create an annual statewide report that analyzes the aggregate TQI and the individual TQI components, and provide the overall results to the public and the individual school reports to district personnel..unfortunately, the state has not highlighted the distribution of teacher quality and the trends in teacher quality over time. Unless the state publicly raises the issue, the issue will remain low on the priority list of state and district policymakers. One important step that state leaders could take is to start a conversation with district administrators about the difference between highly qualified teachers and teacher quality. According to TEA (2010), a highly qualified teacher is a teacher that meets the following requirements: Has obtained full Texas teacher certification, including appropriate special education certification for special education teachers; Holds a minimum of a bachelor s degree; and Has demonstrated subject-matter competency in each of the academic subjects in which the teacher teaches. In this definition, full state certification does not mean that a teacher is fully certified in the sense that the teacher holds a standard certificate but rather holds any type of certificate other than a permit that is granted by the State Board for Educator Certification. The analysis in this study relied on a full standard certificate. In fact, when serving as the Co-Directors of Research at the State Board for Educator Certification, Alexander and Fuller (2004) found that Texas middle school mathematics teachers who had obtained full standard certificates in mathematics were more effective at increasing student achievement than teachers with other full state certificates. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 47

49 Further, this definition requires teachers to demonstrate subject-matter competency through either a major in the field of study or a passing score on a Texas state certification examination for a particular content area. This leaves open the possibility that a person could demonstrate competency by correctly answering 70% of items on a certification examination that arguably could be passed by an honors Algebra II student in a high-performing high school in Texas. In addition, the state only requires alternative certification teachers to have 12 hours of undergraduate coursework in a content area to be considered highly qualified, and the counting of these hours is left to the discretion of those working in the alternative certification programs. Because these requirements are so lax, almost every teacher in Texas (and the nation) is considered highly qualified, but this measure is not based on any empirical evidence related to student achievement. Unfortunately, district leaders rarely acknowledge that they have a problem with distribution of teacher quality because all schools have nearly 100% highly qualified teachers. The state should impress upon district leaders the need to focus on the measures in this report rather than highly qualified teacher status. Provide monetary incentives for districts to address TQI inequities within their own district, and increase the flexibility districts have in addressing their unique needs..this effort holds great promise if districts design and implement the program in thoughtful ways that address the issues brought forth by this study. TEA should provide incentives for districts to specifically address inequities in teacher quality across schools within their district. Further, TEA should allow districts greater flexibility in how to spend money to improve the distribution of teacher quality across campuses. This could be accomplished by restructuring the DATE grant incentives or creating an alternative program that exists alongside the DATE grant program. Adopt and fund a new cost-of-education index..the cost-of-education index (CEI) was created in the early 1990s to provide funding to districts that had difficulty in hiring well-qualified teachers due to factors outside the control of district leaders such as the percentage of economically disadvantaged students. The CEI has never been updated despite repeated efforts by some policymakers and researchers to do so. Thus, some districts receive far more money than they should while other districts do not receive nearly the amount they would based on an updated CEI..The primary hurdle to updating the CEI is cost. The state should construct a new school finance system with an updated CEI and implement the changes over time so that no district loses a substantial amount of money in a short period of time. Support the creation of urban teacher academies in the 10 largest metro areas across the state..urban teacher academies provide opportunities for newly certified teachers to learn under the guidance of master teachers how to be effective teachers of low-performing and economically disadvantaged students. These programs have been shown to be quite effective in building on the training provided by high-quality teacher preparation programs. For more information, see Create an incentive program for preparation programs to produce teachers that meet the demand in their local labor market. Currently, there is no incentive for programs to produce a high school mathematics teacher as opposed to an elementary teachers, even though there is a shortage of mathematics teachers and a surplus of elementary teachers. Increase the requirements to enter teacher preparation programs in Texas, especially alternative certification programs that tend to have lower entrance requirements than traditional university-based or post-baccalaureate programs..while recent additions to the accountability system for educator preparation have dramatically improved the measures used to identify effective preparation programs, there is still room for improvement. For example, the entrance requirements for many alternative certification programs are still abysmally low. Further, some individuals can enter and complete an alternative certification and become employed as a middle- or high-school teacher with as little as 12 undergraduate credit hours in the subject area in which they obtained certification. Yet, individuals from traditional certification programs must complete a major in the subject area in which they obtain certification. Perhaps the state should require a minimum of 24 hours and allow programs to decide on additional content requirements. Finally, the state should require a closely supervised field experience for all teachers, even those from alternative certification programs. 48 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

50 Fund a statewide working conditions study, and encourage all schools to participate in the study. Recent research has found that teacher working conditions have a significant impact on teacher effectiveness, as measured by gains in student achievement, and are the primary factor in improving teacher retention. Without addressing the issue of teacher working conditions, all other efforts to increase the degree of equity in the distribution of teachers will surely fail. Thus, the state needs to fund a high-quality working conditions survey from an organization that can ensure valid and reliable results for schools, districts, and the state. The data should be provided back to schools and districts, and training should be provided on how to use such tools to improve working conditions and improve the equitable distribution of teacher quality. Improve the training of school leaders..research on the relationship between working conditions and teacher turnover has consistently found that school leadership behaviors are the primary factor affecting teachers decisions to stay at or leave a particular school. These behaviors are the underlying force behind the powerful effect that working conditions have on teacher retention. In fact, leadership behavior is a stronger predictor of teacher retention than either student demographics or student achievement. Better training of school leaders concomitant with other changes can increase the likelihood that more schools will have the type of school leader that attracts well-qualified and effective teachers, regardless of the school characteristics. Improve data collection and dissemination efforts..while Texas used to be recognized for having one of the best education data systems in the country, the state has fallen behind a number of other states. Texas has invested additional money and effort into upgrading data systems, including matching students to teachers in every school in the state. Yet, TEA has not addressed some substantial issues regarding existing and missing data related to teacher quality. The state should bring researchers and data system experts together to identify weaknesses and potential solutions to improving the current data on educators. Researchers need better data and greater access to data on the background characteristics of all teachers employed, such as undergraduate institution, grade point average, SAT/ACT scores, certification scores, type of master s degree, and the major/minor for the undergraduate degree. Further, the state needs to invest in creating more accurate data on teacher experience and certification status. Some of this data is currently available through the Education Research Centers at the University of Texas at Austin, the University of Texas at Dallas, and Texas A&M University. Yet, the state could create more efficient and effective ways to make this data accessible without violating the confidentiality of individuals. Provide school-level value-added data..the general consensus of researchers on the accuracy of teacher-level value-added efforts is that such systems are generally not accurate and stable enough to make high-stakes decisions about teachers. However, the state should also provide useful school-level value-added information for each grade level and subject area for which information is available. Moreover, the state should model appropriate use of this data and provide training on how to correctly interpret and use such information in an appropriate manner. The data should not be used for value-added judgments until researchers can decrease the error rate in identifying teachers who are effective and ineffective in raising student achievement. Improve the school accountability system..currently, the state s school accountability system provides a disincentive for well-qualified and effective teachers to move to low-performing schools. The primary driver of this disincentive is the absence of an accurate barometer and recognition of student growth. While the state has implemented measures including Required Improvement and the Texas Projection Measure in an effort to adjust for student growth, both measures have serious methodological flaws and are clearly inferior to having an actual measure of student growth as one component of the school accountability system. Develop a statewide campaign designed to increase the prestige of the teaching profession..currently, many prospective teachers do not view the teaching profession as a prestigious one. A statewide campaign coupled with greater barriers to entry that increase the overall quality of teachers can increase the prestige of the profession and increase the supply of better-qualified entrants into the profession. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 49

51 District-Level Policy Recommendations The following recommendations are targeted to district-level policymakers. Identify and report on the distribution of teacher quality across schools..districts should ignore the highly qualified teacher information and instead focus on the qualifications of teachers associated with student achievement such as the ones included in this study. Acknowledging the existence of a problem is the first step toward solving the problem. Gather input from teachers and principals about the strategies that would be successful in improving teacher quality and increasing the degree of equity in the distribution of teacher quality..as noted above, reform efforts are only successful if those affected by the reform are intimately involved in the development and implementation of the effort. Thus, districts should create methods to gather the perceptions and suggestions of teachers and principals about how to best improve teacher quality and improve the degree of equity in the distribution of teacher quality. Implement a teacher working conditions survey, and work with principals to ensure all schools have positive working conditions..while the state should fund a statewide effort, districts can certainly embark on efforts of their own. Districts should be careful, however, in collecting and analyzing the data and should make certain the data is used to help improve practices rather than punish. Indeed, such data should be collected anonymously. Provide additional fiscal, instructional, and human resources to high-poverty, predominantly minority, and low-performing schools..while the state is ultimately responsible for ensuring all schools have access to the resources they need, districts have a large responsibility to ensure that schools serving the students with the most needs have access to the necessary fiscal, instructional, and human resources necessary to be successful. For example, this could mean ensuring that high-poverty schools are funded at greater levels than low-poverty schools, that incentives are provided to well-qualified teachers to teach in hard-to-staff schools, and that the neediest schools have the best instructional materials possible. Create incentives and support mechanisms to ensure principal stability at high-poverty, predominantly minority, and low-performing schools..recent research has shown that instability of principals is inextricably linked to instability of teachers. Principal turnover is extraordinarily high at all schools and greatest in high-poverty and low-performing schools. District leaders need to create incentives and support mechanisms to ensure much greater stability in school leadership. Without such stability, increasing teacher quality and sustaining school improvement efforts are nearly impossible. Invest in a high-quality teacher mentoring and induction system such as TxBESS..Mentoring and induction programs can have a significant positive effect on the retention of beginning teachers. In fact, Cohen and Fuller (2006) found that teachers in hard-to-staff schools participating in TxBESS were 60% less likely to leave after their first year of teaching than other teachers, even after controlling for other factors. 50 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

52 Appendix A Ordinary Least Squares Regression Results The results of the ordinary least-squares regression results are shown below. Each variable was entered into the regression equation individually. The variables were entered individually because of the high correlation among the variables and the resulting multi-collinearity problems that arose when entering variables simultaneously to 09 Coefficients in bold are statistically significant. Elements of the TQI Ratings by School Level Elementary Schools At the elementary-school level, four variables were found to be statistically significantly related to student achievement: percentage of novice teachers; percentage of teachers who have taught in a Texas public school in three of the past five years; the average one-year teacher turnover rate from to , to , and to ; and the one-year teacher turnover rate from to fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 51

53 Because three of the four variables were related to teacher stability, a TQI based on all four of the statistically significant measures would have heavily weighted teacher stability. Because teacher stability had the least research evidence linking it with student achievement, the decision was made to use only one of the teacher stability measures the percentage of teachers who have taught in a Texas public school in three of the past five years. This measure was chosen because it had the strongest relationship with student achievement..ultimately, there would have been little difference in the final Regress TQI if all four variables had been employed in the construction of the TQI. Indeed, the correlation between the two-variable and four-variable Regress TQIs was.907. The actual weights used are provided in the table. The Avg 1 TQI used the two Regress TQI components and added a third and fourth variable: the percentage of teachers fully certified and the percentage of teachers with a pedagogy certification score at least one standard deviation greater than the average pedagogy certification scores. Finally, the Avg 2 TQI added a final variable related to teacher retention: the average one-year teacher retention rate for three consecutive years to 09 (3) NSS = Measure was found not to be statistically significant in OLS regression analysis. SS = Measure was found to be statistically significant in OLS regression analysis. When placed into a regression analysis that controlled for both prior achievement and student demographics and employed region ESC fixed effects, the Regress TQI was statistically significant at the p <.01 level, while the Avg 1 TQI was statistically significant at the more lenient p <.10 level. Finally, the Avg 2 TQI was statistically significant at the p <.05 level. When placed as the initial variable in the equation, the TQI explained between 3% and 13% of the variance in student achievement. When placed after control variables were entered, the TQI ratings explained roughly 1% of the variance in achievement. 52 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

54 Middle Schools.At the middle-school level, four variables were found to be statistically significantly related to student achievement: percentage of beginning teachers, percentage of teachers fully certified, percentage of teachers assigned out-of-field, and percentage of teachers employed at the same school from to These four variables constituted the Regress TQI. The Avg 1 TQI used the four Regress TQI components and added a fifth variable: the percentage of teachers with pedagogy certification scores at least one standard deviation greater than the average pedagogy certification scores. Finally, the Avg 2 TQI added a final variable related to teacher retention: the average one-year teacher retention rate at the school for three consecutive years. The two Regress variables were statistically significantly related to student TAKS achievement, while three of the four Avg 1 TQI variables were statistically significantly related to student TAKS achievement. Four of the five variables in the Avg 2 TQI were statistically significantly related to student TAKS achievement to 09 (3) NSS = Measure was found not to be statistically significant in OLS regression analysis. SS = Measure was found to be statistically significant in OLS regression analysis. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 53

55 When placed into a regression analysis that controlled for both prior achievement and student demographics and employed region ESC fixed effects, all three TQI ratings were statistically significantly related to student achievement. The Regress TQI and Avg 2 TQI were both statistically significant at the p <.001 level while the Avg 1 TQI was statistically significant at the p <.01. When placed as the initial variable in the equation, the TQI explained between 10% and 24% of the variance in student achievement. When placed after control variables were entered, the TQI ratings explained between 2% and 5% of the variance in achievement. High Schools At the high-school level, 12 variables were found to be statistically significantly related to student achievement but only 10 were included in the creation of the Regress TQI. Two variables were excluded because they were either the inverse of a variable already included in the analysis (for example, the percentage of teachers fully certified and the percentage of teachers not fully certified are the inverse of each other) or the variables were highly correlated with each other. The ten variables that constituted the Regress TQI are: 1. The percentage of novice teachers; 2. Percentage of beginning teachers; 3. Percentage of teachers fully certified; 4. Percentage of teachers assigned out-of-field; 5.. Percentage of teachers from high-performing preparation programs; 6.. Percentage of teachers at the school with pedagogy scores at least one standard deviation greater than average; 7.. Percentage of teachers at the school with pedagogy scores at least one standard deviation lower than average; 8.. Percentage of teachers teaching at least three of the past five academic years; 9. Average percentage of teachers remaining employed at the same school from one year to the next over a four year time span; and, 10. Percentage of teachers remaining at the school for at least three of the past five academic years. The Avg 1 TQI used six of the eight Regress TQI components but excluded the last two statistically significant variables related to teacher stability. In addition, the Avg 1 TQI included the percentage of beginning teachers at a school. Finally, the Avg 2 TQI included all of the eight Regress TQI variables as well as the percentage of beginning teachers at a school, the percentage of teachers teaching at least three of the past five academic years, the average percentage of teachers remaining employed at the same school from one year to the next over a four-year time span, and the percentage of teachers remaining at the school at least four of five years from the academic year to the academic year. 54 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

56 2005 to 09 NSS = Measure was found not to be statistically significant in OLS regression analysis. SS = Measure was found to be statistically significant in OLS regression analysis. SS-Exc = Measure was found to be statistically significant in OLS regression analysis but excluded because it was the inverse of another variable or highly correlated with another variable. When placed into a regression analysis that controlled for prior achievement, student demographics, and geographic location, all three TQI ratings were statistically significantly related to student achievement at the p <.001 level. When placed as the initial variable in the equation, the TQI explained between 29% and 33% of the variance in student achievement. When placed after control variables were entered, the TQI ratings explained about 2% of the variance in achievement. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 55

57 Appendix B 56 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

58 fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 57

59 58 Study on the Distribution of Teacher Quality in Texas Schools fall 2010

60 References Alexander, C. & Fuller, E.J. (2004, April). Does teacher certification matter? Teacher certification and middle school mathematics achievement in Texas. Presented at the annual meeting of the American Educational Research Association. San Diego, CA. Buddin, R. & Zamarro, G. (2010). What teacher characteristics affect student achievement? RAND: Santa Monica, CA. DeAngelis, J.B., Presley, B.R., & White, B.R. (2005). The Distribution of Teacher Quality in Illinois. Chicago, IL: Illinois Education Research Council. Dee, T. (2004). Teachers, Race and Student Achievement in a Randomized Experiment. Review of Economics and Statistics, 86(1), Fuller, E.J. & Alexander, C. (2003, April). Inequitable Access to Highly Qualified Teachers in Texas. Presented at the annual meeting of the American Educational Research Association. Chicago, IL. Fuller, E.J. & Brewer, C. (2005, Nov). The distribution of teachers and opportunity to learn. Presented at the annual meeting of the University Council for Educational Administration. Nashville, TN. Fuller, E.J. & Carpenter, B.W. (2009, March). Has Teacher Quality Changed Under NCLB? The Case of Texas Secondary Schools? Presented at the Annual Meeting of the American Educational Research Association. San Diego, CA. Fuller, E.J., Carpenter, B, & Fuller, G.A. (2008, August). Teacher Quality and Student Achievement in Texas Secondary Schools. Austin, TX: Association of Texas Professional Educators. Fuller, E.J. (2009, February). Distribution of Teacher Quality and Student Achievement in Texas. Presented to the Texas House Public Education Committee, February Fuller, E.J. (2008a). The Distribution of the Texas Teacher Quality Index in Texas. Unpublished manuscript. Austin, TX: The University of Texas at Austin. Fuller, E.J. (2008b, May). Teacher Shortages and Hard-to-Staff Schools in Texas. Prepared for the Texas House Public Education Committee. Austin, TX: University Council for Educational Administration and University of Texas Education Research Center. Fuller, E.J. (2007, February). Teacher Shortages and the Distribution of Teachers in Texas. Presented to the Texas House Public Education Committee. Fuller, E.J. (2003, July). Teacher Shortages and the Distribution of Teacher Quality in Texas. Austin, TX: State Board for Education Certification. Fuller, E.J. (2002, June). Teacher Supply and Demand in Texas. Prepared for the Texas House Public Education Committee. Austin, TX: State Board for Educator Certification. Futernick, K. (n.d.). Retrieved on 7/1/2010. Goldhaber, D., Gross, P., & Player, D. (2007). Are public schools really losing their best? Assessing the career transitions of teachers and their implications for the quality of the teacher workforce. (CALDER working paper). Goldhaber, D. & Brewer, D. (1997). Why don't schools and teachers seem to matter? Assessing the impact of unobservables on educational productivity. Journal of Human Resources, 32(3): Hanushek, R., Kain, J., O Brien, D., & Rivkin, S. (2005). The market for teacher quality. (Technical report). National Bureau of Economic Research. Ingersoll, R. & May, H. (2010). The magnitude, destinations and determinants of mathematics and science teacher turnover. Paper presented at the 2010 annual meeting of the Society for Research on Educational Effectiveness. Washington, DC. fall 2010 Study on the Distribution of Teacher Quality in Texas Schools 59

61 Ingersoll, R. & Smith, T. (2003). What are the effects of mentoring and induction on beginning teacher turnover? American Education Research Journal, 41(3), Jackson, C. K. & Bruegmann, E. (2009). Teaching students and teaching each other: The importance of peer learning for teachers. (NBER Working Paper 15202). Washington, DC: National Bureau of Economic Research. Jackson, C. K. (2010). Match quality, worker productivity, and worker mobility: Direct evidence from teachers. (NBER Working Paper 15990). Washington, DC: National Bureau of Economic Research. Johnson, S. M., Berg, J. H., & Donaldson, M. L. (2005). Who stays in teaching and why? A review of the literature on teacher retention. The Project on the Next Generation of Teachers, Harvard Graduate School of Education. Published in AARP s Education Community. (Available at Kane, T. J., Rockoff, J. E., & Staiger, D.O. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City. (Working Paper No ). Cambridge, MA: National Bureau of Economic Research. Keesler, V Estimating Cause: Teacher Turnover and School Effectiveness in Michigan. Paper presented at the annual conference of the Society for Research on Educational Effectiveness. Washington, DC. Ladd, H.F., Clotfelter, C.T., & Vigdor, J.L. (2010). Teacher credentials and student achievement in high school: A cross subject analysis with fixed effects. Journal of Human Resources 45(3): Ladd, H.F., Clotfelter, C.T., & Vigdor, J.L. (2007). How and why do teacher credentials matter for student achievement? (NBER Working Paper ). Levy, A.J., Ellis, P., Joy, L., Jablonski, E., & Karelitz, T.M. (2010). The causes and costs of turnover for science and other teachers. Paper to be presented at the 2010 American Educational Research Association Annual Meeting. Denver, CO. Marvel, J., Lyter, D. M., Peltola, P., Strizek, G. A., & Morton, B. A. (2007). Teacher attrition and mobility: Results from the Teacher Follow-up Survey. Washington, DC: National Center for Education Statistics. Meier, K.J & and Hicklin, A. (2007). Employee turnover and organizational performance: testing a hypothesis from classical public administration. Journal of Public Administration Research and Theory. 18, Ost, B. (2009). How Do Teachers Improve? The Relative Importance of Specific and General Human Capital. Ithaca, NY: Cornell University. Peske, H. & Haycock, E. (2006). Teaching Inequality: How Poor and Minority Students Are Shortchanged on Teacher Quality: A Report and Recommendations by the Education Trust. Washington, DC: The Education Trust. Presley, B.R., White, B.R., & Gong, Y. (2005). Examining the Distribution and Impact of Teacher Quality in Illinois. Chicago, IL: Illinois Education Research Council. Rice, J. K. (2003). Teacher Quality: Understanding the Effectiveness of Teacher Attributes. Washington, DC: Economic Policy Institute. Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2): Schochet, P.Z. & Hanley S.C. (2010). Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains (NCEE ). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Texas Education Agency. (2010). Highly Qualified Teacher homepage. Retrieved July 21, 2010 at tx.us/index4.aspx?id=4650&menu_id=798. The Education Trust. (2006). Missing the Mark: States Teacher Equity Plans Fall Short. Washington, DC: Author. Wayne, A. & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1): Study on the Distribution of Teacher Quality in Texas Schools fall 2010

62 2010 ATPE