Evaluating Teacher Effectiveness Strategies for Durham Public Schools

Transcription

1 To: Dr. Eric Becoats, Superintendent of Durham Public Schools From: Catalina Hidalgo Date: May 6 th, 2010 Re: Duke School Research Partnership Policy Brief on Teacher Effectiveness Strategies Evaluating Teacher Effectiveness Strategies for Durham Public Schools Introduction Durham Public Schools (DPS) seeks to improve student achievement on standardized tests through a teacher effectiveness model. Research has shown that teacher quality is one of the most influential factors for student learning [1]. This policy brief identifies key characteristics and practices that promote effective teaching, and explores school districts that incorporate teacher effectiveness research in their evaluations systems to improve student achievement. This policy brief accompanies a longer, more thorough research paper that explains the research reviewed, methodologies used to gather information, methodologies used in the research reviewed, and the limitations of these studies. Research Findings Researchers generally agree that teacher experience, teacher s subject-specific training, and standard certification are positively correlated with higher student achievement [2]. Studies also find that that teacher test scores on certification exams are a good predictor of teacher performance and student achievement, particularly in secondary mathematics [3]. However, the impact of advanced degrees on student achievement is inconclusive [4]. Research studies found that common teacher processes that have a positive association with improved student achievement include: the use of a variety of instructional practices; a teacher s expectations for student achievement; a teacher s use of high-order thinking skills in discussion and assignments; and content-focused professional development for teachers [5]. The ingredients to create an effective teacher are highly variable because the definition of effective teaching depends on the metrics used to measure a teacher s impact. Although there is no formal teacher effectiveness model, education administrators are identifying teacher effectiveness patterns at the local level through new teacher evaluation systems. A step toward a teacher effectiveness model is through valid teacher evaluation systems that reveal important aspects of teaching that promote student learning.

2 Recommendations for DPS 1. DPS could enrich teacher recruitment, induction programs, and professional development initiatives by emphasizing the following: Teacher mastery of subject-specific knowledge Individualized instruction and variation of instructional practices Demonstrating clear and measureable teacher expectations for individualized student achievement This recommendation draws from research that found a teacher s subject-specific training positively associated with higher student achievement [6]. Subject-specific knowledge plays an important role in quality teaching, particularly for middle and high school student achievement in mathematics [7]. While this may seem apparent, the research implies that those teachers who major in the subject they teach are more effective teachers than those who major in an unrelated or less specific field. Researchers find that the use of a broad selection of teaching approaches is most effective. A research study found that a teachers ability to individualize instruction to suit the student s needs improved student achievement [8]. Studies also found that highly effective teachers demonstrate high expectations for student performance and measure their students progress [9]. 2. DPS could focus teacher recruitment to those who are traditionally certified or to those who have qualifications similar to traditional certification. This recommendation draws from research that finds teacher preparation and certification important for student learning, particularly in mathematics [10]. Research also finds that teacher effectiveness highly varies among all those who are traditionally certified [11]. One study found that teachers with emergency or provisional certification did no worse than teachers with standard certifications [12]. However, a review of this study found that those teachers had qualifications similar to teachers with standard certification [13]. This information shows DPS that although standard certification is important, qualifications and training are equally important for effective teaching. 3. DPS could conduct a district-wide study to identify effective teaching qualities and strategies specific to its needs and objectives. Currently, this is not a formal model for teacher effectiveness in the United States. It is difficult to generalize the teacher inputs and processes research findings to DPS because the populations studied differ on various levels. The best way to capture effective teacher inputs and processes is to test the specific population. Previous studies provide building blocks for DPS to study its progress and find out what works best to boost student achievement [14]. The NC teacher evaluation tools can serve as a way to identify effective teacher practices once the system is validated for accuracy. DPS could monitor the reliability of its new evaluation tool and use it to capture effective teacher practices.

3 4. DPS could implement a teacher professional development program focused on feedback to teachers with a curriculum content-based approach. Research studies highlight the importance of teacher professional development programs as a means to boost effective teaching and promote student learning [15]. The research points to different aspects of teacher professional development programs that contributed to effective teaching, such as: support to incorporate higher-order thinking skills in the classroom; support to develop laboratory skills for science classes; and content-focused professional development. Teacher professional development was not exclusively examined because it is not focus of this research brief. Nonetheless, it would serve DPS to look into ways to promote professional development initiatives that aim to boost teacher effectiveness from a content- and curriculum-based approach. 5. DPS could explore other measures of student learning outside of standardized test scores. Student learning and teachers roles to improve student learning reach beyond the bounds of standardized tests. Although value-added data is an easy way to observe and calculate student achievement, researchers debate its limitations. Studies have shown that value-added estimates are unstable year to year due to several limitations such as the non-random assignment of students to classrooms, the small sample size in some classrooms, and out-of-school factors that influence student performance [16]. It is important for DPS to incorporate these discrepancies in policy and research discussions when examining future research results on teacher effectiveness. Summary of Literature Review The literature review examines twenty five research articles that study the relationship between teacher impact and student achievement. Teacher effectiveness is a teacher s ability to improve student learning through empirically defined measures, mainly improved standardized test scores. The research examines teacher inputs and teacher processes as variables that influence student achievement [17]. Teacher inputs are the characteristics and qualifications that teachers bring to the classroom. Teacher processes are the practices and instructional techniques used in the classroom. Impact of Teacher Inputs on Student Achievement This policy brief focuses on the impact of advance degrees, subject-area knowledge, certification qualifications, and years of experience on student achievement. Figure 1 illustrates the findings from the literature review. In addition to the four input components, a teacher s scores on standardized tests, such as certification exams and the SAT, were also found to be a good predictor of teacher performance and student achievement [18].

4 Figure 1: Diagram of teacher inputs. Impact of Teacher Processes on Student Achievement Teachers employ a variety of strategies to encourage student learning. This policy brief summarizes studies that examine the relationships between teaching practices and student achievement. It is important to note that the results are generally not statistically significant and do not follow rigorous research designs. Common teacher processes that have a positive association with improved student achievement include: the use of a variety of instructional practices; a teacher s expectations for student achievement; a teacher s use of high-order thinking skills in discussion and assignments; and contentfocused professional development for teachers [19]. Diagram 3 illustrates these common themes. Figure 2: Diagram of Teacher Processes Teacher Evaluation Systems as Step toward a Teacher Effectiveness Model Although there is no official teacher effectiveness model, education administrators are identifying teacher effectiveness patterns at the local level through new teacher evaluation systems. The examples in the following three sections illustrate how researchers and school administrators are working together to incorporate teacher effectiveness research in evaluation systems to identify key qualities and processes.

5 Measures of Effective Teaching (MET) Project Currently, a nation-wide study called the Measures of Effective Teaching (MET) Project is taking place to determine ways in which effective teaching can be measured fairly and consistently [20]. The project seeks to find a composite measure of teacher effectiveness, which will help identify key qualities that promote student learning. Researchers released preliminary findings in December 2010: Teachers who lead students to achievement gains in one year or in one class tend to do so in the future. Teachers with high value-added on state tests tend to promote deeper conceptual understanding in their students. Teachers have larger effects on math achievement than on reading or English achievement. Student perceptions of a given teacher s strengths and weaknesses are consistent among the students they teach. Student perceptions also seem to be accurate in identifying effective teaching. Charlotte-Mecklenburg Schools Student Learning Objectives Program Charlotte-Mecklenburg Schools (CMS) is taking innovative approaches to improve student learning through teacher evaluation and compensation reforms. The Student Learning Objectives (SLO) pilot program seeks to create an accurate evaluation tool to identify effective teachers [21]. Teachers make longterm goals for their students, and administrators evaluate a teachers progress toward the long-term goals. Teachers articulate students improvement needs and administrators use various indicators to evaluate teachers. CMS uses the new NC evaluation tool, but also adds the following indicators to evaluate teacher effectiveness: value-added data, classroom observations, and student surveys. CMS also plans to measure a teacher s contribution to the school and plans to evaluate teachers ability to design and create lesson plans. The district plans to conduct an in-house study that identifies the top performing teachers based on valueadded data to find what top performing teachers are doing differently. Through observations, student surveys, and instructional practice evaluation in the new teacher evaluation system CMS may find what topperforming teachers do differently. Singapore s Teacher Competency Model Singapore s approach to education is an example for the United States because it consistently ranks among the top countries in the world on international rankings of student achievement in math, science, and literacy [22]. Although there are several differences between Singapore s and the United States education systems, there are underlying principles to Singapore s approach that may be applied to improving student achievement in the United States. In the early 2000s, Singapore designed and implemented the Enhanced Performance Management System, which focuses on competencies that lead to exceptional teacher performance [23]. All school teachers are screened based on their academic caliber and demonstration of underlying competencies necessary for success in the classroom. The model includes one core competency, Nurturing the Whole Child, and four other clusters: cultivating knowledge, working with others, winning hearts and minds, and knowing self and others. The model strongly correlates a job holder's performance on the competency scale to successful attainment of work-related goals. Singapore first focuses on competencies, rather than observable characteristics. A similar example in the United States is Teach for America s recruitment model. Teach for America (TFA) focuses to recruit recent graduates with the following attributes: high-achievement, responsible, critical thinker, organized, motivating, respectful, and shares the goals of the organization [24]. The focus on competencies shows that academic credentials are not enough to guarantee an effective teacher. Effective teaching also requires soft attributes, which enhances the measurable teacher characteristics.

6 Further Information There are several resources may be helpful to find more information about teacher effectiveness research and teacher evaluation studies: National Comprehensive Center for Teacher Quality: National Council on Teacher Quality: National Center for Teacher Effectiveness: MET Project: Conclusion Education policy research has evolved to help educators and administrators improve student achievement. This report helps identify common trends among teacher inputs and processes that impact student learning. Although current research is not always consistent in its findings, a significant factor on student achievement is a teacher s academic caliber. Effective teacher qualities and teaching practices are highly variable because the definition of effectiveness depends on the metrics used to measure a teacher s impact. Education administrators are identifying teacher effectiveness patterns at the local level through new teacher evaluation systems. Ultimately, administrators will be able to recruit and train teachers who are most likely to produce high student achievement, and will be able to create professional development programs that promote effective teacher practices. References 1. Rivkin, S., Hanushek, E., & Kain, J. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), ; Sanders, W., & Horn, S. (1998). Research findings from the Tennessee Value-Added Assessment System (TVAAS) database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education, 12(3), Betts, J. R., Zau, A., & Rice, L. (2003). Determinants of student achievement: New evidence from San Diego. San Francisco: Public Policy Institute of California.; Carr, M. (2006). The Determinants of Student Achievement in Ohio's Public Schools. Columbus: The Buckeye Institute for Public Policy Solutions.; Clotfelter, C., Ladd, H., & Vigdor, J. (2006). Teacher-student matching and the assessment of teacher effectiveness. Journal of Human Resources, 41(4), 778.; Clotfelter, C., Ladd, H., Vigdor, J. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), 655.; Goldhaber, D. (2007). Everyone's doing it, but what does teacher testing tell us about teacher effectiveness? Journal of Human Resources, 42(4), 765.; Goldhaber, D., & Brewer, D. (1997). Evaluating the effect of teacher degree level on educational performance. Developments in School Finance, 199.; Goldhaber, D. B., D. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis, 22(2), 129.; Harris, D. N., & Sass, T. R. (2007). Teacher training, Teacher quality and Student achievement. Washington, DC: Urban Institute: National Center for Analysis of Longitudinal Data in Education Research.; Laczko-Kerr, I., & Berliner, D. C. (2002). The Effectiveness of "Teach for America" and Other Under-certified Teachers on Student Academic Achievement: A Case of Harmful Public Policy. Education Policy Analysis Archives, 10(37) 3. Boyd, D., Lankford, H., Loeb, S., Rockoff, J., & Wyckoff, J. (2008). The narrowing gap in New York City teacher qualifications and its implications for student achievement in high-poverty schools. Journal of Policy Analysis and Management, 27(4), ; Clotfelter, C., Ladd, H., Vigdor, J. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), Clotfelter, ; Clotfelter, 2010.; Rivkin, Cohen, D. H., H. (1998). Instructional policy and classroom performance: The mathematics reform in California. Philadelphia: Consortium for Policy Research in Education.; Frome, P., Lasater, B., Cooney, S., & Board, S.R.E. (2005). Well-qualified Teachers and High Quality Teaching: Are They the Same? : Southern Regional Education Board.; Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis: National Comprehensive Center for Teacher Quality.; Kane, T., Taylor, E., Tyler, J., & Wooten, A. (2010). Identifying Effective Classroom Practices Using Student Achievement Data NBER Working Paper. Cambridge: National

7 Bureau of Economic Research.; Kannapel, P., Clements, S., Taylor, D., & Hibpshman, T. (2005). Inside the black box of high-performing high-poverty schools. Report, Prichard Committee for Academic Excellence.; McCaffrey, D., Hamilton, L., Stecher, B., Klein, S., Bugliari, D. & Robyn, A. (2001). Interactions among instructional practices, curriculum, and student achievement: The case of standards-based high school mathematics. Journal for Research in Mathematics Education, 32(5), ; Newmann, F., Bryk, A., & Nagaoka, J. (2001). Authentic Intellectual Work and Standardized Tests: Conflict or Coexistence? Chicago: Consortium on Chicago School Research.; Rivkin, 2005.; Wenglinsky, H. (2000). How Teaching Matters: Bringing the Classroom Back into Discussions of Teacher Quality. Princeton: Educational Testing Service.; Wenglinsky, H. (2002). The Link between Teacher Classroom Practices and Student Academic Performance. Education Policy Analysis Archives, Betts, 2003; Frome, 2005; D. Goldhaber, & Brewer, D., 1997; Harris, 2007; Rowan, B., Chiang, F., & Miller, R. (1997). Using research on employees' performance to study the effects of teachers on students' achievement. Sociology of Education, 70(4), Ibid. 8. Wenglinsky, Rowan, 1997; Frome, 2005.;Kannapel, Betts, 2003; Clotfelter 2010; Darling-Hammond 2000; Goldhaber & Brewer 1997; Goldhaber 2000; Kane, T. J., Rockoff, Jonah, E. & Douglas O. (2008). What Does Certification Tell Us about Teacher Effectiveness? Evidence from New York City. Economics of Education Review 27(6).; Laczko-Kerr, I., & Berliner, D. C. (2002). The Effectiveness of "Teach for America" and Other Under-certified Teachers on Student Academic Achievement: A Case of Harmful Public Policy. Education Policy Analysis Archives, 10(37). 11. Kane, Goldhaber & Brewer, Darling-Hammond, Berry, & Thoreson, Kane, Frome, 2005; Harris, 2008; Kannapel, 2005; Wenglinsky, 2000 & Baker, E. L. et. al. (2010). Problems with the Use of Student Test Scores to Evaluate Teachers. Washington, DC: Economic Poilcy Institute. 17. Goe, Boyd, 2005; Clotfelter, 2006; Clotfelter Cohen, 1998; Kannapel, 2005; McCaffrey, 2001; Rowan, 1997; Wenglinsky, MET. (2011). Measures of Effective Teaching. Retrieved April 1, 2011 from Schools, C. M. (2011). Teacher Incentive Fund - Leadership for Educators' Advanced Performance Retrieved April 2, 2011, from Paine, S. L., & Schleicher, Andreas. (2011). What the U.S. Can Learn from the World's Most Successful Education Reform Efforts Policy Paper: Lesson From PISA: McGraw-Hill Research Foundation. 23. Steiner, L. (2010). Using Competency-Based Evaluation to Drive Teacher Excellence. In P. Impact (Ed.), Building an Opportunity Culture for America's Teachers. Chapel Hill: Public Impact. 24. Walsh, K., & Tracy, C. (2004). Increasing the Odds. National Council on Teacher Quality. Retrieved from