Evaluating the Effect of Teacher Degree Level on Student Achievement in Taiwan.

Transcription

1 Evaluating the Effect of Teacher Degree Level on Student Achievement in Taiwan. I. Introduction The question of how teacher characteristics affect student learning has long been of concern to researchers, policymakers, educators, and parents. Much of the research published in past few decades has showed that high quality teachers raise student achievement. In fact, it appears that teachers are the most important school factor influencing student performances (Ferguson, 1998; Goldhaber, 2002; Goldhaber, Brewer & Anderson, 1999; Hanushek, Kain & Rivkin, 1999; Wright, Horn & Sanders, 1997). While researchers tend to agree that teacher quality is an important determining factor in influencing student outcomes, there is little consensus about the relationship between specific teacher credentials (e.g., experience and degree level) and characteristics (e.g., age, race, and ethnicity) and teacher effectiveness (Goldhaber & Anthony, 2007; Hanushek & Rivkin, 2006). Recent statements by the European Commission (2007) and the Organisation for Economic Co-operation and Development (2005) emphasized the importance of the quality of the teacher as a factor in improving student learning. Efforts to improve the quality of teachers in different nations have been widespread and varied, and one frequently advanced proposal is to require teachers to obtain a master s degree. For instance, in 2008, the United Kingdom introduced a new Masters qualification specially designed for teachers called the Masters in Teaching and Learning (MTL) and expects all teachers to obtain a master s degree over the course of their careers (Department for Children, Schools and Families, 2008). Lately, Taiwan s Ministry of Education established a goal to increase the proportion of teachers with master s degree by up to 4% each year (Ministry of Education, 2006). 1

2 Does a teacher master s degree matter? The research literature provides mixed evidence on the effects of a master s degree on teacher quality. A significant numbers of studies show that master s degrees typically do not predict teacher effectiveness (Goldhaber and Brewer, 1997; Hanushek, 1986, 1997), but some studies do not concur. Greenwald, Hedges, and Laine (1996), for example, found a relationship between teacher master s degree and student achievement. In sum, the impacts of teacher degree level on student learning are inconsistent across studies and remain puzzling. Through empirical investigation, this paper contributes to the discussion on the effects of master s degrees by looking at the question in Taiwan junior high schools, with the purpose of determining whether variations in teacher degree level explained teacher effects on pupils test scores. II. Literature Review Relationship between Teacher Quality and Student Achievement Social scientists have been interested in assessing the relationship between teacher attributes and student achievement ever since the release of the Coleman Report (Coleman et al., 1966). Coleman et al. (1966) found that of all the school-related factors that affect student learning, teacher quality is the most important, even though the school resources for learning reported little relation to achievement when students of comparable backgrounds were controlled. Since the Coleman Report, hundreds of papers have been published relating various teacher characteristics to student outcomes (Hanushek, 1997), but mostly focused on two qualifications: degree level and level of experience (Goldhaber, 2007). It is not surprising that these studies have showed a very mixed picture about the relationship between these teacher qualities and student achievement. 2

3 In a detailed synthesis of 147 studies, Hanushek (1986) found no systematic pattern in the effect of teacher education, teacher salary, and teachers years of experience on student achievement. Notwithstanding, Hanushek (1986) also reviewed several classroom-level studies that yielded a clear image about the differences in teacher effectiveness (e.g., Hanushek, 1971; Murnane & Phillips, 1981). A decade later, in a similar review of the literature, Greenwald et al. (1996) concluded that variables like teacher academic ability, teacher education, and teacher experience strongly related to student achievement. One explanation for these inconsistent results is that the quality of the teacher improves student performance, but the specific characteristics of an effective teacher are not easily identified, observed, or measured (Sørensen & Morgan, 2000). Effects of Teacher Degree Level on Student Test Scores Accordingly, there is a discrepancy in the literature on the effects of teacher degree level on student achievement. Greenwald et al. (1996) found that stronger relations between teacher quality and student achievement in studies that coded teacher degree level dichotomously as Master's degree or above as compared to no advanced degree. Contradictory, an amount of evidence suggested that graduate education of teachers bears no systematic relationship to student outcomes. For example, presenting a counterintuitive finding, Goldhaber and Brewer (1997) found that the teacher degree level variable is not a statistically significant factor in affecting 10 th -grade students test score (in four subject areas: mathematics, science, English and history). This particular study implies that teachers with master s degrees are no more (or less) effective than those without advanced degrees. Recently, Nye, Konstantopoulos and Hedges (2004) followed the research strand, employing dichotomous coding of teacher degree level on the data from a four-year 3

4 experiment in which teachers and students were randomly assigned to classes to predict teacher effectiveness on student achievement. The estimated effects of teacher degree level on achievement gains were generally small and negligible in grades 1 and 2, somewhat larger at grade 3 (in reading and mathematics, respectively), and statistically significant only for grade-3 mathematics achievement gains (Nye et al., 2004). In short, the empirical evidence regarding teacher effectiveness or degree level is weak and incompatible across studies. Background on Teacher s Degree Level in Taiwan Figure 1 shows the ratio of junior high school teachers obtaining master s degree in Taiwan from 1996 to It is clear that the proportion of teachers with graduate degrees in Taiwan increased steadily since the end of 1990s. As of 2008, almost 25 percent of the teachers in junior high school have at least a master s degree, compared to only five percent in The percentage of teachers who hold advanced degrees in junior high school level is expected to continue to grow in the near future as a result of the release of the Teacher Education Quality Improvement Program (TEQIP) from the government of Taiwan in 2006 (Ministry of Education, 2006). One of the major policy strategies stated in the TEQIP is to promote graduate education among the teachers in primary and secondary school. To achieve the aim of augmenting the amount of teachers with master degree in a short-term period, Taiwan s Ministry of Education was essentially calling for more teacher training institutions to offer continuing education programs of master degree level for local teachers. Furthermore, the authority of education in Taiwan was planning to design a new Postgraduate Certificate of Education (PGCE) program for master degree holder to attain the qualification of teaching (Ministry of Education, 2006: 13). 4

5 % Figure 1 Percentage of Junior High School Teachers with Master s Degree in Taiwan, Source: Ministry of Education (2009) III. Methodology Empirical Framework In this analysis, we focus specifically on the impact of teacher degree level on student achievement and investigate the effects of various levels of degree. We estimate a standard educational production function model to investigate the relationship between change in test scores from the 7 th to the 9 th grade, Y ij, and key question predictors and controls. This analytic approach allows us to investigate how the teacher degree level relates to student test score gains, holding constant family background and other schooling characteristics. Because there could be multiple students per teacher, we estimate random effects specifications of the models. The present educational production function model, which apportions students scores to different factors, takes the following form: Y ij = α + βx i + γs i + θt j + δl j + ε i, 5

6 where X i is a vector of individual and family background variables for student (including student s gender and race/ethnicity, 7 th -grade math ability, student educational expectation, math cramming in 9 th -grade, father s education levels, monthly family income, and parental educational expectation) and β is the return to these individual student and family background attributes. S i is a vector of school- and class-level variables (such as the type of school, school location, learning attitude of classmates, intense academic competition among classmates), and γ is the return to these school and class variables. T j is a vector of teacher variables other than teacher degree level (including teacher preparation background, major subject, years of teaching experience in junior high school, seminars attending rates for last semester and semesters teaching in the class of surveyed students), and θ is the return to these teacher characteristics. The teacher degree level variables are included in L j and the vector δ represents the impacts of teachers who hold different level of degrees. Data The primary source of data of this study is the data sets collected by the Taiwan Education Panel Survey (TEPS) in 2001 and 2003 (Chang, 2003, 2005), which include information on student and family background, schooling experiences, and student achievement on standardized tests. In 2001, TEPS, using a multistage stratified sampling method, surveyed a nationally representative cohort of 20,004 seventh graders attending 333 junior high schools. The sampled students were resurveyed in ninth grade. The follow-up sample size was 18,903. The unique feature of TEPS is that it provides detailed teacher and class level information that is tied directly to individual students by subject. For this study, we used the 2001 and 2003 student data, the 2001 parent data, and the 2001 and 2003 math teacher data from the restricted TEPS database. The sample size of student as merging from the 2001 and 6

7 2003 student data is 13,283. After deleting cases that have no information about math ability test scores (27 cases for 7 th grade and 158 cases for 9 th grade) and teacher degree level (5 cases), the sample size reduces to 13,093 (matched with 969 math teachers in total). The outcome measures in this study are students 9 th grade mathematics test score, and the primary variable of interest for this research is teacher degree level. For TEPS, Item Response Theory (IRT) has been applied to generate psychometrically sound estimates of cognitive ability based on a battery of aptitude test items including mathematics, among other subject domains (Yang, Tam & Huang, 2003). The using of IRT helps to place students on a common scale, even if they take tests of varying difficulty. Furthermore, the math test administered by the TEPS study is explicitly designed to be used to assess both level and growth in subject matter knowledge. Thus, the data permit the estimation of value-added or gain score production functions (Hanushek, 1986). The TEPS data set is ideal for this study because teachers can be linked with the students in their classes and because it includes detailed information on each teacher s degree level. Specifically, in the second follow-up of TEPS, 9 th grade teachers were asked, What is your highest education qualification? Teacher could respond: Diploma, Bachelor, Master or Ph.D. Since there are no 9 th grade teachers who hold a Ph.D. degree, we, in this study, divide teacher degree level into two groups: Master and Diploma/Bachelor. Our educational production function model and the dichotomous coding of teacher degree level as Master s degree or not follow closely those of Goldhaber and Brewer (1997), Nye et al. (2004) and the studies reviewed by Greenwald, Hedges, and Laine (1996) to allow a comparison of our findings with theirs. In addition to variables of the student math test scores in the 9 th grade and the 7

8 degree level of math teachers, all other 17 factors used as control variables in the multiple regression models are acquired from the TEPS 2001 and Most of these control variables are factors that were generally considered in previous empirical studies of teacher quality or studies of student achievement. These variables can be categorized into three groups: (1) student s individual characteristics and family background, (2) school and class aspects, and (3) math teacher characteristics. Such variables as student and parent educational expectation, cram schooling, class-level learning climate are barely available or used for the preceding studies of teacher quality, and our analytic models show that it is crucial to take into account all of these variables in estimating the effects of teacher degree level. Summary statistics of selected variables are presented in Table 1. Table 1 shows that in both of 7th-grade and 9th-grade math test scores, there is a substantial achievement gap between the students (2,363 in total) with math teachers holding master s degreea and the students (10,730 in total) whose math teachers do not have an advanced degree. Anyway, does teacher master s degree matter? This is the major hypothesis to be tested in this study. To compare the teachers with master s degree (or more) with other teachers, it is noticeable that the means of most of the variables are nearly equivalent. This implies that we are examining two groups of teacher with different degree level but mostly identical in other backgrounds. However, there are some notable differences in means of variables that exist between these two groups of math teacher. These are the factors that may influence the estimation of the effects of teacher degree level on student performance, which are the critical variables to be observed in the analytical models, including types of school, school location, teacher preparation background (whether or not graduated from traditional teacher preparation program of normal university), and years experience of teacher in junior high school. 8

9 Table 1 Means and Standard Errors of the Select Variables (N=13,093) Student and Family Background variables Student is male Student is Aborigine Student educational expectation High school diploma College degree Graduate degree Math cramming in 9 th grade Father s education level High school or below College Graduate School Monthly family income Under NT$20,000 NT$20,000-less than NT$50,000 NT$50,000-less than NT$100,000 NT$100,000 or above Parent educational expectation High school diploma College degree Graduate degree 7 th -grade math ability 9 th -grade math ability Number of students School and Class variables Attend private school City Classmates often discuss homework or study together Intense academic competition among classmates Teacher variables Major in subject Graduated from traditional teacher preparation program of normal university Teacher years junior high school experience 1 Seminars attending rates for last semester Semesters teaching in this class Teacher has Master s degree (or more) Means (Std. Err.).50(.50).03(.18).19(.39).49(.50).32(.47).46(.50).73(.45).24(.43).03(.18).10(.29).41(.49).35(.48).15(.36).12(.36).59(.49).29(.45).02(1.00).63(1.28) 2,363.18(.388).64(.48) 2.91(.77) 2.88(.85).94(.24).36(.48) 3.84(1.43) 2.84(1.79) 2.93(1.71) Teacher has Diploma or Bachelor degree Means (Std. Err.).51(.50).04(.20).20(.40).52(.50).28(.45).44(.50)..74(.44).23(.42).03(.18).11(.31).42(.49).34(.47).13(.44).12(.32).62(.49).27(.44) -.04(1.01).55(1.30) 10,730.10(.30).53(.50) 2.88(.78) 2.84(.84).91(.29).54(.50) 4.46(1.43) 2.42(1.73) 3.03(1.74) 1 Teacher years junior high school experience: 1 means less than 1 year, 2 means 1 year to less than 3 years, 3 means 3 years to less than 6 years, 4 means 6 years to less than 9 years, 5 means 9 years to less than 20 years, and 6 means 20 years and above. 9

10 IV. Empirical Results Table 2 reports the regression findings for models examining the relationship between math teacher degree level and student achievement in junior high school. Column 1 of Table 1 shows a model that only includes teacher degree level. This unconditional estimation model indicates that a significant achievement gap exists between 9 th grade students who math teachers have master s degrees and those teachers without advanced degree. However, the effects of teacher degree level on students test score moderated once the other variables included in the model (as shown in Column 2 to Column 4). We estimated the models sequentially, first adding only individual and family background characteristics, then including school, class, and teacher variables, respectively. Column 2 shows the results when student individual and family background variables are added to the model. Here, most of the estimated coefficients of these indicators are statistically significant and in the expected direction. These results generally remained the same in the following models shown in Column 3 and Column 4. It is notable that these variables alone account for the majority of the variation that we were able to explain with our full models and the effects of teacher advanced degree on student achievement disappeared once these variables were considered in the model. Consistent to most of the previous studies of student academic achievement, students from higher socioeconomic background performed better. For example, levels of father s education and family income are positively related to test scores. Beside that, students who have higher educational expectation have higher test scores, as do those who have higher educational expectations from their parents. Aboriginal students have lower predicted scores on average, compared to those are non-aboriginal students in Taiwan, a result typical of the literature. 10

11 Table 2 Multiple Regression of Teacher Degree Level on Student Achievement (N=13,093) Student and Family Background variables Student is male Student is Aborigine Student educational expectation College degree Graduate degree Math cramming in 9 th grade Father s education level College Graduate School Monthly family income NT$20,000-less than NT$50,000 NT$50,000-less than NT$100,000 NT$100,000 or above Parent educational expectation College degree Graduate degree 7 th grade math ability School and Class variables Attend private school City Classmates often discuss homework or study together Intense academic competition among classmates Teacher variables Teacher has Master s degree (or more) Major in subject Graduated from traditional teacher preparation program of normal university Teacher years junior high school experience Seminars attending rates for last semester Semesters Teaching in this class Model 1 Model 2 Model 3 Model 4 Coefficient (Std. Err.) Coefficient (Std. Err.).02(.02) -.65(.07)***.59(.04)***.95(.04)***.28(.03)***.36(.03)***.54(.07)***.18(.05)***.30(.05)***.40(.06)***.45(.04)***.91(.05)*** Coefficient (Std. Err.).00(.02) -.60(.07)***.57(.04)***.90(.04)***.28(.03)***.33(.03)***.54(.07)***.16(.05)**.26(.05)**.31(.06)***.43(.04)***.86(.05)***.33(.04)***.01(.03) -.07(.02)***.11(.02)***.11(.04)**.04(.03).05(.03)+.00(.04).06(.03)*.04(.01)*** -.01(.02).03(.01)*** Coefficient (Std. Err.) -.01(.02) -.19(.05)***.20(.03)***.31(.03)***.17(.02)***.14(.02)***.29(.05)***.09(.04)*.12(.04)**.14(.05)**.09(.03)**.27(.04)***.83(.01)***.13(.03)*** -.06(.02)** -.02(.01).05(.01)***.04(.02)+ -.06(.03)+.03(.02).03(.01)***.00(.01).02(.01)* Constant.687(.016)*** -.912(.055)*** (.104)*** -.156(.081)+ R 2 Adjusted p<.1 *p<.05 **p<.01 ***p<

12 The results in the previous model would lead one to the conclusion that teacher graduate degree has no impact on student achievement, which is in line with much of the previous literature. Nevertheless, at least in our sample, the use of school, class, and other teacher information is critical in interpreting the effects of the teacher degree level on students test scores. Column 3 shows the results when the school, class, and teacher characteristics are included in the model. Interestingly, while almost all coefficients of individual and family background variables decreased, the coefficient of teacher degree level increased slightly and it is approaching statistical significance. In Model 3, several results are worth highlighting. Few of the school, class, and teacher coefficients are statistically significant in the expected direction, but some results are contrary to common sense. Specifically, for school-level factors, students who attend private junior high school performed better than their peers from public school. The school location variable is not statistically significant in this model. For class-level aspects, we found the counterintuitive result that students whose classmates often discuss homework or study together have lower math ability. In this case, the variable of intensity of academic competition among classmates is positively related to test scores. For teacher characteristics, those who graduated from the traditional preparation program of a normal university, more years of teaching experience in junior high school, and more semesters teaching in the class of surveyed students are positively related to student achievement. The last model, Model 4, further includes the predictor of prior math ability as measured by the math ability IRT scores in the 7 th grade. Since the prior math ability is an outcome of both measured and omitted variables in the 7 th grade, the addition of this variable can help us to examine if the analysis without this variable is vigorously biased in any way. Column 4 shows that the prior math ability has a powerful 12

13 moderation impact on others variables in the estimation of effects on 9 th grade math test scores. Most of the variables remained statistically significant, but their coefficients reduced markedly. The effects of the indicators of classroom learning climate (classmates often discuss homework or study together) and teacher preparation background (whether graduated from traditional normal university or not) dissolved. Surprisingly, the results show that students whose school is located in a city performed worse than those who go to junior high school in a suburban or rural area. Similar to the results in Model 3, while the relationship between most of the variables and students test scores has weaken in Model 4 after adding the prior math ability of students, the variable of math teacher advanced degree remained close to the statistically significant. A closer examination of the results in these two models reveal that the variable of teacher degree level has an interaction effect with class, school, and other teacher aspects on students test scores. This suggests that the effects of teacher degree level on student performance may take place at the school or class level. But, there is at least one reason to be cautious about over-interpreting changes in coefficient estimates and statistical significance. In fact, the coefficient of the teacher master s degree (or more) changes very little in magnitude (only 0.01), even when its significance level is altered. V. Discussion and Conclusion This study provides a first look at the effects of teacher degree level on student achievement in Taiwan. The results of this study suggest that a teacher master s degree, as currently promoted by Taiwan s authority of education, does not necessarily present a signal of teacher quality, which implies that advanced degree is not markedly valuable to junior high schools, at least for the math teachers. If out of mathematics, in other subjects, teacher degree level also has no significant impact on 13

14 students performance, then we would not expect a master s degree holder teacher to be more affective. Thus, policy which requiring all teachers to obtain master s degree would be a waste of educational resources. There is some evidence that teachers with different levels of degree are not randomly distributed across students or schools. For instance, private junior high schools tend to hire more teachers who hold a master s degree. One possibility of this finding is that the private schools in Taiwan may be more or less reliant on teacher advanced degree as a signal of quality. Regrettably, while our analysis shows that students from private school have higher test scores, there is little evidence that master s degrees obtained by their teachers made the difference. In short, our findings closely parallel the results found by Goldhaber and his colleague (Goldhaber, 2007; Goldhaber & Brewer, 1997), and Nye et al. (2004) in the U.S. school sample, suggesting that teacher advanced degree is not associated with increased student achievement. Additionally, our results provide an auxiliary piece of evidence to support the thesis that promoting teacher graduate degrees to enhance the teacher effectiveness might only be efficient if there are other parallel educational improvement programs at the school-level. Unfortunately, our data do not permit us to investigate this issue, but it is something important that researchers may wish to pursue. Further, we find that teachers who have more years of teaching experience in junior high school and more semesters teaching in the class of surveyed students are positively related to student achievement. These findings indicate that the time, regarding to the teacher experience and the relationship between instructors and pupils, is a critical factor in the estimation of effects of teacher quality. Therefore, we might also imagine that difference in years of graduating from graduate school may have difference impacts in teacher performance or student learning. Again, our data 14

15 do not have the details for the testing of this hypothesis, which will be a subject for future research. References Chang, L. Y. (2003). Taiwan Education Panel Survey: Base Year (2001) Student Data and Parent Data [restricted release computer file]. Center for Survey Research, Academia Sinica [producer, distributor]. Chang, L. Y. (2005). Taiwan Education Panel Survey: Base Year (2001) Teacher Data [restricted release computer file]. Center for Survey Research, Academia Sinica [producer, distributor]. Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D. & York, R. L. (1966). Equality of Educational Opportunity. Washington, D.C.: U.S. Department of Health, Education & Welfare. Department for Children, Schools and Families (DCSF, UK) (2008, March). New Masters Qualification to Boost Teaching. Retrieved December 29, 2008, from EC (European Commission) (2007). The Commission proposes to improve the quality of teacher education in the European Union. Retrieved December 29, 2008, from Ferguson, R. (1998). Teachers Perceptions and Expectations and the Black-White Test Score Gap. In C. Jencks & M. Phillips (Eds.), The Black-White Test Score Gap (pp ). Washington, D.C.: Brookings Institution Press. Goldhaber, D. D. (2002). The Mystery of Good Teaching: Surveying the Evidence on Student Achievement and Teachers Characteristics. Education Next, 2(1),

16 Goldhaber, D. D. (2007). The Importance of Methodology in Teasing Out the Effects of School Resources on Student Achievement (CRPE Working Paper #2007-5). Washington, D.C.: Center on Reinventing Public Education. Goldhaber, D. D., & Anthony, E. (2007). Can teacher quality be effectively assessed? National board certification as a signal of effective teaching. Review of Economics and Statistics, 89(1), Goldhaber, D. D., & Brewer, D. J. (1997). Evaluating the Effect of Teacher Degree Level on Educational Performance. In J. W. Fowler (Ed.), Developments in School Finance 1996 (pp ). Washington, D.C.: National Center for Education Statistics. Goldhaber, D. D., Brewer, D. J., Anderson, D. (1999). A Three-Way Error Components Analysis of Educational Productivity. Education Economics, 7(3), Greenwald, R., Hedges, L. V., & Laine, R. D. (1996). The Effect of School Resources on Student Achievement. Review of Educational Research, 66(3), Hanushek, E. A. (1971). Teacher characteristics and gains in student achievement: Estimation using micro data. American Economic Review, 61(2), Hanushek, E. A. (1986). The Economics of Schooling: Production and Efficiency in Public Schools. Journal of Economic Literature, 24(3), Hanushek, E. A. (1997). Assessing the Effects of School Resources on Student Performance: An Update. Educational Evaluation and Policy Analysis, 19(2), Hanushek, E. A., Kain, J. F., Rivkin, S. G. (1999). Do Higher Salaries Buy Better Teachers? National Bureau of Economic Research Working Paper No Cambridge, Mass.: National Bureau of Economic Research. 16

17 Hanushek, E. A., & Rivkin, S. G. (2006). Teacher Quality. In E. A. Hanushek & F. Welch (Eds.), Handbook of the Economics of Education (pp ). Amsterdam; London: Elsevier. Ministry of Education (Taiwan R.O.C.) (2006). Improving Program of Quality of Teacher Education (Shizi Peiyu Suzi Tisheng Fangan). Taipei, Taiwan: Author. (in Chinese) Ministry of Education (Taiwan R.O.C.) (2009). Percentage of Teachers with Master s Degree in Junior High School, Taiwan, (in Chinese) Retrieved May10, 2009, from Murnane, R. J., & Phillips, B. R. (1981). What do effective teachers of inner-city children have in common? Social Science Research, 10, Nye, B., Konstantopoulos, S., & Hedges, L. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), OECD (Organisation for Economic Cooperation and Development) (2005). Attracting, Developing and Retaining Effective Teachers (Final report: Teachers Matter). Paris: OECD. Sørensen, A. B., & Morgan, S. L. (2000). School Effects: Theoretical and Methodological Issues. In M. T. Hallinan (Ed.), Handbook of the sociology of education (pp ). New York: Kluwer Academic/Plenum. Wright, P., Horn, S., & Sanders, W. (1997). Teachers and Classroom Heterogeneity: Their Effects on Educational Outcomes. Journal of Personnel Evaluation in Education, 11(1), Yang, M. L., Tam, T., & Huang, M. H. (2003). Psychometric Report for the Ability Tests of TEPS Center for Survey Research, Academia Sinica [producer, distributor]. 17