UNC Teacher Quality Research Campus Video Conference Session

Transcription

1 Teacher Preparation in North Carolina: Impacts on Student Test Scores UNC Teacher Quality Research Campus Video Conference Session Spring 2009 Gary T. Henry, UNC at Chapel Hill Charles L. Thompson, ECU

2 Why Impacts? Ultimately, the contribution of UNC prepared teachers to public education should be judged by the increases in what students know and are able to do after being taught by these teachers In , 50.9 percent of North Carolina s 3 rd 8 th graders were proficient in both reading and math Only 33.3 percent of these students who live in poverty were proficient in both subjects In , high school students in North Carolina passed 68.4 percent of their End of Course exams in 10 core subjects The graduation rate for North Carolina was 70.3 percent in and failure to make sufficient progress for on-time promotion to the next grade is a significant predictor of dropping out of high school

3 Assessment of Impacts of Teacher Preparation Findings presented today Average impacts on high school end of course tests in 8 subjects (483,269 test scores in 25,981 classes) in & and middle school reading (367,950 test scores in 22,907 classes) and math (389,246 in 23,091 classes) in , , & Teachers with 10 or fewer years experience Models estimate gains in student scores Models account for numerous variables that are beyond the control of the teacher prep programs (categories for control variables are on next slide) Mixed models (HLM) accounting for nesting of students within classes within schools Models compare gains for teachers prepared by UNC teacher prep programs with all other sources of teachers Fine grained analysis of two functions of teacher prep programs selection of students into the programs and training provided by the programs (often in conjunction with departments in arts and sciences)

4 Assessment of Impacts of Teacher Preparation Control Variables (factors not under the control of UNC teacher prep programs) Student 1. Prior test scores 2. Classmates prior test scores (peer effects) 3. Days absent 4. Gender 5. Race/ethnicity 6. Poverty 7. Parental education 8. Gifted 9. Disability 10. Currently limited English proficient 11. Previously limited English proficient 12.Overage for grade (held back or retained at least once) 13. Underage for grade (promoted two grades) 14. Grade level

5 Assessment of Impacts of Teacher Preparation Control Variables (factors not under the control of UNC teacher prep programs) (cont.) Classroom 1. Number of students 2. Advanced curriculum 3. Remedial curriculum 4. Heterogeneity of prior achievement within classroom 5. Teacher characteristics (added to some models) School 1. School size (ADM) 2. Suspension rate 3. Violent acts per 1,000 students 4. Total per pupil expenditures 5. District supplements 6. Racial/ethnic composition 7. Concentration of poverty

6 Interpreting the Impacts of Teacher Preparation on Student Test Scores All impact estimates are presented in terms of effect sizes or standard deviation units and Middle School Test Scores are measured on different scales therefore presenting the impacts in standard deviation units is a way to put them on a common scale In effect size terms a coefficient of 0.20 indicates a substantial effect indicating a noteworthy impact on student achievement Impacts: An effect size of.20 equals 1.78 points on the high school EOC courses An effect size of.05 equals 0.45 points on the high school EOC courses The difference between the mean of top performing ¼ high schools and bottom performing ¼ high schools is about 5 points Middle School Impacts: An effect size of.05 equals 0.70 points on the middle school reading tests On average middle school students gain approximately 2.5 points per year in reading

7 Impacts of UNC Traditionally Prepared, MAT, and Teaching Fellows (bold* = significant) End of Course Exams Middle School Math Middle School Reading UNC Traditionally Prepared Teachers 0.017* UNC MAT Teachers UNC Teaching Fellows 0.028* 0.031* All coefficients are effect sizes that have been adjusted for control variables previously listed

8 Impacts of Other Important Teacher and Student Variables (bold* = significant) End of Course Exams Middle School Math Middle School Reading Out of Field Teaching 0.024* 0.067* 0.050* First Year Teacher 0.106* 0.063* to 2 Years Experience 0.032* to 4 Years Experience Advanced Degree * 0.030* NBC 0.033* Praxis II Performance 0.027* Students per Classroom 0.001* Advanced Curriculum 0.141* 0.037* 0.054* Remedial Curriculum * Classroom Ability Dispersion * Classroom Peer Prior Achievement 0.142* 0.171* 0.157*

9 Institutional Program Impacts: Number of Programs with Significant Positive and Negative Impacts by Test Area Positive Impact Negative Impact Average Impact High School Overall High School English High School Math High School Science Middle School Math Middle School Reading

10 Unpacking Program Impacts: Content Overall English Math Science Institutional Program Impacts (example) 0.047* * 0.055* Program s overall positive impact on high school test scores due to positive impacts in science and math English could be focus for increasing program s positive impact on NC high school achievement

11 Unpacking Program Impacts: Content Overall English Math Science Institutional Program Impacts (example) 0.073* * Program s overall negative impact is concentrated in math When test scores available: U.S History and Civics/Economics will be added Science broken down into Chemistry, Biology, Physics and Physical Science

12 Unpacking Program Impacts: Preparation and Selection Middle School Math Total Impact Middle School Math: Controlling for Experience, Assignment, and Post Graduate Training Middle School Math: Controlling for Ability Prior to College, Experience, Assignment, and Post Graduate Training Institutional Program Impacts (example) 0.073* * Institution s program has positive effect on middle school students math score gains Impacts due in large part to selecting more able students into program

13 Unpacking Program Impacts: Preparation and Selection Middle School Math Total Impact Middle School Math: Controlling for Experience, Assignment, and Post Graduate Training Middle School Math: Controlling for Ability Prior to College, Experience, Assignment, and Post Graduate Training Institutional Program Impacts (example) * Program has neither positive nor negative impact on middle school students math score gains Negative impacts likely due to issues with preparation received in program

14 Program Comparison Models, Standardized EOC Scores Compare Traditionally Trained UNC Teachers by Program to All Other Teachers** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Controls1** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Controls2** Teacher/Classroom Coefficient Coefficient Coefficient UNC Teacher Prep Programs FSU WCU ECU ASU UNC W NCCU UNC C NC A&T * 0.073* WSSU 0.073* 0.075* NCSU 0.047* 0.034* 0.044* UNC A ECSU UNC CH UNC P UNC G * 0.031

15 Program Comparison Models, Standardized EOC Scores by Subject Area Compare Traditionally Trained UNC Teachers by Program to All Other Teachers English Only** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Math Only** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Science Only** Teacher/Classroom Coefficient Coefficient Coefficient UNC Teacher Prep Programs FSU 0.075* * WCU 0.056* ECU ASU UNC W NCCU UNC C NC A&T WSSU * NCSU * 0.055* UNC A * ECSU * UNC CH * UNC P * UNC G

16 Middle School Program Comparison Models, Standardized EOG Math Scores Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Math** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Math with Controls1** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Math with Controls2** Teacher/Classroom Coefficient Coefficient Coefficient UNC Teacher Prep Programs FSU WCU ECU ASU UNC W NCCU 0.045* 0.065* UNC C NC A&T WSSU * NCSU * 0.060* UNC A ECSU UNC CH 0.073* 0.059* UNC P UNC G

17 Middle School Program Comparison Models, Standardized EOG Reading Scores Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Read** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Read with Controls1** Compare Traditionally Trained UNC Teachers by Program to All Other Teachers Read with Controls2** Teacher/Classroom Coefficient Coefficient Coefficient UNC Teacher Prep Programs FSU WCU ECU ASU UNC W NCCU UNC C * NC A&T WSSU NCSU UNC A ECSU UNC CH UNC P UNC G

18 What s Next for the Impact Models? Additional outcomes to be added Elementary grades math and reading Science (chemistry, biology, physics, and physical science) high school exams analyzed separately U.S. history and civics/economics high school exams 8th grade science exam Assess impact of experience versus leaving Are less effective teachers leaving in first three years? Can programs (internships or induction) reduce first year slump? Assess impacts of programs on groups of students Are some programs more effective with academically disadvantaged students? Are lower performing students more positively affected by higher ability peers than higher performing students? Assess impacts of school leadership programs on teachers, students, and schools