Transcription

1 The Value of State Data Systems Example: The Impact of Remediation Prof. Bridget Terry Long Associate Professor Education and Economics Harvard Graduate School of Education National Center for Postsecondary Research Kickoff Event October 11, 2006

2 The Importance of Data Lack of data limits research on educational outcomes and policies Traditional data sources (e.g. NCES) only give an incomplete picture Potentially important cohorts missing (mid-90s) Insufficient sample sizes to study particular state or to look within schools to assess teacher effects Sample attrition over time Incomplete information on peers, teachers and administrators 2

3 A New Resource: State Unit Record (SUR) Data Example: Florida K-20 Data Warehouse A single repository of data concerning students in the K-20 public system as well as educational facilities, curriculum and staff involved in instructional activities Contents Student: Demographics, Enrollment, Courses, Test Scores, Financial Aid, Awards, Employment Staff: Demographics, Certified Staff, Instructional Activities Educational Curriculum and Institutions 3

4 The Advantages of SUR Data The Advantages of SUR Data Example: Florida K-20 Data Warehouse Longitudinal can track students before, during, and after transitions and policy changes Complete census of a classroom, school, or college Can connect students to their teachers, administrators, faculty, and peers Ability to examine innovative state policies and population subgroups 4

5 Research Example: Does College Remediation Work? Nationally 35% to 40% of 1 st year students (55% at community colleges) (1) Underprepared recent high school graduates (2) Adult students seeking developmental courses Bulk of remediation is provided at non selective public colleges, the point of entry for 80% of 4yr students and 99% of 2yr students The remediation placement exam taken once arriving on campus has become the key academic gate keeper to postsecondary study 5

6 Meanwhile, there is little research... Most of the research is descriptive and/or based on a single institution Little about the impact on outcomes Studies do not allow for causal inferences Conjecture and criticism have filled the void created by the lack of basic information. Does College Remediation improve the outcomes of underprepared students? Joint with Eric Bettinger, Case Western Reserve University 6

7 The Data: Ohio Public Higher Education System Links college applications and transcripts with ACT & SAT scores and surveys (HS information) Nearly 66,000 first time freshman in Fall 1998 (fulltime, traditional age, 4yr degree intent) Time Period: Fall 1998 Spring 2003 (6 years) Can track across Ohio publics after stopping out Selective Universities Non-selective Universities Community Colleges Math or English Remediation 14.25% 34.65% 52.20% Observations 20,538 12,145 23,385 7

8 The Organization and Delivery of Remediation in Ohio Statewide standards to distinguish between remedial and college level courses, but schools differ in how they interpret the standards Most students are identified using combination of placement exams, standardized tests, and transcripts 2/3rds restrict enrollment in classes until remediation complete Implications for further study, college major, time to degree, & financial aid Remedial students differ systematically from other students in ways that might affect outcomes If under prepared, less likely to do well regardless of whether you are in remedial courses or not 8

9 Ohio Community Colleges Student Body ACT Distribution Maximum Likelihood Estimate of Remediation Cutoff COMMUNITY COLLEGES Difference in Log from Max ACT Math Score ACT Math Score Although the distribution of ACT scores at a set of schools may be similar, analysis shows that the likely cutoff for remediation differs. 9

10 Estimation Strategy Intuition John and Jim are observationally similar John lives near Cleveland & attends Cleveland College Jim lives near Cincinnati & attends Cincinnati College Cleveland & Cincinnati Colleges have similar student bodies but different remediation policies (i.e. cutoffs, courses, and selection processes) John is placed in remediation due to Cleveland College s stringent policy while similar student Jim is not due to Cincinnati College s lax policy ==> Compare the outcomes of John and Jim 10

11 Research Findings Students in remediation had better subsequent outcomes in comparison to students with similar backgrounds and preparation who were not required to take the courses Reduced the likelihood of dropping out and increased the likelihood of completing a degree English remediation also appears to reduce the likelihood of transferring to less selective/lower level Continuing research What is the best way to offer remediation? Characteristics of remediation program What policies should states follow? 11

12 Growing Use of Administrative Data Other Examples Florida K-20 Data Warehouse (CCRC work) New York CUNY system Washington - Community & technical college system Tennessee Higher Education Commission Connecticut Texas and California 12