The Impact of Early Colleges on Postsecondary Performance

Transcription

1 The Impact of Early Colleges on Postsecondary Performance Julie Edmunds, SERVE Center at University of North Carolina at Greensboro Fatih Unlu, Abt Associates Jane Furey, Abt Associates Elizabeth Glennie, RTI International Background: North Carolina s Early College model is the subject of an IES-funded eleven-year longitudinal experimental study that utilizes a lottery-based design. Previous analyses have found positive impacts on enrollment in postsecondary education and on early attainment of postsecondary credentials (Edmunds et al., 2012; Edmunds et al., in press; Edmunds, Willse, Arshavsky, & Dallas, 2013). This paper supplements this research by examining the impact of the early colleges on students performance in postsecondary education. Purpose: The specific research questions driving this study include: 1. What is the impact of the early college on the number of college credits earned by students? 2. What is the impact of the early college on students postsecondary GPA? 3. What are the impacts for students who are taking college courses while in high school and for students taking college courses after high school? Population: The sample analyzed for this paper includes a total of 2941 students who applied to 19 early colleges in North Carolina from through Students in the study were randomly assigned through a lottery process to either attend the early college (treatment group) or to attend business as usual, usually the comprehensive high school in the district (control group). The analytic sample includes 1698 treatment and 1243 control students. TABLE 1 HERE Intervention: Early colleges, as studied in this project, are small high schools of choice located on college campuses. Targeted at students who are underrepresented in college and serving students in grades 9-12 or 13, the schools aim to provide students with a high school diploma and an associate s degree or two years of transferable college credit. Students have early exposure to college courses, frequently starting in the 9 th grade. Students in the last two years of their schooling are often taking almost all college courses. As implemented in North Carolina, early colleges are also expected to implement a set of Design Principles as shown in Figure 1. FIGURE 1 HERE Research Design: This paper presents results from a longitudinal experimental study in which students who apply to attend the early college are selected through a lottery process. Data Collection and Analysis: This paper utilizes transcript data collected from the University of North Carolina (UNC) system and from the North Carolina Community College System (NCCCS) as well as school enrollment data from the National Student Clearinghouse (NSC). Data from these three sources are linked to high school data collected by the North Carolina

2 Department of Public Instruction and housed at the North Carolina Education Research Data Center at Duke University. Because students take college courses in high school and after high school, we look at our two outcomes relative to college courses taken starting in 9 th grade and ending six years later, which amounts to 4 years in high school and 2 years post high school for students who finish high school in four years. The two specific measures of postsecondary performance include: College credits attained. Credits were determined by the number of credits given for a college course. Students who take Advanced Placement exams while in high school also receive college credits according to the score and the subject of the course. Grade Point Average for all college courses. Although these primary outcomes are cumulative in nature and include experience in high school, it is quite likely that students performance in college courses may differ during high school, where they have substantial supports, and in their time after high school, when they no longer have those supports. As a result, we also break out credits earned and GPA by those two time periods. Analytic Approach: A challenge for these analyses is that the two outcome measures are only available for students in public institutions of higher education (IHEs) in NC and not available for students who are enrolled in a private institution or in an IHE in another state, although we do have enrollment information for these students from the NSC. We address this challenge by relying on the What Works Clearinghouse s (WWC) attrition standards for experimental studies. Initially, we treat the outcomes of students indicated in the NSC as enrolled in a private or outof-state institution as missing so they are essentially counted as attriters. We have calculated that the corresponding attrition rates are 22.7% for the treatment group and 22.2% for the control group 1, which leads to an overall attrition rate of 22.5% and a differential attrition rate of 0.5%. These rates are under the thresholds set by WWC, which implies that such attrition levels are not expected to lead extensive bias. Having met the WWC attrition standards, we then impute the missing outcome values using multiple stochastic imputations 2. The imputation procedure uses all available information for students including their baseline characteristics, high school outcomes, and any postsecondary enrollment during and after high school (which is available from NSC). We expect that the richness of this information will help obtain highly reliable imputed values. Each outcome measure (including the original and imputed values) will be used as the dependent variable in multivariate regression models that include lottery indicators, baseline covariates, and a treatment group indicator, which yield the estimated impact of the ECHS on that outcome. 1 Students who have never enrolled in a college through Grade14 (according to the NSC, UNC, and NCCCS data) are not included in the attrition rate because they will still be included in the analysis with their outcomes set to zero. 2 We plan to employ the Multiple Imputation (MI) routine of Stata, which implements concepts introduced by Rubin (2014) and Schafer (2010).

3 Statistical inference is based on cluster-robust standard errors calculated based on the ECHS or regular high school in which students attended the longest. Results: The analyses are underway and will be completed by late Fall 2016 in time for the spring conference.

4 References Edmunds, J. A., Bernstein, L., Unlu, F., Glennie, E., Willse, J., Smith, A., et al. (2012). Expanding the start of the college pipeline: Ninth grade findings from an experimental study of the impact of the Early College High School Model. Journal for Research on Educational Effectiveness, 5(2), Edmunds, J. A., Unlu, F., Glennie, E., Bernstein, L., Fesler, L., Furey, J., et al. (in press). Smoothing the transition to postsecondary education: the impact of the early college model. Journal of Research on Educational Effectiveness. Edmunds, J. A., Willse, J., Arshavsky, N., & Dallas, A. (2013). Mandated engagement: The impact of early college high schools. Teachers College Record, 115(7), Rubin, D. B. (2004). Multiple imputation for nonresponse in surveys (Vol. 81). John Wiley & Sons. Schafer, J. L. (2010). Analysis of incomplete multivariate data. CRC press.

5 Figure 1: Early College Logic Model ECHS Design Principles College Ready Articulated program of study, grades 9-12 or 13, leading to Associate s degree or 2 yrs college credit, college readiness activities Powerful Teaching and Learning High-quality, rigorous, and relevant instruction, student collaboration and discussion, formative and multiple assessments Personalization Academic and affective supports, Supportive relationships Leadership Common vision, collective responsibility and decisionmaking Professionalism Ongoing professional development, collaboration among staff Intermediate Outcomes Increased student attendance Increased frequency of higher level courses Improved attitudes toward self and school Improved behavior Increased aspirations toward college Improved student achievement Long Term Outcomes Increased high school graduation rates Increased enrollment in college Improved performance in college courses Increased graduation from college Purposeful Design Autonomous governance, location on college campus, small size, flexible use of time, integration with college

6 Table 1. Sample Characteristics, by Treatment Status Whole Treatment Control Sample Group Group T-C Difference Effect (N=2941) (N=1698) (N=1243) Sizes Mean Mean Mean Difference P-Value Race & Ethnicity American Indian 0.9% 0.9% 1.0% -0.2% Asian 1.1% 1.1% 1.1% 0.0% Black 27.6% 28.4% 26.6% 1.8% Hispanic 7.8% 8.1% 7.5% 0.6% Multi racial 3.2% 2.9% 3.6% -0.7% White 59.4% 58.7% 60.2% -1.5% Gender Male 40.9% 40.8% 41.1% -0.3% Age Socioeconomic Background First Generation College 40.1% 39.1% 41.4% -2.3% Free/Reduced Price Lunch Eligibility 48.9% 49.1% 48.6% 0.6% Exceptionality Disabled/Impaired 2.2% 1.7% 2.9% -1.1% Gifted 8.4% 7.8% 9.2% -1.4% Retained 3.5% 2.9% 4.4% -1.5% 0.04* th Grade Achievement Math - Z score Reading - Z score Notes: a The proportions are weighted by students probability of being selected into the ECHS. *Statistically significant at p<.05.