Institutional and Student Characteristics that Predict Graduation and Retention Rates

Transcription

1 Institutional and Student Characteristics that Predict Graduation and Retention Rates Dr. Braden J. Hosch Director of Institutional Research and Assessment Central Connecticut State University, New Britain, CT ************************************************************************ Through regression analyses of national data from the Consortium for the Study of Retention Data Exchange, this study explores how various institutional characteristics such as size, selectivity, housing, and diversity relate to retention and graduation rates of full-time, first-time students. Factors most usefully predictive of six-year graduation rates (R 2 = 0.764) were admission scores, the proportion of students living in campus housing, and first-semester grade point average in college. High school rank was also highly correlated with graduation rates but collinearity with admission scores excluded it from regression equations. Formulas presented in this study can be used to evaluate graduation and retention rates of an institution in the context of academic inputs and institutional characteristics. ************************************************************************ For a wide range of reasons, graduation rates of full-time, first-time students can be flawed measures of institutional effectiveness (Adelman, 2006), but their presence in ranking systems, accountability initiatives, and institutional strategic plans has become pervasive. Astin (1993, 2004) has offered various interpretive frameworks for understanding graduation rates to provide often missing context for these figures. Much of this work culminated in a study of data collected by means of the CIRP Freshman Survey that provides models by which to calculate expected institutional graduation rates based on a profile of an entering class of full-time, first-time students by gender, race/ethnicity, admission scores, and self-reported high school grades (Astin & Oseguera, 2005). These models are useful but can be difficult to apply because of the need to convert high school grades into a categorical scale. Additional research has examined how specific factors such as race/ethnicity and gender (Hood, 1992; Grayson, 1995), first-semester academic performance (Hosch, 2008; Redlinger & Associates, 2008), and institutional characteristics (Horn, 2006) affect graduation rates, and U.S. News and World Report (2007) uses a proprietary predicted graduation rate based on institutional characteristics and student characteristics in its rankings of national universities and liberal arts colleges. Most of these studies or approaches still omit seemingly important factors that may contribute to an institution s graduation rate, such as the proportion of students receiving Pell grants or the proportion living on campus. This study provides several models to identify factors related to institutional retention and graduation rates of full-time, first-time students by using multivariate regression techniques on data sets from the Consortium for the Study of Retention Data Exchange (CSRDE), supplemented by data from the IPEDS Peer Analysis System. These models can help institutions and policymakers evaluate retention and graduation rates as well as Paper presented at the North East Association for Institutional Research Annual Meeting, Providence, Rhode Island on November 4, 2008

2 to understand more fully which factors may most influence these rates. These models can provide a more robust context for an institution to evaluate its performance than peer group comparison and also may assist in formulating targeted institutional policy. Methodology Using data from the CSRDE about the cohort of full-time, first-time students who entered college in Fall 2001, this study investigated factors associated with one-year retention rates and six-year graduation rates of full-time, first-time students using step-wise multivariate regression. The CSRDE is a voluntary consortium of public and private fouryear post-secondary institutions that share detailed data about the progress rates and characteristics of full-time, first-time students. Data are shared among members, and this study does not identify any institution that participated in data sharing per the terms of consortium membership. Data from Appendix 3 and 3a of the 2008 CSRDE report (Hayes, 2008) were requested in electronic format; importantly these files contain institutionally-reported information about high school rank of students in each institutional full-time, first-time cohort as well as performance data about these students during their first semester in college. Reported mean SAT and ACT scores also appear in this data set; ACT scores were converted to SAT scores. Of the 466 institutions in the original data set, 69 were excluded because they were missing high school or college performance data or were Canadian institutions. As a caveat, while these institutions represent a broad cross-section of American colleges and universities, it is worth noting that the institutional decision to participate in the exchange of data about graduation and retention rates suggests some level of institutional commitment to improving these rates and to benchmarking against peers. By virtue of this institutional commitment, these institutions may not truly be representative of all American post-secondary institutions. Nevertheless, the study population is broad enough to generate at least some initial findings. This data set from CSRDE was matched to a range of variables from the IPEDS Peer Analysis System (PAS). Items entered into the analysis from PAS included headcount and full-time equivalent (FTE) enrollment, financial aid awarded to the 2001 GRS cohort; 4-, 5-, and 6-year graduation rates for cohorts entering from 1991 to 2001; SAT and ACT scores; institutional characteristics; and enrollment characteristics. In instances where data were missing from the CSRDE data set, data from PAS were used to complete or substitute for missing variables. For missing admission scores in the CSRDE data, an estimated median SAT or ACT score was calculated from the 25 th and 75 th percentile scores reported in IPEDS; ACT scores were converted to SAT scores. Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 2

3 Table 1. Profile of Institutions Included in Study Carnegie Classification in 2001 Private Public Total Baccalaureate-Associate's Baccalaureate-General Baccalaureate-Liberal Arts Others Master's I Master's II Doctoral/Research Intensive (DRI) Doctoral/Research Extensive (DRE) Grand Total Results Correlations between the independent variables and the dependent variables in this study are illustrative of factors that are related to graduation rates, but it is important to observe that all of these factors are substantially inter-correlated. As in all research of this nature, such factors are associated with each other but are not necessarily causal in nature. Bi-variate correlations unsurprisingly reveal the six-year graduation rate of an entering cohort is highly correlated the cohort s five-year graduation rate (R 2 = 0.933) as well as the one-year retention rate (R 2 = 0.790) for the same cohort. Since the proportion of students who graduate in six-years is a function of students who persist and those who graduate in four or five years following entry, this finding is practically trivial, except to reinforce that principle that graduation rates are trailing indicators (Davis, 2008) that depend to a very high degree on what happens in the first year of college enrollment. The six-year graduation rate of the cohort entering the prior year also was highly correlated with the six-year graduation rate (R 2 =0.929), which more or less confirms the principle that change over time tends to be incremental. Table 2. Correlations (Pearson's R) Between One-Year Retention Rates and Five-Year Graduation Rates with Six-Year Graduation Rates Factors Related to 6-Year Graduation Rate N Pearson s R R 2 Current Cohort s Five-Year Graduation Rate Previous Cohort s Six-Year Graduation Rate Current Cohort s One-Year Retention Rate (Fall to Fall) Among the variables that serve as leading indicators of graduation rates, this investigation confirmed Astin s findings that institutional mean admission scores (SAT or ACT) were in general most predictive of six-year graduation rates (R 2 = 0.635) and the percent of students in the top quarter of their high school class was close behind (R 2 =0.552). Such findings again should not be surprising, and they explain the preoccupation institutions have had with raising the academic inputs of entering students. Finally, these findings also in a way confirm the focus of accreditation until the latter portion of the 20 th century on academic inputs because in general, assuring that academic Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 3

4 inputs were high also appears to correlate with positive outcomes like high graduation rates. Figure 1. Relationship between Six-Year Graduation Rates and One-Year Retention Rates Six-Year Graduation Rate N= 395 institutions, DF=2, R 2 =0.790, p<0.001; 6-Year Graduation Rate = *One-Year Retention Rate. Addition of a dummy variable for public control increases the power of the model by3.6%, and indicates public institutions have a graduation rate about 6.5% below private institutions with the same one-year retention rate: 6-Year Graduation Rate = *One-Year Retention Rate 6.54*Public. Performance in the first semester of college was also found to be significantly related to six-year graduation rates, a factor that is especially useful to institutions because it reflects student performance during a period when an institution may more directly influence student behavior and engagement. The mean first semester grade point average (GPA) of an entering cohort (R 2 = 0.364) and the closely related factor of those earning below a 2.0 first-semester GPA (R 2 =0.324) explained about a third of the variation among institutional six-year graduation rates (see Figure 2). Institutional characteristics in terms of the population served by institutions, are captured by variables such as the percent of all undergraduates who attend part-time (R 2 =0.373), the percent of students in the cohort over age 24 (R 2 =0.259), and the proportion of students in the cohort from underrepresented race/ethnicity groups (R 2 =0.126). Institutional resources also appeared to be correlated with six-year graduation rates, although the relationships among these factors should be interpreted with special caution. The percent living in campus housing (R 2 =0.278) in their first year could account for about a quarter of variation among institutional graduation rates, although this effect was most pronounced for institutions reporting 90% or more of first-year students lived on campus. Further, it could be a mistake to assume that campus housing produces better outcomes rather than attracts students who may be more likely to graduate with the sixyear time frame. Thus, building additional campus housing could indeed have some effect on institutional graduation rates, but would do so by attracting a different type of student than those who attend commuter campuses (see Figure 3). Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 4

5 Figure 3. Relationship between Six-Year Graduation Rates and First-Semester Grade Point Average Six-Year Graduation Rate N= 350 institutions, DF=2, R2=0.324, p< Year Graduation Rate = *1stTermGPAUnder20. Regression excludes institutions reporting fewer than 3% of the full-time, first-time cohort earning a first semester GPA below 2.0, on the basis that these institutions are not representative of most post-secondary institutions. The institution s expenditures on instruction and academic support also appeared to correlate with six-year graduation rates (R 2 =0.180), although this relationship too should be interpreted with caution. First, institutional financial data can vary based on how the college or university is organized, and the institution s focus on graduate programs can inflate expenditures in these areas. Indeed, when controlling for institutional type by Carnegie Classification, this relationship between expenditures and graduation rates disappears, although fragmentation of the study population tends to reduce the power of statistical tests that can be applied. If this finding were to hold in a broader study, then it would lend credence to the position that in higher education, degree attainment appears negligibly affected by additional expenditures, although it does appear to be the case that students who are more likely to graduate are attracted to institutions with greater levels of resources. Finally, student financial aid was observed to be modestly related to six-year graduation rates. The percent of students in the entering cohort who received Federal grants higher percentages in this indicator are generally considered an indicator of higher levels of poverty in the student body was negatively correlated with graduation rates and accounted for about a fifth of the variation among institutions (R 2 =0.208). The proportion of students in the entering cohort receiving institutional grants exhibited only a small but positive relationship to institutional graduation rates (R 2 =0.073). Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 5

6 Table 3. Correlations (Pearson's R) with Six-Year Graduation Rates Factors Related to 6-Year Graduation Rate N Pearson s R R 2 SAT (Math + Verbal) or ACT Composite Score High School Rank (Percent in Top Quartile) High School Rank (Percent in Top Half) High School Rank (Percent in Top Decile) Percent of All Undergraduate Who Attend Part-Time Cohort s Mean First Semester GPA Cohort s One-Semester Retention Rate (Fall to Spring) Percent of Cohort with First Term GPA Below Percent of Cohort that Resided in Campus Housing (1 st Year) Percent of Cohort Over Age Financial Aid: Percent of Cohort Receiving Federal Grants Headcount of Cohort Expenditures on Instruction and Academic Support per FTE Total Enrollment (FTE) Percent of Cohort from Underrepresented Minority Groups Total Headcount Enrollment of Institution Percent of Undergraduates in Headcount Undergraduate Enrollment (Headcount) Financial Aid: Percent of Cohort Receiving Institutional Grants Public Institutional Control (dummy var.) Financial Aid: Percent of Cohort Receiving Loans Regression Models Linear regression models were created in SPSS using one-year retention rates and sixyear graduation rates as dependent variables; stepwise entry of independent variables was used to identify the most predictive variables. Because of the high level of correlation between SAT scores and high school class rank, regression models were also created using high school class rank but deliberately excluding SAT scores. On the whole, the models that included SAT scores were 3-5% more powerful than those that did not include them. For all models constructed, academic inputs (SAT scores or HS class rank), and the percent of entering students residing in campus housing were the most important components of the models For one-year retention rates, the three most predicative factors were combined Math and Verbal SAT scores, a dummy variable for baccalaureate institutional type, and the proportion of the cohort that resided in campus housing. This model accounted for 64% of variation in institutional one-year retention rates (see Table 4). One-year retention rates rose one point for every 15 point increase in the mean institutional SAT score. Baccalaureate institutions had one-year retention rates 4.3 percentage points lower than other institutions (although it should be noted that few highly selective baccalaureate institutions participate in the CSRDE). Finally, retention rates rose one point for about every 17 percentage points of the entering cohort that resided in campus housing, Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 6

7 although additional analysis of this variable suggests its relationship may be non-linear, with a significant jump in the association between high success rates and students living on campus observed only when 90% or more of the entering class lived on campus (see Figure 3). Figure 4. Relationship between Percent of Cohort Residing in Campus Housing and Success Rates Percent Retained/ Graduated (N=35) (N=22) (N=29) (N=43) (N=46) (N=73) (N=75) Percent of Cohort Residing in Campus Housing 1-Year Retention Rate 6-Year Graduation Rate In instances where an average institutional SAT or ACT score may not be available (or for institutions that prefer not use such scores), the resulting regression model had 4.5% less predictive power, accounting for 59.5% of variation in institutional one-year retention rates. For about every 3.5 percent of the entering cohort of students who graduated in the top quartile of their high school class, the institutional retention rate rose 1 percentage point. In this model, institutional type as a baccalaureate institution reduced one-year retention rate by 5.6 percentage points, and the retention rate rose one point for every nine percent of the entering cohort that lived in campus housing, with the same caveat as before that most benefit of campus housing appeared to accrue when the proportion of students living in housing exceeded 90% (see Table 5). Table 4. Multivariate Regression Models to Predict Success Rates (SAT Scores included) Institutional One-Year Retention Rate (Adj. R 2 =0.642) β S.E. t Sig. (Constant) Combined Math and Verbal SAT score *** Baccalaureate Institution (dummy var.) *** Percent of Cohort that Resided in Campus Housing *** Institutional Six-Year Graduation Rate (Adj. R 2 =0.764) β S.E. t Sig. (Constant) Combined Math and Verbal SAT score *** Percent of Cohort that Resided in Campus Housing *** Percent of Cohort with 1 st Term GPA Under *** *** Significant at p<0.001 Includes converted composite ACT scores. For One-Year Retention Rate: Addition of following factors could increase model power by4.1% (R 2 =0.681): percent graduating in the top quartile of HS class; percent of cohort receiving student loans, and the percent of the cohort receiving federal grants; Percent of Cohort with 1st Term GPA Under 2.0. For Six-Year Graduation Rate: Addition of following factors could increase model power by 4.5% (R 2 =0.811): Percent of all undergraduates who attend part-time, baccalaureate institution (dummy var.), percent graduating in the top quartile of HS class; percent of cohort receiving student loans, and the percent of the cohort receiving federal grants. Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 7

8 For six-year graduations rates, the three most predictive factors were mean institutional combined SAT scores, the proportion of the cohort that resided in campus housing, and the percent of the cohort that earned a first semester GPA below 2.0. This model accounted for about 76% of variation in graduation rates among the institutions in the study. In this model, the institutional six-year graduation rate rose one point for every additional ten points added to the mean institutional SAT score. It rose one point for every additional six percent of the entering cohort that resided in campus housing, and it decreased one percentage point for every 3.7 percent of the cohort that earned a firstsemester GPA below 2.0 (see Table 4). When precluding entry of SAT scores into the model, the four most predictive factors for the six-year institutional graduation rate were the percent of the cohort that graduated in the top quartile of their high school class, the percent that lived in campus housing, the percent that received federal grants, and the percent of the cohort earning a first-semester GPA below 2.0. This model accounted for 73% of variation among institutional six-year graduation rates, and is only slightly less powerful than the model that includes SAT scores. Perhaps most significantly, this model was the first to include the percent of the cohort receiving federal grants (a higher proportion typically is related to lower socioeconomic status in the student body). Additionally, the association between graduation rates and housing was magnified so that the six-year graduation rate increases one percentage point for every five percent of the cohort living in campus housing. The percent of the cohort earning a first-semester GPA below 2.0 also became magnified, so that for every three percent of the cohort earning below a 2.0 GPA in the first semester, the six-year graduation rate decreased by one percentage point (see Table 5). Table 5. Multivariate Regression Models to Predict Success Rates (Excluding SAT Scores) Institutional One-Year Retention Rate (Adj. R 2 =0.595) β S.E. t Sig. (Constant) Percent of Cohort that Graduated in top Quartile of HS Class *** Percent of Cohort that Resided in Campus Housing *** Baccalaureate Institution (dummy var.) *** Institutional Six-Year Graduation Rate (Adj. R 2 =0.732) β S.E. t Sig. (Constant) Percent of Cohort that Graduated in top Quartile of HS Class *** Percent of Cohort that Resided in Campus Housing *** Percent of Cohort that Received Federal Grants *** Percent of Cohort with 1 st Term GPA Under *** *** Significant at p<0.001 Includes converted composite ACT scores. For One-Year Retention Rate: Addition of following factors could increase model power by 6.7% (R 2 =0.662): percent of the cohort receiving federal grants; expenditures on instruction and academic support per FTE; percent of cohort with a 1st term GPA under 2.0, public (dummy var.); percent of undergraduates who attend part-time, and percent of the cohort receiving student loans. For Six-Year Graduation Rate: Addition of following factors could increase model power by 5.6% (R 2 =0.788): Percent of all undergraduates who attend part-time, ; percent of cohort receiving student loans; baccalaureate institution (dummy var.); expenditures on instruction and academic support per FTE. Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 8

9 Discussion and Conclusions Results of this study appear to confirm previous research (Horn, 2006; Astin & Oseguera, 2005) that show strong relationships between admission scores and high school performance with retention and graduation rates. Perhaps significantly, this analysis did not identify race/ethnicity or gender as factors that are associated either positively or negatively with student success rates after controlling for academic inputs, such as admission scores and high school rank. Policy implications of this research suggest that institutional graduation rates minimally should be evaluated in the context of an appropriate peer group of similar institutions. Given the challenges of identifying peer group, using predictive equations, such as those developed in this research may provide a better avenue for evaluating institutional performance. Additional communication to the public about the factors that contribute to institutional graduation rates is also necessary, since graduation rates appear significantly related to student academic inputs both the inputs measured before entry, such as high school performance and admission test scores, as well as the inputs applied through the first semester in terms of effort and engagement with the curriculum that would contribute toward first semester GPA. Indeed, this study also confirms previous research that student success matters (Hosch, 2008; Redlinger, et al., 2008; and Adelman, 2005). After colleges and universities admit an entering class, they have most influence on student behavior during orientation and the first semester at the institution. Focus on the first-year experience has become widespread in recent years, but the implication of student failure in the first semester dramatically impact graduation rates six years later. This study indicates that for every 3-4 percentage points of the first-year cohort that earns below 2.0, the six-year graduation rate drops by one percentage point. Institutions should disaggregate their own graduation rates by this important variable and share results widely among faculty, staff, students, and parents. In such conversations, it will be important to emphasize that faculty are not being asked to inflate grades but rather to examine how to promote higher levels of student learning and promote adoption of effective practices. This said, it is not unlikely that some institutional gaming of success rates is already occurring. In this study, private institutions reported entering students earned a mean first semester GPA of 2.91 while the mean GPA reported by public institutions was Private institutions reported just 12% of their entering students earned a first-semester GPA below 2.0 compared to 17% at public institutions. Conversely, the most recent National Post-Secondary Student Aid Study indicates that at for-profit institutions, where revenue depends almost solely on tuition dollars, 28% of students earned mostly grades of A. By comparison, 17% of students at private not-for-profit institutions and just 10% of students at public four-year institutions earned similarly high grades (NCES, 2007). To the extent that institutions realize benefit from awarding higher grades, the evaluation of institutional effectiveness by means of graduation and retention rates is open to gaming. Ongoing examination of how institutions adapt to an environment that places tremendous emphasis on completion rates is certainly warranted and researchers and policy makers alike should explore more robust metrics for evaluating institutional success. Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 9

10 References Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, D.C.: U.S. Department of Education. Astin, A. W. (2005). Making sense out of degree completion rates. Journal of College Student Retention: Research, Theory, & Practice, 7(1), Astin, A. W., and Oseguera, L. (2005). Degree attainment rates at American colleges and universities: Effects of race, gender, and institutional type. Los Angeles: Higher Education Research Institute. Davis, E. B. (2008). Colleges Need to Offer Clear Paths to Leadership, The Chronicle of Higher Education 54.45, A64, 18 July, Grayson, J. P. (1995). Race and first year retention on a Canadian campus. Ontario, Canada: York University, Institute for Social Research. (ERIC Document Reproduction Service No. ED ). Hayes, R. (2008) CSRDE retention report: the retention and graduation rates of entering baccalaureate degree-seeking freshmen cohorts in 438 colleges and universities. Norman, OK: Consortium for the Study of Retention Data Exchange. Horn, L. (2006). Placing college graduation rates in context: How 4-year college graduation rates vary with selectivity and the size of low-income enrollment. U.S. Department of Education. Washington, DC: National Center for Education Statistics. Hosch, B. (2008). The tension between student persistence and institutional retention: The relationship between first-semester GPA and student progression rates of first-time, full-time students. Association for Institutional Research Annual Forum, Seattle, WA. National Center for Educational Statistics. (2007). IPEDS Peer Analysis System. Retrieved October 12, 2007, from Redlinger, L. J., Etheredge, S., Zhao, X, and Stigdon, A. (2008). Identifying student and institutional factors that contribute to undergraduate student success: Results for the analyses of three FTIC cohorts. Association for Institutional Research Annual Forum, Seattle, WA. U.S. News and World Report. (2007). Undergraduate ranking criteria and weights in America s Best Colleges Retrieved from ut/weight_brief.php on June 11, Institutional and Student Characteristics that Predict Graduation and Retention Rates Page 10

11 Appendix. Institutional and Student Characteristics by Carnegie Classification and Six-Year Graduation Rate 6-Year Grad Combined SAT Pct in HS 1st Quartile Pct 1st TermGPA<2.0 Pct Res Housing Pct Underrep Minorities FT FT Pct Women Carnegie Classification Rate N Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Baccalaureate/ Other Below Total Masters Below and above Total Doctoral Below and above Total Grand Total

12 Appendix. Institutional and Student Characteristics by Carnegie Classification and Six-Year Graduation Rate 6-Year Grad Pct 24 Yrs + Pct UG Part-Time Pct Any Aid Pct Federal Grants Pct Inst Grants Pct Student Loans Carnegie Classification Rate N Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Baccalaureate/ Other Below Total Masters Below and above Total Doctoral Below and above Total Grand Total

13 Appendix. Institutional and Student Characteristics by Carnegie Classification and Six-Year Graduation Rate 6-Year Grad Retn Fall to Spring Retn Fall to Fall Total Headcount UG Headcount Pct UG Instr. & Ac Supp Exp. Per FTE ($) Carnegie Classification Rate N Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Baccalaureate/ Other Below Total Masters Below and above Total Doctoral Below and above Total Grand Total