C o l l e g e P a r t i c i p a t i o n, P e r s i s t e n c e, G r a d u a t i o n a n d L a b o r M a r k e t O u t c o m e s :

Transcription

1 C o l l e g e P a r t i c i p a t i o n, P e r s i s t e n c e, G r a d u a t i o n a n d L a b o r M a r k e t O u t c o m e s : An Input-Adjusted Framework for Assessing the Effectiveness of Tennessee s Higher Education Institutions David L Wright Chief Policy & Research Officer Tennessee Higher Education Commission William F. Fox, Ph.D. Professor of Economics Center for Business and Economic Research Matthew N. Murray, Ph.D. Professor of Economics Center for Business & Economic Research Celeste K. Carruthers, Ph.D. Assistant Professor of Economics Center for Business and Economic Research Grant Thrall, Ph.D. University of Florida Dept. of Geography September 2012

2 A b s t r a c t Institutions of higher education have come under increased scrutiny because of concerns over their effectiveness in ensuring student retention, timely graduation and the preparedness of graduates for the workforce. These concerns have motivated policymakers to develop outcomes-based funding schemes that are intended to promote institutional effectiveness. Funding policy must provide the right incentives by taking into account the characteristics of students since these attributes are likely to be important determinants of postsecondary outcomes. For example, while an institution may have a high graduation rate because it is effective in the delivery of instructional services, it is also possible that high graduation rates simply reflect the self-selection of better-quality students. The research presented here seeks to identify the effectiveness of Tennessee s public institutions of higher education while controlling for the characteristics of students upon entry to college. Using the 2002 cohort of first-time freshmen in Tennessee, the analysis considers a range of outcomes including progression, transfer, graduation and near-term labor market success (employment and earnings). Fixed effects are used in a multivariate regression setting to isolate institutional effectiveness. Specific findings indicate that females, higher college entrance examination scores and adult status were associated with more timely degree completion while low-income status delayed completion. Females and students with higher college entrance examination scores were associated with higher earnings in the years immediately following college, and a significant black-white earnings gap favored white students. Census and tapestry profiles did little to improve the explanatory power of the models. More generally, the results show that individual institutions varied in their effectiveness, justifying outcomes-based incentive schemes that reward and incentivize college quality. The relative effectiveness of campuses changed, however, when student characteristics were accounted for, leading us to conclude that outcomes-based schemes could be improved with adjustments for student inputs and that caution should be exercised in developing specific funding allocations without a deeper understanding of institutional features and programs that explain why some institutions appear more effective than others. These findings should be of value to policymakers and higher education officials charged with the task of evaluating the quality of higher education institutions. 2

3 I n t r o d u c t i o n Do higher education institutions serve their students well, and, if so, how can we tell? Traditional indicators of college quality include mean test scores of incoming students, graduation rates, student-faculty ratios and published rankings, but these do not reflect the value that institutions add to students and states investments in education or the relative quality of institutions serving similar student populations. For example, do the outcomes of elite universities reflect the performance of the university or the quality of incoming students? In reality, both matter. Students sort deliberately into the colleges of their choosing, and matriculation decisions are heavily influenced by personal preferences, aptitude and background. Later outcomes such as graduation and labor market performance are dependent on these student characteristics as well as the effectiveness of institutions. Comparing graduation rates and other college outcomes without considering students background and inherent aptitude will misstate institutional effectiveness and tend to favor colleges that draw from more advantaged pools of students. College rankings that rely on raw, average student outcomes reinforce this bias by spotlighting the best institutions for successive cohorts of high school students. College administrators have long reacted to the implicit incentives in college rankings, and feedback loops between rankings and students matriculation choices are not new. In recent years, college outcomes have taken on increased importance as they are factored into nascent performance funding schemes for higher education systems. Outcomes-based funding can be an efficient resolution to principal-agent problems where colleges and universities have different objectives from the taxpayers and higher education systems to which they answer. Attaching financial incentives to outcomes that matter to society at large (e.g., college completion) will theoretically motivate higher education institutions to achieve those objectives at low cost. Unintended consequences of performance pay, however, might be manipulation and illusory gains in student outcomes. By rewarding colleges for higher graduation rates, for instance, performance funding may incentivize campuses to adjust their admissions criteria, become more selective or admit more students who have historically completed college reliably. This fashion of manipulation would run counter to the objectives of outcomes-based funding, improve appearances rather than add value for students and have significant implications for the allocation of resources across institutions as well as student sorting. In the K-12 policy literature, researchers have demonstrated that growth-based accountability models circumvent some of the manipulation and targeting arising from threshold accountability models that reward schools for levels of student achievement (Ladd and Lauen, 2010). But measuring growth in college-level skills is difficult for a number of reasons, including the variety of paths a student may choose to take before, during and after college. An alternative solution is to reward institutions for achieving student outcomes over and above expectations, taking into account incoming students aptitude and backgrounds. 3

4 An input-adjusted performance pay scheme like this should be preceded by two general questions that this study seeks to address. First, do student inputs matter for predicting outcomes such as college progression, completion and labor market participation? If not, then manipulating those inputs would provide no advantage to institutions, and a simple outcomes-based incentive scheme without adjustments for student characteristics may suffice. Second, if student inputs matter, do institutions vary in the effectiveness with which they serve equivalent students? That is to say, is college quality completely a function of student characteristics, or do institutions matter as well? If institutional effectiveness is not uniform across colleges and universities, then input-adjusted funding formulas can feasibly reward policies, practices and campuses that advance postsecondary objectives without unintentionally rewarding the manipulation of admitted pools of students. Results indicate (1) that students pre-college characteristics are important predictors of their later success and (2) that, even when these characteristics are controlled for, institutions vary in the effectiveness with which they advance postsecondary persistence, graduation and labor market outcomes. Together, these findings validate the idea of outcomes-based funding per se but point to input-adjusted formulas as potential improvements over schemes that reward raw values of student outcomes without adjustments for inputs and expectations. In the spring of 2010, Tennessee enacted the Complete College Tennessee Act (CCTA), a comprehensive reform agenda aimed at transforming public higher education through changes in academic, fiscal and administrative policies at the state and institutional level. At the center of these reforms is a desire for more Tennesseans to be better educated and trained, while also acknowledging the state's diminished fiscal capacity to support higher education. At the heart of the CCTA is a new Public Agenda for Tennessee Higher Education, which establishes the direct link between the state s economic development and its educational system. A primary state policy lever for addressing the state s educational needs is a new public higher education funding formula, which incorporates student outcomes in lieu of enrollment. As shown in Table 1, the formula bases the entire institutional allocation of state appropriations on the basis of outcomes, including but not limited to degree production, research funding and graduation rates at four-year colleges and universities, and student remediation, job placements, student transfer and associate degrees at two-year community colleges. Each of these outcomes is uniquely weighted at each institution to reflect its mission, its Basic Carnegie Classification and the institutional priority assigned to each outcome. For instance, as seen in Table 2, at the University of Tennessee, Knoxville, a Research University (very high research activity), sixyear graduation rates and research/service are weighted more heavily than at the University of Tennessee, Martin, a Master s University (medium programs). Conversely, UT Martin affords twice as much weight to degree production. Individual institutions have no direct control over the components or weights of the formula. Additional details on Tennessee s funding formula can be found online at by following the Complete College Tennessee Act Summary link. 4

5 The purpose of this study is not to evaluate the full scope of Tennessee s newly implemented funding formula, though findings will inform future research and funding policies in Tennessee and elsewhere. Instead, the focus falls on identifying the effectiveness of different campuses in promoting key postsecondary outcomes, including credit hours accumulated, degree receipt, the likelihood of transferring from a community college to a four-year institution, and near-term earnings in the labor market. Evaluating these achievements is an important step forward in understanding institutional effectiveness. Nevertheless, we hesitate to conclusively rank Tennessee s institutions according to their effectiveness, since our models exclude institutional characteristics that likely explain why some campuses perform better than others (e.g., private funding, faculty, program strength and so forth), even when student inputs are accounted for. Inputadjusted funding systems will benefit from additional research identifying the underlying policies and advantages that explain variance in quality across campuses. In the end, both outcomes and the best practices of institutions are important candidates for inclusion in performance-based funding programs. Our empirical models identify the correlates of college and post-college success, including graduation and post-college earnings, for the cohort of fall 2002 first-time freshmen in Tennessee s public two-year and four-year higher education institutions. We compare the model fit and predictive power of several different sets of covariates, including combinations of administrative data from the Tennessee Higher Education Commission (THEC), 2000 Census fields mapped to students home addresses and consumer tapestry profiles mapped to home addresses. These profiles (called LifeModes ) have seen extensive use in private sector market research, but to date their value in public sector and policy research arenas has not been explored. Although Census fields and LifeModes provided interesting descriptive sketches of student populations, their inclusion as explanatory variables added very little to the overall model fit of the equations we estimated. Thus, our preferred models that we emphasize below control for THEC administrative fields and institutional fixed effects. Results from these models indicate that several variables are significant factors in predicting student success. In particular, females, students with higher ACT/SAT scores and adult students are associated with timely degree completion. Results for adult degree completion stand at odds with some of the existing research on nontraditional college students (e.g., Taniguchi and Kaufman, 2005). Indeed, we find that adults are much less likely to earn degrees on average, but somewhat more likely when we control for observable student characteristics and institution fixed effects. Black students have delayed progress toward completion in two-year schools, and low-income students are less likely to complete a bachelor s degree. The student characteristic that was perhaps most important in predicting degree completion was an indicator for whether the THEC was missing ACT/SAT data for a student (which was more common for community college enrollees or students who left college before their information was fully processed), underscoring the necessity of understanding and accounting for incomplete student data. With regards to post-college labor market outcomes, males received an earnings premium while black students received lower earnings. Higher 5

6 ACT/SAT scores were associated with modestly higher earnings. After earning bachelor s degrees, older students had much higher earnings than other four-year college completers. Most important for Tennessee s policy context, the apparent effectiveness with which campuses advance students through college and into the workforce changes after the characteristics of incoming students and in particular, their aptitude as measured by ACT/SAT scores are accounted for in the empirical analysis. Tennessee s funding formula includes institutional weights that are designed with the unique mission and Carnegie Classification of each campus in mind. To the extent that institutional missions are correlated with student inputs (e.g., if higher-achieving students tend to sort into more research-focused institutions), adjustments for weights will mitigate some of the bias that arises from failing to account for student backgrounds. That is to say, institutions will not be financially penalized for serving the same student population they targeted when the weights were created. But moving forward, static weights for progression outcomes that fail to adjust for student backgrounds may incentivize institutions to recruit and admit less challenging student populations, perhaps through more selective admissions procedures or smaller freshman classes. One potential solution is to estimate progression and graduation benchmarks for each institution and cohort, based on some of the important observable pre-existing student characteristics identified here (e.g., ACT scores and low-income status), and reward campuses that exceed expectations. Lastly, we show that postsecondary persistence produces mixed outcomes, suggesting that meaningful persistence and completion should be incentivized over raw enrollment and credit figures. As might be expected, we show that completers earn relatively higher wages than non-completers. However, conditional on degree receipt, additional semesters in college reduce earnings later on. This latter finding is consistent with recent research demonstrating that excessive years of schooling can be a negative signal to employers (Flores-Lagunes and Light, 2010). L i t e r a t u r e a n d C o n t r i b u t i o n This study is an important entry in the evolving literature on input-adjusted postsecondary student progression. i In contrast to elementary and secondary education research, relatively little attention has been paid to the simple correlates and causal factors in college and post-college success, especially for community college students. A limited research base in this area has consequences for higher education policy, practice and performance. Institutions are often evaluated and ranked according to easily measured outcomes, even though researchers have long acknowledged that raw postsecondary measures such as graduation rates measure selectivity more so than institutional effectiveness (Bailey and Xu, 2011). Research identifying the factors associated with postsecondary progression tends to rely on aggregate IPEDS institutional data, and findings indicate that higher average SAT scores, higher shares of female students, more traditional campus settings (i.e., with fewer commuting and part-time students) and higher 6

7 expenditures are associated with higher graduation rates in four-year colleges and universities. ii Research on input-adjusted progression in two-year colleges is even scarcer but indicates that smaller enrollments, lower shares of minority students and non-urban campuses are associated with higher shares of degree recipients and/or higher certificate completion rates. A primary limitation of using aggregate data is that campus profiles cannot capture the underlying heterogeneity of students and institutional resources that affect outcomes. We join Cuhna and Miller (2011) and Bailey et al. (2005) in using longitudinal, student-level data to predict graduation and other outcomes as functions of student backgrounds. Aggregate, campus-level data characterizing student progression are subject to omitted variable biases if campuses differentiate in ways that affect graduation measures and the types of students who choose to enroll. For instance, the research body in this area suggests that higher shares of nontraditional adult or commuting students are associated with lower graduation rates (e.g., Porter (2000)). But this does not necessarily mean that nontraditional students are more challenging to educate in a college setting. Some campuses might rationally offer more relaxed admissions criteria and more flexible enrollment options to stand apart from traditional, more selective institutions with more on-campus housing and younger enrollees. The simple correlation between nontraditional student shares and lower graduation rates, then, would be a reflection of differentiation rather than variance in college quality. In contrast to studies using aggregate data, we find that adult enrollees classified as first-time freshmen accumulate more credits, enroll for more semesters, are more likely to graduate and, conditional on degree receipt, earn much higher wages after college. Longitudinal data on students and outcomes allow us to circumvent biases from unobserved, institutional features to better assess input-adjusted college quality. This study also contributes to the broad research literature on pecuniary returns to schooling and, in particular, the literature on heterogeneous returns to college quality. Years of schooling and degree receipt are associated with meaningful and significant gains in earnings, even with controls for the tendency of higher-ability individuals to stay in school longer. iii The returns to college quality are less clear. Research in this vein typically constructs a proxy for college quality (e.g., mean SAT scores of incoming students, expenditures per student or a composite of multiple quality measures) to estimate the wage or earnings premiums associated with attending a better school. Several studies have found large returns to college quality using this approach. iv By contrast, Dale and Krueger (2002, 2011) argue that these studies failed to adequately control for unobserved student ability. Dale and Krueger proxy for inherent ability using the average SAT score of incoming students in all of the colleges each student applied to, not just the college they ultimately attended. Results indicate that the returns to college quality are statistically insignificant overall but large and positive for black and Hispanic students. We use statewide administrative data on students postsecondary progression to estimate (rather than proxy for) college quality. Our primary objective in this study is not to estimate the wage returns to college quality 7

8 but, rather, to assess whether college quality is merely a function of incoming student ability or whether institutions matter as well. We produce college fixed effect parameters that can be interpreted as an index of institutional effectiveness, conditional on student ability and backgrounds. Collectively, these fixed effects are statistically significant even with a full set of student background controls, which provides strong evidence that some components of college quality cannot be attributed to characteristics of incoming students. Future research in this vein might include (1) investigations into the features that explain variance in college quality and (2) estimates of the long-run wage returns to input-adjusted college quality. D a t a Our sample includes all first-time college freshmen entering Tennessee s two-year public community colleges and four-year colleges and universities in the fall of We focus on first-time freshmen for three principal reasons. First, they likely had similar objectives for college progression and completion, two outcomes valued by Tennessee s funding formula that are analyzed in great detail here. Second, first-time freshmen are sensibly grouped into cohorts, and admissions decisions will be more stable within a single cohort of students than across years and cohorts. Finally, students grouped into one cohort and adhering to the common definition of first-time freshmen would have confronted similar labor markets in a given year as opposed to students from different cohorts and different points in the age-earnings profile. But we acknowledge that limiting our scope to first-time freshmen omits up to two-thirds of Tennessee s postsecondary students from the analysis. We emphasize that conclusions do not necessarily apply to nontraditional and/or returning students who may or may not be seeking a degree. We note the institution where each student started college in the fall of 2002; all of their postsecondary and labor market outcomes are associated with that college. v We use a panel of student background, college enrollment, labor market participation and degree receipt information for this cohort spanning Student background and outcome data come from four major sources: administrative enrollment and award records collected by the THEC, students home neighborhood data from the 2000 U.S. Census, proprietary consumer tapestry profiles for Census block-groups and zip+4 codes (produced by Esri, vi a private firm) and, finally, quarterly in-state wage data from the state s unemployment insurance system. Descriptive Statistics Tables 3 6 contain a rich set of descriptive statistics for the 2002 cohort of 24,630 first-time freshmen at Tennessee s public two-year and four-year colleges. Table 3 summarizes the postsecondary outcomes of interest in this study. Four-year matriculants in the 2002 cohort (including subsequent dropouts and degree recipients) attempted credit hours, on average, which corresponds to about 8.4 semesters at 14 credit hours (the mean) per term. Two-year enrollees ultimately attempted 73.6 credit hours, on average, which includes any credits earned after transferring to a four-year institution. Among all four-year students in the 8

9 2002 cohort, 53.3 percent received a bachelor s degree within eight calendar years of the fall of 2002 (i.e., 200 percent of the normal time to degree). Across four-year institutions, this ranged from 38.6 percent to 66.5 percent. Only 29.2 percent of community college matriculants received an associate or bachelor s degree within 200 percent of normal time, ranging from 17.1 to 39.8 percent across two-year community colleges in the state. For 2009 wage outcomes, we limited the sample to students who had left college (with or without a degree) by January 2009, because these individuals were more likely to have been fully absorbed in the labor market. Throughout this subsample, 66.5 percent were observed with Tennessee earnings in the 2009 calendar year, falling to 63.8 percent in Table 4 details student characteristics found in administrative THEC records. Freshmen in four-year colleges were more likely to be male and tended to be younger than their two-year counterparts. ACT scores were higher among four-year students and much less likely to be missing. vii For students with missing ACT scores, we assigned the median score of the student s institution and then included a dummy variable to control for these unique observations in the empirical models. Mean ACT scores ranged widely across institutions, from 17.3 at one community college to 24.3 at a four-year public university. The highest mean ACT score among all two-year colleges in Tennessee (19.6) was actually higher than the lowest mean ACT score among fouryear colleges (18.7). Just 6.7 percent of first-time freshmen were considered adult enrollees. Here, we define adult students as those who were 25 or older at the time of enrollment. viii Among all first-time freshmen in the cohort, 38.2 percent were likely low-income, based on the median income of their home neighborhood. Both adult and low-income students were more prevalent in two-year schools. An important measure of student aptitude is missing from this analysis: We do not observe high school GPA for the 2002 cohort. GPA is available for later cohorts, however, and we would expect our models predictive power to increase with its inclusion. Table 5 summarizes pre-college home neighborhood characteristics as of the 2000 U.S. Census. Students home addresses were drawn from Free Application for Federal Student Aid (FAFSA) records. As indicated in Table 4, 26.5 percent of two-year and four-year students are missing home addresses, largely because they did not apply for federal aid. Students may have failed to apply for aid for a variety of reasons, chief of which include inadequate information about federal aid for college (which would tend to affect less affluent students ix ), a lack of financial need or an aversion to debt. For students without known home addresses, we impute each Census variable using average values from other students who attended the same high school. Students without home addresses were more likely to be male, white and higher-income. They had equivalent ACT scores, on average, as their peers with home addresses and FAFSA records. Models control for whether students were missing their home address, and results indicate that these individuals accumulated fewer semesters regardless of whether they started in two-year or four-year schools, and if they were fouryear enrollees, they were more likely to receive a bachelor s degree within 150 percent or 200 percent of normal time, and they tended to have higher in-state wages after college. Together, these observations imply 9

10 that students without home addresses and FAFSA records were more affluent, on average, and that imputing their home neighborhood characteristics with those of their high school peers likely understated their socioeconomic status. Nevertheless, principal results regarding the relatively limited explanatory power of models relying on Census controls are robust to the exclusion of students without FAFSA records and/or home addresses. In terms of racial and ethnic composition, Tennessee s public college students closely resembled the demographics of the state. But relative to the state, median incomes were higher in neighborhoods with college-going students (particularly so for four-year students), and a higher share of women participated in the labor market in the same places. Tennessee neighborhoods with two-year matriculants were 36 percent rural, much like the state as a whole, whereas four-year students tended to come from less rural areas. Table 6 lists the share of four-year and two-year entrants who were defined by each of Esri s LifeMode profiles. Households are segmented into one of 12 LifeModes by a proprietary methodology developed by Esri. Data inputs include Census fields and miscellaneous consumer and real estate data. From Esri s Reference Guide: To create segmentation at this level, Esri used the InfoBase-X data from Acxiom Corporation. Acxiom compiles its lists and data from an unprecedented number of data sources including public real estate information, purchased data from catalogs, auto dealerships, consumer surveys, publications, product registrations, and telephone directories. (Esri, Tapestry Segmentation Reference Guide) Student profiles were mapped from their home addresses, as fine as the zip+4 level. In parallel to our treatment of students without home addresses for Census data, we used average LifeMode shares within high schools to impute composite LifeModes for students without home addresses. Although we expect this to understate the socioeconomic status of students with imputed LifeModes, the overall fit and explanatory power of models using LifeModes are similar when these students are excluded. Among all college-going students in the 2002 cohort, the Factories and Farms LifeMode was the best-represented, accounting for 16.9 percent of four-year college enrollees and 30.2 percent of two-year enrollees. Other popular profiles were American Quilt, which generally describes more rural areas, Senior Styles, where median ages tend to be higher, and High Society and Upscale Avenues, the two most affluent LifeMode profiles. LifeModes describing more urban or ethnically diverse neighborhoods were least common. Echoing descriptive Census statistics in Table 5, we see that four-year students tended to come from more affluent and less rural areas than two-year students. 10

11 M e t h o d o l o g y Simple Higher Education Production Functions We estimate higher education production functions and models of post-college earnings by multivariate regression analysis (i.e., ordinary least squares x ) to determine the correlates of postsecondary outcomes of interest. First, consider a simple linear model of outcome Y for student i entering college c in the fall of 2002: xi Y ic= α 0 + α c + ε i Y ic postsecondary student outcomes summarized in Table 3 α 0 constant term α c college fixed effects for institutions c = 1,,C, where one institution is omitted ε ic error term In this naïve model, college fixed effects merely rank campuses based on their mean, unconditional values of Y i, without controlling for student inputs. Note that these raw measures of aggregate student progression are implicitly used in Tennessee s funding formula, albeit alongside weights that are unique to each institution. (1) The education production function described by Equation (1) would be appropriate if students randomly sorted into campuses across the state. Any significant, cross-institution difference in student progression, completion and labor market success would be evidence that some institutions were more effective than others. If, however, some unobserved component of the underlying education production function is correlated with students college choice and/or admissions profile, ability and later outcomes, estimates of institutional effectiveness will be biased. For instance, Equation (1) does not control for students inherent ability prior to entering college. If higher-ability students have an above-average likelihood of graduating on time regardless of which college they attend, and if they tend to gravitate to one institution for reasons other than college quality (e.g., proximity to home or amenities), their chosen campus s actual effectiveness will be overstated. Equation (1) contains no student background controls to mitigate this possibility, nor does it control for students pre-college, home neighborhood environment (i.e., median income, labor force participation and so forth). These neighborhood effects may be important in capturing pre-college peer effects and other unobservable student inputs. A richer set of controls for student background may add explanatory power to Equation (1) and protect against omitted variable bias. Accordingly, we expand Equation (1) by including a set of observable student characteristics, B i: Y ic = α 0 + B iα 1 + α c + ε ic (2) Three broad classes of student characteristics are employed as controls. Observable, individual pre-existing student characteristics from THEC administrative data. Variables include gender, race/ethnicity, ACT and SAT scores xii, and other variables summarized in Table 4. 11

12 Features of students narrowly defined home neighborhood, mapped to 2000 U.S. Census block groups. Variables include neighborhood-level gender, race, ethnicity, income, employment, mobility and other data summarized in Table 5. The Esri LifeMode profile associated with each student s home address. The American Quilt LifeMode category is omitted, so coefficients are interpreted relative to the average outcome of students from American Quilt neighborhoods (which tend to be in rural areas with smaller households), holding all other variables equal. For a particular postsecondary outcome (e.g., total semesters enrolled), the α c parameters can be interpreted as an index of the extent to which campuses advance that outcome, conditional on observable student characteristics. Equation (2) delivers estimates of how well each institution served the 2002 cohort, with limited controls for background variables that were correlated with students college choice, underlying ability and later success. We are careful to assess the statistical power of institutional fixed effects (i.e., the unique influence of each institution on student outcomes). Controlling for fixed effects is not necessary if students selection of campuses was completely a function of observable characteristics included in B i. Under those circumstances, we could estimate Equation (2) without campus fixed effects because student sorting (i.e., self-selection) across campuses would be pseudo-random, conditional on the observable B i. Similarly, we could control for random campus effects rather than less efficient fixed effects if the unobserved elements of college effectiveness were uncorrelated with B i. For each postsecondary outcome, and for each version of B i reported here, we reject the hypothesis that all campus fixed effects are jointly equal to zero, leading us to believe that college fixed effects are an appropriate part of the model described by Equation (2). For practical purposes, this means that some unobserved component of colleges (interpreted broadly as college quality ) affects student outcomes, even when student characteristics are controlled for. This is a necessary condition of outcomes-based funding schemes that seek to reward and incentivize college quality. Moreover, the fixed effects themselves to the extent that they are individually significant have value as indicators of campuses relative effectiveness at advancing each outcome. We strongly caution against inserting campus fixed effects such as these in funding formulae, however, without first identifying the contribution of institutional characteristics that we omit from this analysis (including, importantly, external funding) that may affect college quality. It is especially important to control for those characteristics overwhich the institution has no direct control. The Returns to Schooling For the subset of 2002 entrants who left college by 2009, we estimate Equations (1) and (2) for post-college labor market outcomes to determine the returns to schooling among all college-going Tennessee students in this sample. xiii Y i outcomes of interest are: zero/one indicators of in-state employment seven years after initial enrollment; and real annual earnings seven years after initial enrollment. 12

13 Since years of schooling or semesters enrolled can be a negative signal for non-completers (Flores-Lagunes and Light, 2010), we also estimate these models separately for degree recipients and non-completers, controlling for semesters enrolled. xiv A comparison of college effectiveness indices from education production and earnings models is illustrative. We find that, in general, institutions that are more effective at producing completion and progression measures are also relatively effective at producing human capital valued in the short term by the labor market. A shortcoming of this portion of the analysis is the fact that wage data are limited to Tennessee employers who were covered by unemployment insurance. Thus, students who moved out of state for work after college, as well as students who became self-employed or worked in the military or agriculture sectors, will be excluded from the analysis. This could bias our findings against schools with particularly effective career pipelines if, for instance, those pipelines were more likely to lead out of state. Alternatively, schools with very poor pipelines would be underrepresented in wage equations if individuals are not able to secure gainful employment, which could overstate the labor market returns to attending those schools. Table 3 shows that 66.5 percent of the 2002 cohort who were no longer enrolled as of 2009 had observable wages in that year. Students who left college but were not observed in the wage data tended to accumulate fewer college credits and semesters, were less likely to earn a degree, originated from lower-income neighborhoods and attended colleges closer to their home. This suggests that students without wage data were dominated by unemployed and/or less affluent students, more so than high-achieving students who moved to more lucrative job markets out of state. Thus, we estimate the likelihood of Tennessee wages as an outcome in and of itself, as well as models of returns to schooling using non-zero, in-state wage data. Nevertheless, the limited and incomplete nature of wage data bears additional emphasis: We do not observe investment earnings, nor can we consider non-pecuniary returns to education such as marriageability, health or reduced criminality. Evaluating the Explanatory Power of Each Model Results from Equations (1) and (2) are interesting on their own, but evaluating the explanatory power of each model will inform future research on the value and necessity of each set of student background variables. A variety of student and family information is available to institutions through their own administrative data, and these data may be sufficient to control for student inputs. Census data and consumer LifeMode profiles mapped to students home addresses may be superfluous, or they may add unprecedented explanatory and predictive power to education production functions and models of returns to schooling. We assess the extent to which different versions of Equation (2) add to the naïve model of unconditional college rankings with several in-sample and out-of-sample diagnostic indicators. For each model, we report the adjusted R 2 statistic, the overall F statistic, an F statistic for the hypothesis that all college fixed effects are equal to zero, the root mean square error (RMSE), the 13

14 Akaike information criteria and the Bayesian information criteria. The latter two statistics will inform us as to the model s fit but penalize excessive control variables. A random sample of the 2002 cohort is reserved from the main analysis for the purpose of out-ofsample testing. We use results from Equation (2) to predict each outcome for the reserved sample of students, based on their own background characteristics and the estimated fixed effects of their institutions. Residuals, equal to the difference between actual and predicted outcomes, xv are used to calculate the out-of-sample RMSE of each model. Although the main analysis focuses on the 2002 entering cohort, we have also collected college, background and wage data for the 2003 entering cohort of first-time freshmen. We follow the same procedure as described for the reserved subsample of 2002 freshmen to test the relative power of each model in predicting college outcomes and earnings for a different cohort. R e s u l t s We estimate Equation (2) for several postsecondary and post-college outcomes summarized in Table 3, separately for two-year college entrants and four-year college entrants, controlling for institution fixed effects (specifically, an indicator for which campus a student entered in 2002) and various sets of control variables representing student inputs. In each model, one institution s indicator is excluded for identification of campus fixed effects. So that these fixed effects may be compared across models and outcomes, we exclude the same indicator throughout the analysis. xvi Our discussion of results focuses on outcomes included in Tennessee s funding formula (e.g., graduation rates, transfers out of community colleges), other progression outcomes (total credits attempted, total semesters enrolled), as well as labor market outcomes. Part of our objective is to assess the explanatory value of new sources of student data. Accordingly, we estimate seven versions of Equation (2) with different combinations of explanatory variables. Model 0, equivalent to Equation (1), controls for institution fixed effects only. Coefficients return campus-level means, relative to the mean outcome of the excluded institution. Model 1 controls for institution fixed effects and THEC student data. Model 2 controls for institution fixed effects and 2000 Census data. Model 3 controls for institution fixed effects and LifeMode profiles. Model 4 controls for institution fixed effects, THEC student data and 2000 Census data. Model 5 controls for institution fixed effects, 2000 Census data and LifeMode profiles. Model 6 controls for institution fixed effects, THEC student data, 2000 Census data and LifeMode profiles. Discussion of results proceeds in three sections. First, we review the results of Models 1 3 to identify which student characteristics are strong indicators of postsecondary completion and post-college earnings. Then in Section V.b., we diagnose the overall fit and explanatory power of all seven models. Finally, in Section V.c., 14

15 we use the preferred models to assess whether estimates of college effectiveness change relative to the naïve, unconditional means of Model 0. a) Which features of student backgrounds matter for postsecondary progression and post-college earnings? Tables 7 12 report coefficient estimates from Equation (2), Models 1 3, controlling for institution fixed effects and one of three classes of student background characteristics: student data from THEC administrative records, 2000 Census data mapped to students pre-college address, or LifeMode profiles mapped to students pre-college home address. Note that sample sizes are 75 percent of the population of first-time freshmen summarized in descriptive tables; the other 25 percent are reserved from the main analysis for outof-sample testing. Table 7 reports coefficients and t-statistics for postsecondary outcomes estimated by Model 1 for four-year college entrants, controlling for campus fixed effects and THEC student data. Males were significantly less likely than females to receive a bachelor s degree within 150 percent or 200 percent of normal time (six or eight years, respectively), by an economically meaningful 8.8 and 7.1 percentage points. Relative to nonwhite, nonblack students (the excluded race/ethnicity), white and black students were no more or less likely to receive a bachelor s degree. Higher ACT scores significantly increased the likelihood of college completion, xvii but by far the largest predictor of completion was a binary indicator equal to one for students missing ACT and SAT data, who were 52.1 percentage points less likely to graduate within eight years. Recall from Table 4 that ACT/SAT scores were missing for 27.6 percent of incoming four-year students in the 2002 cohort. In later cohorts, ACT data were much more prevalent as test-taking and recording rates increased in response to lottery-funded scholarship eligibility criteria. Within the 2002 cohort, a student s ACT score may have been missing for a variety of reasons, often because he or she left college before pre-college information was digitized in administrative systems. Older enrollees were 10 percentage points more likely to graduate within six years, and students from low-income neighborhoods were percentage points less likely to graduate within six eight years. Students who traveled farther to attend college were somewhat more likely to graduate, as were students without home addresses (i.e., students without FAFSA records). We define adult students as those who were 25 or older at the time of enrollment. xviii We consistently find that adults classified as first-time freshmen tend to accumulate more credits than younger students and are more likely to graduate within 150 percent or 200 percent of normal time, which is inconsistent with other research on nontraditional students (Taniguchi and Kaufman, 2005). Our methodology departs from earlier research in a number of ways, most notably the inclusion of institution fixed effects. Figure 1 illustrates that adult enrollment shares are higher in colleges with lower overall graduation rates. Across colleges, we see much lower completion rates among adults (21.3 versus 45.4 percent for younger students), but Equation (2) 15

16 with institution fixed effects permits us to examine completion likelihoods within institutions, holding a variety of other factors constant. This approach attributes more success to adult first-time freshmen, ceteris paribus. However, the vast majority of adult enrollees were missing ACT/SAT data, which substantially reduces their projected likelihood of graduating. Tennessee s funding formula provides a 40-percent premium in rewarding colleges and universities for progression and graduation by adult and low-income students. Progression premiums for low-income students reward institutions for advancing a historically more challenging student population and, to some degree, deter institutions from filtering out low-income applicants. We find evidence in support of the Tennessee funding formula s premiums for low-income student progression. Low-income students (as defined by the median income in their pre-college Census block groups) were percentage points less likely to attain a bachelor s degree within 150 percent or 200 percent of normal time, all else equal, and if they initially enrolled in a two-year community college, they were 3.7 percentage points less likely to eventually transfer to a four-year school. Turning to Table 8 results, we find that males, students without FAFSA records or known home addresses, and students who traveled farther for college were less likely to be observed with UI-covered Tennessee wages in the 2009 calendar year, seven years after their initial enrollment. The last three columns report interesting similarities and contrasts between the student characteristics that matter for postsecondary progression versus labor market outcomes. Even though male entrants in four-year colleges were less likely to graduate than females, we observe a significant gender gap favoring males in Tennessee wages. Higher ACT scores were associated with higher earnings, and lower-income enrollees tended to have lower wages after college, particularly if they left college without a degree. Adult enrollees earned $17,720 more than younger students, conditional on degree completion. This large nominal return to schooling may have less to say about returns to higher education than these adults human capital formation prior to college. This is a question that warrants additional research. Wage models for degree completers and non-completers control for the number of semesters a student was enrolled in college. Flores-Lagunes and Light (2010) show theoretically and empirically that for college completers, additional time in school can be a negative signal to employers. Conversely, for non-completers, additional schooling can be a positive signal that leads to higher wages. Throughout this study, our results consistently support the former hypothesis and find some evidence for the latter. Degree recipients in fouryear schools were penalized $1,420 on average for each additional semester they were enrolled. In addition to the signaling mechanism, students who graduated faster had more time to accumulate human capital. Table 9 reports Model 1 results for postsecondary progression outcomes among two-year college entrants, again controlling for institution fixed effects and THEC student data. Echoing results for four-year entrants, 16

17 males were less likely to receive degrees, as were students with lower ACT scores or missing ACT scores. The ACT is not typically a requirement for community college applicants, and ACT data were missing for more than half of two-year enrollees. Adult enrollees accumulated more credits and enrolled for more semesters, and they were more likely to earn a degree. Unlike the four-year results, we find that black students in community colleges were significantly less likely to earn a degree and tended to enroll for fewer credits and semesters. Distance from home had a significant and positive relationship with progression and transfer outcomes but no relationship with degree completion. Estimates of two-year students labor market outcomes, controlling for THEC student data and institution fixed effects, are reported in Table 10. We find a significant gender gap in the form of relatively higher earnings for males, but not for degree completers, and significantly lower earnings for black students, regardless of degree completion. Higher ACT scores had a modest association with higher wages. Older twoyear enrollees earned a substantial wage premium, and non-completers earned a premium for each additional semester completed. Tables 11 and 12 list coefficients and t-statistics from Model 2 and 3 estimates of students postsecondary outcomes, respectively controlling for 2000 Census or LifeMode profiles data mapped to pre-college home addresses. xix In general, Census variables were much weaker predictors of student outcomes than THEC data on student backgrounds. Students originating from neighborhoods with more Hispanic households had weaker progression and degree completion outcomes in four-year schools. Other Census variables were insignificantly or inconsistently linked to student outcomes. LifeModes may be preferable control variables to Census fields if the former collapse neighborhood information into more meaningful indicators of student backgrounds. Marketing researchers in the private sector have utilized LifeModes and similar products for many years to quickly and accurately characterize finely grained locales, but to date we know of few studies that test the extent to which LifeModes precisely predict individual outcomes. At first glance, Table 12, reporting Model 3 postsecondary results for four-year enrollees, supports the idea that LifeModes are superior to Census variables in predicting student outcomes. Several profiles are significant predictors of student progression and degree completion. For instance, High Society students were 16.5 percentage points more likely to earn a bachelor s degree within eight years than the omitted category ( American Quilt ). Two of the lower-income LifeMode profiles are High Hopes and Global Roots, but their contrasting coefficients indicate that income is not the sole determinant of degree receipt. Collectively, however, LifeModes are weak predictors of student success in college and the workforce. Foreshadowing our discussion of model diagnostics in the following section, F-statistics and R 2 values in LifeMode models are similar to those from Census models, and both indicate much weaker explanatory power than models with THEC student data. xx We experimented with a collapsed set of LifeModes, and while there was marginal improvement in model fit, our preferred models for estimating progression and wages outcomes are those with THEC data alone. 17