Working Paper: Can Simplifying Financial Aid Information Impact College Enrollment and Borrowing? Experimental and Quasi-Experimental Evidence

Transcription

1 EdPolicyWorks Working Paper: Can Simplifying Financial Aid Information Impact College Enrollment and Borrowing? Experimental and Quasi-Experimental Evidence Kelly Rosinger 1 Recent policy efforts focus on providing simple, salient, and timely information to help consumers evaluate decisions, including college enrollment and borrowing. In 2012, the Obama administration announced the release of a simplified financial aid award notification intended to help students make informed college decisions and provide information about a college s graduation and loan default rates compared to other colleges. I examine the impact of this informational intervention using: 1) administrative data from a field experiment among admitted and enrolled students at a public university, and 2) college-level data from a quasi-experiment among the 203 four-year colleges that initially adopted the format. Experimental findings indicate a limited effect while quasiexperimental results demonstrate a decrease in the share of students borrowing federal loans. The impact differed based on how a college s graduation and loan default rates compared to other colleges, indicating reference points relating to a college s outcomes may have been particularly salient. 1 University of Virginia May 2016 EdPolicyWorks University of Virginia PO Box Charlottesville, VA EdPolicyWorks working papers are available for comment and discussion only. They have not been peer-reviewed. Do not cite or quote without author permission. Working paper retrieved from: Acknowledgements: I am grateful for the hard work and dedication of the administrators who assisted with this research at the participating university. I would like to thank Kelli Bird, Angela Boatman, Benjamin Castleman, Alissa Fishbane, James Hearn, Nicholas Hillman, Erik Ness, Asher Rosinger, and Robert Toutkoushian for their helpful comments on earlier versions of this manuscript. The research reported here was supported by a fellowship from the Jack Kent Cooke Foundation and funding from the Institute of Education Sciences, U.S. Department of Education, through Grant R305B to the University of Virginia. The opinions expressed are my own and do not represent views of the U.S. Department of Education or the Jack Kent Cooke Foundation. Working Paper 2016 Rector and Visitors of the University of Virginia. For more information please visit or contact EdPolicyWorks@virginia.edu

2 CAN SIMPLIFYING FINANCIAL AID INFORMATION IMPACT COLLEGE ENROLLMENT AND BORROWING? EXPERIMENTAL AND QUASI-EXPERIMENTAL EVIDENCE Kelly Rosinger I. INTRODUCTION In recent years, policy efforts within the federal government have focused on improving transparency and providing low-cost, simplified information to consumers as they evaluate complex decisions. Such efforts often draw on insights from the behavioral sciences to provide consumers with timely and salient information (Loewenstein, Sunstein, & Golman, 2014; Sunstein, 2014, 2011; White House Social and Behavioral Sciences Team, 2015). For example, the U.S. Department of Agriculture replaced its food pyramid with a food plate a format that is salient and easy-tounderstand for people wanting to make healthy eating choices. Similarly, the Environmental Protection Agency and Department of Transportation developed a fuel economy label for vehicles designed to help consumers better understand and estimate fuel costs and savings (Sunstein, 2011). Related efforts extend to the college-going process, during which students face a number of complex procedures and decisions regarding whether to enroll, where to enroll, and how to pay for college. Small missteps along the way missing admission or financial aid deadlines or failing to complete a form can limit a student s college options. For instance, as many as 20-30% of collegeintending high school graduates do not matriculate in part due to tasks they must complete after high school (Castleman & Page, 2014) while more than 2 million college students who might be eligible for federal student aid never submit the necessary but relatively complex Free Application for Federal Student Aid (FAFSA) (The While House, 2015). Once in college, many students underestimate how much they have borrowed or are unaware they have borrowed (Akers & Chingos, 2014; Matherson, 2016); incomplete or complex information about debt could impact subsequent borrowing decisions in college. Complexity then may prevent students from making optimal enrollment and borrowing decisions. Not surprisingly, there is growing research and policy interest in improving transparency in college costs, simplifying the college-going process, and reducing informational barriers (e.g., Advisory Committee for Student Financial Assistance [ACSFA], 2005; Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012; Castleman & Page, 2016, 2015; College Affordability and Transparency Center, 2016; Dynarski & Scott-Clayton, 2006; Long, 2010; Scott-Clayton, 2013; Whitsett & O Sullivan, 2012). In 2012, the Obama administration announced the release of the shopping 1

3 sheet, a simplified financial aid award notification developed by the U.S. Department of Education (ED) and the Consumer Financial Protection Bureau (CFPB). Award notifications are how colleges communicate information to students about specific costs and borrowing options. Notifications vary across colleges and often lack information about net cost, specific terms and conditions of loans, and may use different language to describe awards (Kantrowitz, 2010, The Institute for College Access and Success [TICAS], 2013, Whitsett & O Sullivan, 2012). As a result, they have proven difficult to interpret and, for students who receive awards from more than one college, to compare costs and borrowing options across colleges. An initial group of nearly 700 postsecondary institutions agreed to use the shopping sheet during the academic year (ED, 2012). By 2016, more than 3,000 postsecondary institutions had adopted the shopping sheet for some or all students in awarding financial aid. 1 Nearly three-quarters of undergraduate students enrolled at institutions that participate in federal student aid programs attend a college that uses that shopping sheet (ED, 2016), and a preliminary draft of the upcoming reauthorization of the Higher Education Act (2014) would require institutions to use a standardized format in awarding aid. The shopping sheet is intended to help students make informed college decisions by simplifying information students receive from colleges about costs and borrowing options and making it easier to compare awards across colleges. The shopping sheet also provides information that compares a college s performance relative to other colleges on two outcomes: graduation rate (ranked as low, medium, or high relative to other colleges) and loan default rate (relative to the national average). In addition to simplifying information then, the shopping sheet provides comparative information and reference points to help students assess a college s performance on these outcomes. Previous research demonstrates the context and framing of information can impact decisions (e.g., Allcott, 2011; Kahneman & Tversky, 1979; Tversky & Simonson, 1993). By providing explicit reference points, the shopping sheet may make colleges that rank higher (or lower) on these metrics more (or less) attractive relative to other colleges. In doing so, students may be more likely to enroll at colleges that perform better relative to others. Similarly, students may adjust borrowing decisions based on a college s performance. 1 An executive order (Exec. Order No. 13,607, 2012) requires postsecondary institutions that agree to comply with the VA Principles of Excellence to use a standardized award format for students eligible to receive federal military and veterans educational benefits. Of the initial 700 postsecondary institutions that adopted the shopping sheet, 581 provided it to all students who received a financial aid award and 118 provided it only to students eligible for military and veterans educational benefits. 2

4 In this study, I draw on experimental and quasi-experimental data to evaluate how this informational intervention affects enrollment and borrowing decisions. In particular, I investigate the impact of the shopping sheet when colleges are close to the reference points colleges that might be perceived as average and when colleges rank better (or worse) than an average college. The first data source comes from a field experiment conducted among admitted and enrolled students at a public university. The university is a relatively average college with a graduation rate in the medium range and a loan default rate near the national average. As such, it provided an appropriate context in which to examine the impact of the shopping sheet at an average college. I then draw on college-level data from a quasi-experiment to examine the overall impact among the broader range of 203 primarily bachelor s degree granting colleges that adopted the shopping sheet during its first year. For these colleges, I consider the potentially different impact the informational intervention may have had depending on how a college s graduation and loan default rates compared to other colleges. Findings from the field experiment indicate a limited effect of the shopping sheet while results from the quasi-experiment among the initial group of adopting colleges demonstrate a small decrease in the share of students borrowing federal loans. The reference points indicating how a college s outcomes compared to others mattered: 2 a larger share of admitted students enrolled at colleges with high graduation rates while a smaller share enrolled at colleges with low graduation rates relative to those in the medium range. Decreases in the share of students who borrowed were particularly pronounced at shopping sheet colleges with high loan default rates and were attenuated at colleges with high graduation rates, demonstrating students were less likely to borrow at colleges that performed below average as indicated by reference points. In Section II, I draw on insights from behavioral economics to understand why students encounter difficulty interpreting and comparing award notifications and how this could lead to suboptimal college decisions. Section III describes the field experiment examining the shopping sheet s impact on enrollment and borrowing at a public university. Section IV describes the quasiexperiment that uses data from primarily bachelor s degree granting colleges to examine how the shopping sheet influenced enrollment and borrowing at the group of colleges that initially adopted 2 The shopping sheet categorized a college s graduation rate as low (below 37.3%), medium (between 37.3 and 56.7%), and high (higher than 56.7%). Loan default rate was provided in comparison to the national average (13.4%). 3

5 the shopping sheet. Section V concludes with a discussion of experimental and quasi-experimental results and their policy implications. II. CONCEPTUAL FRAMEWORK The traditional approach to studying students decisions about whether and where to enroll as well as how much to borrow has been from a cost-benefit perspective (Becker, 1962). However, insights from behavioral economics suggest students do not always make decisions this way, particularly when decisions involve processing complex information. Previous research demonstrates that relatively low-cost interventions designed to simplify the college-going process and reduce informational barriers can have a large impact. Experimental research, for example, indicates that providing assistance with the FAFSA (Bettinger et al., 2012) and sending text messages reminding college students to re-file the FAFSA (Castleman & Page, 2016) can improve college enrollment and persistence. Financial aid award notifications might be difficult for students to interpret and to compare across colleges for a number of reasons. Award notifications contain multiple pieces of information about costs, grant and scholarship aid, and different types of loans. The amount of information students receive may make it difficult to know what information is most important to consider in making enrollment and borrowing decisions. Unless information stands out or seems particularly salient, students may ignore it (Loewenstein et al., 2014). Students also may receive award notifications from multiple colleges, making the process of interpreting and comparing awards more complicated because colleges differ in what information is included and the language used to describe awards. Colleges also differ on multiple dimensions, e.g., cost, quality, and location, attributes that are non-alignable in that they often involve tradeoffs of some kind between attributes (e.g., a student may travel further to attend a college that is perceived to have higher quality than another college). Decisions that involve non-alignable attributes require a great deal of cognitive effort, and people often put off making a decision or regret their decision when faced with such choices (Gourville & Soman, 2005). Processing information relating to nonalignable incentives may be especially difficult when information students receive differs at each college. Complex or unclear (and sometimes missing) information also may make it difficult to assess the outcomes of college decisions. Typically, award notifications do not contain information about student outcomes, for example, the share of students who graduate and who make payments on 4

6 their loan balances. Because individuals have limited attention and are biased toward readily available information (Kahneman, 2011), students may not seek out this information despite its importance in determining the outcomes students might expect when they leave college. Students decisions also may be biased toward the present, for example, by borrowing the maximum amount they can to have more funds in the present rather than focusing on the potential long-term impact of high debt levels (Barr, Bird, & Castleman, 2016). Complexity associated with college enrollment and borrowing decisions is compounded by students limited experience making college decisions. Most students make enrollment and borrowing choices just a few times in their lives. As a result, they have little familiarity evaluating financial aid awards and few opportunities to get decisions correct. Feedback on decisions is delayed, and students may not know until after they leave college whether they borrowed an appropriate amount. As a result, students are not able to learn from and correct previous mistakes when they decide where to enroll and how much to borrow each year. A few recent and ongoing studies designed to help students make informed borrowing decisions demonstrate that interventions delivered to students as they evaluate financial aid awards can impact borrowing. These studies indicate that providing information about cumulative debt and future monthly payments (Lorin, 2014; Schmeiser, Stoddard, & Urban, 2015), changing default borrowing options (Marx & Turner, 2016), and sending text messages with information about loans and offers of counseling assistance (Barr et al., 2016) can decrease the amount students borrow. In this study, I provide an early evaluation of a recent federal policy effort designed to help students evaluate financial aid awards. The shopping sheet is intended to provide simplified and salient information at the point at which students evaluate their specific college costs and borrowing options. Colleges that have adopted the shopping sheet typically include the one-page document as a supplement to the information already provide to students. The shopping sheet has several features distinguishing it from many award notifications (Appendix A provides the shopping sheet), in particular, it provides 1) net cost, which is the cost of attendance adjusted for grant aid that students do not have to repay, and then lists loans and other options for paying net cost, and 2) information about a college s graduation rate, loan default rate, and median borrowing. Graduation and loan default rates are compared to figures at other colleges: graduation rate is ranked as low, medium, or high relative to other colleges that primarily offer the same degree (e.g., bachelor s); loan default rate 5

7 is provided in comparison to the national average. 3 In this study, I examine how information in the shopping sheet impacts college decisions, asking the following questions: 1. How does the shopping sheet impact enrollment and borrowing decisions? 2. How do reference points in the shopping sheet that compare a college s graduation and loan default rates to other colleges impact enrollment and borrowing decisions? Rather than influencing students college decisions in one particular direction, the shopping sheet is intended to help students make more informed decisions by allowing them to easily understand and compare awards. However, by clearly distinguishing loan offers from grant aid and providing comparative information about a college s graduation and loan rate, the shopping sheet may make loans particularly salient. As a result, the shopping sheet may influence students decisions about whether to borrow and how much to borrow and, for students who receive financial aid awards from more than one college, could potentially influence enrollment decisions. In particular, reference points relating to a college s graduation and loan default rates may prove particularly salient to enrollment and borrowing decisions. By providing reference points relating to a college s outcomes, the shopping sheet may make colleges that score better (or worse) on these metrics more (or less) attractive. In other contexts, changing reference points increased credit card tips for taxi drivers (Grynbaum, 2009), and providing ratings for households energy consumption relative to neighbors lowered use in high-consumption households (Allcott, 2011). In the context of the shopping sheet, students enrollment and borrowing responses may differ based on whether a college ranks higher or lower relative to the reference point. I hypothesize that students will be more likely to enroll at colleges that score better on these metrics, i.e., those with higher graduation rates or lower loan default rates relative to other colleges. Likewise, students borrowing decisions may differ depending on these same metrics because students are more easily able to assess students financial outcomes at colleges. In particular, I hypothesize that students will be less likely to borrow and those who do borrow will borrow less at colleges that perform worse on these metrics, i.e., colleges with low graduation rates or higher than average loan default rates. I anticipate the impact of the shopping sheet on enrollment and borrowing decisions will be larger at colleges in the middle colleges with graduation rates in the medium range than at colleges that rank high or low relative to others. Finally, I hypothesize that the impact of the shopping sheet may be stronger at colleges that 3 The shopping sheet does not provide comparison information on median borrowing. 6

8 are framed as a potential loss with the introduction of reference points, i.e., colleges at which a smaller share of students graduate or a higher share of students default on loans. People tend to be loss averse, prefering to avoid losses than experience gains (Kahneman & Tversky, 1979), indicating students may be particularly responsive to information that a college ranks lower relative to others. III. EXPERIMENTAL EVIDENCE I collaborated with administrators at a public university during the spring of 2013, the first year colleges adopted the shopping sheet, to evaluate the impact of this informational intervention on students enrollment and borrowing decisions. The university was a broad access college, admitting more than 80% of applicants, and enrolling primarily students from the state and region in which it is located. The average annual net cost was around $15,000, just below the national average of $16,500 for primarily bachelor s degree granting colleges (College Scorecard, 2016). Students at the participating university graduate at about the same rate as their peers at similar colleges: the sixyear graduation rate was in the medium range (~45%) relative to other colleges that primarily award bachelor s degrees. The loan default rate (~14%) was near the national average (13.4%), and median borrowing was around $14,500, which is below the average of $17,800 for students at primarily bachelor s degree granting colleges. The relative similarity of the participating university to the average four-year college provides an appropriate context in which to evaluate the impact of this informational intervention at an average college. Sample and Data The informational intervention focused on two groups of students: 1) students who had been admitted to the participating university, and 2) students who were currently enrolled in their first year at the university. The sample of admitted students facilitated the examination of college enrollment and borrowing among a group of students making these decisions for the first time; the sample of enrolled students facilitated a consideration of borrowing decisions students make while in college. Data come from admissions, financial aid, and enrollment records at the participating university. Outcomes include: 1) a dichotomous variable indicating whether a student enrolled at the participating university (for admitted students), 2) a dichotomous variable indicating whether a student borrowed, and 3) a continuous variable indicating the amount borrowed. Borrowing outcomes relate to federal loans and include subsidized and unsubsidized Stafford and Perkins 7

9 loans. 4 For students who did not enroll at the participating university, data on borrowing was not available. Rather than conditioning these analyses on enrollment, which could result in biased estimates if there were unobserved differences between students who did not enroll in treatment and control groups, I coded these values as 0 and included a dichotomous indicator for missingness in borrowing outcomes. 5 The independent variable of interest was a dichotomous variable indicating whether a student was assigned to receive the shopping sheet in his/her financial aid award notification. Administrative records also included data on students academic (high school GPA, ACT score, whether a student had taken more than 15 credit hours for students who were already enrolled), socioeconomic and financial (amount of grant aid received for students who enrolled at the university, whether a student was eligible for the federal Pell grant, parent income, whether a student had at least one parent with a college degree, amount previously borrowed for students enrolled at the university, whether a student was an in-state resident), and demographic (ethnicity, gender) characteristics. Intervention Design Students received financial aid awards during Spring/Summer 2013, and I observed enrollment and borrowing outcomes during the academic year. Students assigned to treatment received the shopping sheet in addition to the award notification traditionally used by the participating university; students assigned to the control group received the traditional award notification. 6 Students received award notifications at the participating university through their online financial aid account. The university s traditional notification includes several screens of information that students click through to find information about and accept their financial aid. Appendix B provides the screen that lists a student s aid award. Because notifications vary across colleges, results from the field experiment may be more generalizable to institutions with similar notifications to the university in this study. However, the shopping sheet (provided in Appendix A) differs from many traditional award notifications, including the one used by the participating university, by providing: 4 Less than 1% of students at the participating university had data on non-federal (private) loans that were reported to the university. 5 Results from models conditioning on enrollment indicate larger effect sizes (because missing values are no longer coded as 0) but are similar in significance to those presented. 6 Colleges can use the shopping sheet as a supplement or as a replacement to the award notifications they already provide students. Colleges that have adopted the shopping sheet, including the participating university, generally include the shopping sheet in addition to information already provided. 8

10 1) cost and aid information together on one page, 2) loan offers separate from grant aid and listed after net cost (cost of attendance adjusted for grant aid), and 3) a college s outcomes relating to graduation rate, loan default rate, and median borrowing. Students in the treatment group could view the shopping sheet in their online account and received a paper copy in the mail. When students accessed their account, they could click on multiple tabs, including one for the shopping sheet, to view their complete award. Technological limitations prevented tracking what tabs students viewed online or how long they accessed each page. As a result, estimates for all models reflect the intent to treat, or the effect of being assigned to treatment, on enrollment and borrowing rather than the actual treatment effect. Sample Size and Baseline Equivalence Research provides mixed evidence for whether and how information affects college decisions (e.g., Hoxby & Turner, 2013; Oreopoulos & Dunn, 2013). For example, effect size was.16 standard deviations in college expectations for one intervention that provided information about college costs, earnings, and financial aid to high school students (Oreopoulos & Dunn, 2013; What Works Clearinghouse, 2013) and.12 standard deviations on enrollment in a study that provided information to high-achieving, low-income students on the college application process and college costs in addition to application fee waivers (Hoxby & Turner, 2013; What Works Clearinghouse, 2014). I selected sample sizes that accounted for the possibility that the shopping sheet had a small influence on behavior. 7 The sample size of 2,655 admitted students (N = 1,100 in treatment group) provided statistical power to detect an effect size of.11 on enrollment, or about one-tenth of a standard deviation difference in treatment and control group means, at 80% power,.05 significance, and 10% variance explained. The sample size of 855 enrolled students (N = 437 in treatment group) provided statistical power to detect a.19 effect size using the same criteria. Table 1 provides treatment and control group means for baseline covariates and differences in means between groups for the admitted student sample (columns 2-4). I find no significant differences between treatment and control group means on baseline covariates among admitted students. I also conducted an F-test by regressing treatment status on pre-treatment covariates to determine whether covariates jointly predicted assignment to treatment. The test failed to reject the null hypothesis that coefficients were equal to zero (p = 0.68), suggesting observable covariates did not jointly predict assignment to treatment. 7 Power analyses were conducted using PowerUp, a tool developed to determine minimum detectable effect sizes and sample sizes in education research (Dong & Maynard, 2013). 9

11 In the sample of students enrolled at the participating university when the study was conducted (columns 6-8), students in the treatment group were six percentage points more likely to have completed more than 15 college credit hours than students in the control group (54% of treatment group compared to 48% of control group). This difference was marginally significant at the.10 level. Students in the treatment group were more likely to have had a parent with a college degree than those in the control group: 75% of students in the treatment group had at least one parent with a college degree or higher compared to 68% in the control group, a difference that was significant at the.05 level. Although observable covariates did not jointly predict assignment to treatment (p = 0.13), the estimated impact of the shopping sheet for this sample may reflect observable or unobservable differences between treatment and control groups that relate to previous course taking or parents educational attainment. Analytic Technique I used linear probability models to estimate the effect of being assigned to treatment on whether a student enrolled and whether a student borrowed and linear models for the amount borrowed. 8 I first estimated a simplified model using treatment status to predict enrollment and borrowing outcomes and then estimated a model with the inclusion of baseline academic, socioeconomic, financial, and demographic covariates. The full model can be expressed:! " = $ + & ' (h*++,-.(h//0 " + & 1 2 " + 3 " where! " is the outcome variable, (h*++,-.(h//0 " indicates whether a student received the shopping sheet and & ' is the intent-to-treat effect. 2 " is a vector of baseline student-level covariates, which reduce unexplained variance and lead to more efficient estimates and increased statistical power (Murnane & Willett, 2011). I estimated all models using robust standard errors. Receipt of the shopping sheet also may have had a stronger effect on the enrollment and borrowing decisions of low-income students than those of their higher-income peers. Lower-income students face higher informational barriers surrounding costs and aid and are more likely to overestimate college costs than higher-income groups (Avery & Kane, 2004; Grodsky & Jones, 2007; Horn, Chen, & Chapman, 2003). As a result, these students may be more responsive to information in the shopping sheet relative to their higher-income peers. I analyzed data for 8 Estimates from logistic, probit, and tobit models were similar in signs and significance to those presented in the following section. 10

12 heterogeneous intent-to-treat effects for this subpopulation of students by interacting Pell grant eligibility status with treatment status. 9,10 Results Table 2 presents results from the intent-to-treat analysis for admitted (columns 1-6) and enrolled (columns 7-10) students. For each outcome, the first column presents results from a simplified model using only treatment to predict outcomes; results in the second column for each outcome come from a full model that includes baseline covariates. Results indicate that assignment to receive the shopping sheet did not have a statistically significant effect on whether a student enrolled, whether a student borrowed federal loans, or how much a student borrowed. However, in some cases, results may actually point to some potential unintended consequences of this particular policy effort. Admitted students who were selected to receive the shopping sheet were less likely to enroll, on average, than students in the control group. When it came to borrowing, both admitted and enrolled students selected to receive the shopping sheet were slightly more likely to borrow and to borrow somewhat more on average. Overall, results demonstrate a potentially limited impact of the shopping sheet on enrollment and borrowing at the participating university although lack of statistical precision in estimates makes it difficult to rule out potentially small negative impacts on enrollment as well as a increased likelihood of borrowing and an increase in the amount borrowed. Table 3 presents results from models examining whether the shopping sheet had heterogeneous intent-to-treat effects on enrollment and borrowing decisions of Pell eligible students. Columns 1-6 provide results for admitted students, and columns 7-10 provide results for enrolled students. In most cases, the interaction between treatment and Pell eligibility status had the same sign as the treatment effect. The effects, however, were not individually or jointly (with treatment) significant. 9 Focusing on this subgroup resulted in a loss of statistical power. The 1,217 admitted students who were eligible for the Pell grant provided power to detect an effect size of.15 on enrollment. For 339 Pell eligible students who were enrolled at the participating university, statistical power allowed for a minimum detectable effect size of.29 on borrowing decisions. 10 I tested the robustness of heterogeneous intent-to-treat effects to an alternate measure of financial need: a dichotomous variable indicating whether a student s expected family contribution was zero, which indicates a limited ability to contribute to educational expenses. Results from these models were similar to those presented in the following section. 11

13 IV. QUASI-EXPERIMENTAL EVIDENCE When the U.S. Department of Education first released the shopping sheet, 203 primarily bachelor s degree granting public and private colleges adopted the new format for all students who received a financial aid award (author calculation using data from the College Solution website, 2013). In the field experiment, I focused on the impact of this informational intervention at one university. In the second part of this study, I evaluated the adoption of the shopping sheet at a broader range of public and private colleges that primarily award bachelor s degrees. I first considered the overall impact of the shopping sheet and then examined how reference points relating to a college s graduation and loan default rates in the shopping sheet influenced enrollment and borrowing. Sample and Data Data came from the National Center for Education Statistics Integrated Postsecondary Education Data System and the U.S. Department of Education s College Scorecard ( The sample included 549 public and 1,161 private fouryear primarily bachelor s degree granting colleges in the United States observed annually from the to the academic year. Colleges first used the shopping sheet in Spring 2013 to award financial aid for the academic year, providing six years of pre- and one year of post-policy observations. Nearly 25% of primarily bachelor s degree granting public (N = 136) and 6% of private (N = 67) colleges in the sample adopted the shopping sheet. I defined shopping sheet colleges as those that provided the shopping sheet to all students who received a financial aid award. An additional 31 primarily bachelor s degree colleges adopted the shopping sheet for students who received funding through federal military and veterans educational benefits programs. I excluded these colleges from analyses because they did not provide the shopping sheet to every student who received a financial aid award notification. Outcome variables were the percent of admitted students who enrolled (admissions yield rate), the percent of full-time, first-time undergraduates who borrowed federal loans (subsidized and unsubsidized Stafford and Perkins loans), and the average dollar amount that full-time, first-time undergraduates borrowed in federal loans (among those who borrowed). These outcomes were measured at the college-level but correspond to student-level outcomes examined in the field experiment. 12

14 The independent variable of interest was a dummy variable indicating whether a college used the shopping sheet in years after the adoption of the new format ( ). 12 I also controlled for several college-level covariates that were likely to influence whether a college adopted the shopping sheet as well as enrollment and borrowing. These included tuition and fees, average amount of grant aid per student, percent Pell grant recipient enrollment, median borrowing at time of departure from the college, 13 six-year graduation rate, percent of applicants who were admitted, and total undergraduate enrollment. Financial figures were inflation-adjusted using the Consumer Price Index and scaled to ease interpretation of coefficients. Analytic Technique The adoption of the shopping sheet by this initial group of colleges provided a quasiexperiment in which students at adopting colleges received the informational intervention and students at non-adopting colleges did not. Since the data covered years before and after adoption, I used a difference-in-differences fixed effects estimation strategy to examine changes in enrollment and borrowing before and after adoption at shopping sheet and non-shopping sheet colleges. In a regression context, the model can be expressed: 4 "5 = & 0 + & 1 (h*++,-.(h//0 "5 + & 2 2 "5 + 6 " "5 where 4 "5 is the outcome variable; (h*++,-.(h//0 "5 is a dummy variable equal to 1 for shopping sheet colleges in post-policy years (2014), 2 "5 is a vector of college-level covariates; 6 " are college fixed effects; 7 5 are year fixed effects; and 3 "5 is the error term. The inclusion of college fixed effects accounted for time-invariant characteristics of colleges that could also relate to the decision to adopt the shopping sheet or the outcomes. Year fixed effects captured common shocks in enrollment and borrowing trends that shopping sheet and comparison colleges may have experienced. Robust standard errors were clustered at the college level (Bertrand, Duflo, & Mullainathan, 2004). While these models provided an overall estimate of how efforts to simplify financial aid awards impact enrollment and borrowing decisions, the shopping sheet s prominently featured information on graduation rate, loan default rate, and median borrowing may have shaped the outcomes of this informational intervention in different ways. In particular, reference points that the 11 Year was defined as the end of the academic year (e.g., 2014 indicated data came from the academic year). 12 The list of colleges that adopted the shopping sheet came from College Solution, a college advising website ( The U.S. Department of Education updates its list of colleges using the shopping sheet periodically to reflect the most recent colleges adopting the format. Data from the first year of adoption was not publicly available through the ED. 13 Median borrowing data, which come from the College Scorecard, represented a two-year average. 13

15 shopping sheet provided comparing a college s graduation and loan default rates to other colleges may prove to be particularly salient anchors that could have influenced students enrollment and borrowing decisions. The shopping sheet ranked colleges graduation rate as low, medium, or high relative to other primarily bachelor s degree granting colleges and provided colleges loan default rate relative to the national average. I defined graduation rate as low (below 37.3%), medium (between 37.3 and 56.7%), and high (higher than 56.7%) using the ED s defined cutoff points on the shopping sheet and created dichotomous variables for each category, using colleges in the medium range as the referent category. Between 27% and 41% of both public and private colleges graduation rates fell in each category. I defined loan default rate as above the national average if a college s default rate was greater than (or equal to) the national average of 13.4%, using colleges below the national average as the referent group. Around 85% of both public and private colleges in the sample had a loan default rate below the national average. I interacted each of these dichotomous variables with the treatment indicator to examine whether the shopping sheet had a different impact on enrollment and borrowing at colleges with high and low graduation rates (relative to colleges with medium graduation rates) or loan default rates above the national average (relative to colleges with default rates below the national average). The difference-in-differences analysis assumes treatment and comparison colleges experienced similar enrollment and borrowing trends in years prior to the adoption of the shopping sheet. Figure 1 plots the three outcomes over time at public shopping sheet and comparison colleges. The admissions yield rate at shopping sheet and comparison colleges was between 45 and 40% from While both followed a similar trend, shopping sheet colleges experienced a slight dip in admissions yield rate in 2011, recovered in 2012, and by 2014 (the first year after shopping sheet adoption) experienced a small increase relative to comparison colleges. Initially higher at shopping sheet colleges, the percent of students who borrowed increased at both groups of colleges until 2012 at which point a higher share of students at comparison colleges borrowed. After 2012, the percent of students who borrowed declined at both but more sharply at shopping sheet colleges. The amount borrowed among borrowers increased and then leveled off at both shopping sheet and comparison colleges. Figure 2 provides a visual depiction of trends in outcomes at private colleges. The admissions yield rate at private shopping sheet colleges was lower throughout the period but the gap narrowed until 2013 and diverged in 2014, the first year after the shopping sheet. The percent of 14

16 students who borrowed followed a similar trend at shopping sheet and comparison colleges after Trends in the amount borrowed among borrowers were similar at both groups of colleges, with a slight increase at shopping sheet colleges in 2013 one year before shopping sheet adoption before meeting again in I used several strategies to account for potential differences in enrollment and borrowing trends. First, I estimated separate models for public and private colleges because motivations for adopting the shopping sheet as well as enrollment and borrowing figures and trends are likely to be different for each. For example, public colleges may have been more likely than private colleges to adopt the shopping sheet because of political pressure or a state system s decision. Borrowing, in particular, differed for public and private colleges with higher rates and levels of borrowing at private colleges in the sample. By separating public and private colleges, then, shopping sheet and comparison colleges were more similar within each sector. Second, I estimated a model that included college-specific linear trends as controls to account for differences in enrollment and borrowing trends at each college in the sample. Results from these models, presented below, were similar in signs and significance to models that did not include college-specific trends. Finally, I conducted placebo tests using pre-policy years ( ) in which I assigned treatment to each year prior to actual policy adoption ( ) to determine whether there were significant differences in enrollment and borrowing between shopping sheet and comparison colleges prior to actual adoption. Summary Statistics Public and private colleges that adopted the shopping sheet had similar selectivity levels (measured by the percent of applicants admitted) but higher tuition and fee levels, slightly higher graduation rates, and smaller shares of Pell grant recipients than comparison colleges. Students at shopping sheet colleges also borrowed somewhat more, on average, than students at comparison colleges. Summary statistics for shopping sheet and comparison colleges are provided in Tables 4 (public colleges) and 5 (private colleges). The first two columns of each table provide summary statistics for shopping sheet colleges in the first year, 2008, and last year, 2014, of analysis; the last two columns provide summary statistics for comparison colleges in each of these years. Financial figures are in constant 2013 dollars. Public colleges Tuition and fees were around $1,000 higher at shopping sheet colleges in both 2008 ($7,700) and 2014 ($9,000) than at the comparison group of colleges (Table 4). The average amount of grant 15

17 aid students received was around $6,200 in 2008 and $7,600 in 2014 at both shopping sheet and comparison colleges. The average student at a shopping sheet college left with $14,000 in debt in 2008 and nearly $17,000 in At comparison public colleges, students left with around $1,000 less in debt on average in each year. A little more than half of students graduated with a bachelor s degree within six years at shopping sheet colleges while just under half graduated within six years at comparison colleges in both 2008 and Admission rates one measure of a college s selectivity were similar at shopping sheet and comparison colleges. Shopping sheet colleges enrolled smaller shares of Pell recipients (29% in 2008; 37% in 2014) than comparison colleges at which 32% of students received the Pell grant in 2008 and 43% in Private colleges Similar trends emerged between shopping sheet and comparison colleges in the private sector (Table 5). Both tuition and grant levels were higher at private shopping sheet colleges over the same period than at comparison colleges. In 2014, tuition at private shopping sheet colleges was $28,000 and aid was $19,000, about $2,000 higher for both measures than comparison private colleges in the same year. Students left private shopping sheet colleges with around $1,000 more debt than comparison colleges in 2008 and Students borrowed an average of $19,000 at shopping sheet colleges and $18,000 at comparison colleges in The share of applicants admitted was similar at shopping sheet and comparison colleges. A little more than one-quarter of students at shopping sheet colleges received the Pell grant in 2008, a figure that increased to slightly more than one-third by At comparison colleges, Pell recipients represented around one-third of students in 2008 and two-fifths in Results Public colleges Table 6 presents difference-in-difference estimates of the shopping sheet s overall impact on admissions yield rate (columns 1-3), the share of students who borrowed federal loans, (columns 4-6), and the average amount borrowed among students who took out federal loans (columns 7-9) at public colleges. The first column for each outcome variable provides estimates from models that include the treatment indicator and college and year fixed effects, the second column adds collegelevel covariates, and the third column includes college-specific trends. Results indicate the shopping sheet led to a decline in the share of students who borrowed federal loans to finance their education. The direction and significance of this effect was consistent across models when college-level covariates and college-specific trends were included. Findings 16

18 demonstrate the shopping sheet decreased the share of students who borrowed by 2.6-percentage points, controlling for college and year fixed effects (column 4). When I included college-level covariates in the model, the shopping sheet was associated with a 2- percentage point decrease in borrowing (column 5). In the final model that includes college-specific trends, the shopping sheet led to a 1.4-percentage point decrease in the share of students borrowing (column 6). This finding was statistically significant across the three models. The shopping sheet did not have a statistically significant influence on admissions yield rate or the average amount borrowed among borrowers. I next examined how reference points relating to graduation and loan default rates impacted enrollment and borrowing decisions. Table 7 lists each of these categories in the first column followed by the shopping sheet treatment indicator and the interaction of the treatment indicator with the college characteristic of interest. Columns 1 and 2 present the difference-in-differences estimates for each of these variables on admissions yield rate; columns 3 and 4 for the percent of students who borrowed; and columns 5 and 6 for the average amount borrowed. The first column for each outcome provides estimates from models that include college covariates and college and year fixed effects, and the second column provides estimates that also include college-specific trends. Results indicate that the shopping sheet s impact varied based on how a college s graduation and loan default rates compared to other colleges. The shopping sheet led to a small but statistically significant decrease in admissions yield rates at colleges with low graduation rates and an increase at colleges with high graduation rates, relative to colleges with graduation rates in the medium range. The shopping sheet led to a statistically significant decrease in the share of students borrowing across all groupings, with the largest decrease occurring at shopping sheet colleges with low graduation rates and the smallest decrease at shopping sheet colleges with high graduation rates. Whether the shopping sheet categorized a college s loan default rate as above or below the national average also influenced students decision to borrow, with the largest decline occurring at colleges with default rates above the national average. Finally, findings indicated some evidence of loss aversion in response to information about how a college performed relative to others in graduation and loan default rate. The size of the coefficients at colleges that performed worse than others relative to the reference point those with low graduation rates and those with loan default rates above the national average suggested students were more responsive at colleges that ranked below the reference point than at those above, suggesting students may demonstrate some loss aversion in response to this information. 17