Changing Distributions: How Online College Classes Alter Student and Professor Performance

Changing Distributions: How Online College Classes Alter Student and Professor Performance Eric Bettinger, Lindsay Fox, Susanna Loeb and Eric Taylor

Changing Distributions: How online college classes alter student and professor performance Eric Bettinger, Lindsay Fox, Susanna Loeb. and Eric Taylor Stanford University August 2014 Abstract: Online courses for college students are rapidly expanding, especially at private forprofit institutions, yet little extant research identifies the effects of online courses on students academic success. We study students and professors at DeVry University the fourth largest forprofit institution in the country where two-thirds of classes are conducted online. Using an instrumental variables approach, this study finds that on average students perform worse in online classes, compared to how they would have performed in a traditional in-person class setting. Our instrument limits identifying variation to differences in online versus in-person course taking that arise from variation over time in which courses are offered in-person at the student s local DeVry campus. Additionally, we find that student performance is more variable online than in traditional classrooms. This greater variation is not, however, due to greater variation in professor performance. In fact, individual professors explain less of the variation in student performance in online settings than in traditional classrooms. The results highlight differences in students abilities to productively use online offerings, and the potential of online settings to reduce within-course variation in quality.

I. Introduction Online college courses are a rapidly growing feature of higher education. One out of three students now takes at least one course online during their college career, and that share has increased 319 percent over the past decade (Allen and Seaman 2013). The promise of cost savings, partly though economies of scale, fuels ongoing investments in online education by both public and private institutions. Broad access and for-profit institutions, in particular, have aggressively used online courses. Yet there is little systematic evidence about how online classes affect student outcomes. In this paper we estimate the effects of taking a course online, instead of in-person, on student achievement in the course and persistence in college after the course. We examine both the mean difference in student outcomes and how online courses change the variance of student outcomes. We give specific attention to professors contributions to student outcomes, and how teaching online changes those contributions. We study students and faculty at DeVry University, which enrolls more than 100,000 undergraduate students. DeVry students on average take two-thirds of their courses on-line and their other courses in-person at one of DeVry s over 100 campus locations. 1 Online and inperson classes follow the same syllabi, use the same text books, and have the same class sizes. However, in online sections all communication lecturing, class discussion, group projects occurs in online discussion boards, and most of the professor s lecturing role is replaced with standardized videos. In online sections participation is often asynchronous, while in-person sections meet on campus at scheduled times. 1 We track 102 campuses. DeVry opens and closes campuses each year, and hence, our number may differ from the current number of active campuses. 1

The challenge when seeking to understand the effects of online courses on mean student outcomes and on the variance of outcomes is that students themselves often choose whether they will take a given course online or in a more traditional in-person classroom setting. Thus, even if we observe students in online classes doing worse (better), that difference may simply arise because lower-skilled, less-prepared (higher-skilled, more-prepared) students choose to take classes online more often. To address this potential selection we take an instrumental variables approach which limits identifying variation to differences in each local campus s course offerings over time. A student can choose to take a course in-person, instead of online if the course is offered at their local campus, but local campus offerings vary from term to term (DeVry divides its academic calendar into six eight-week terms.). This variation in in-person offerings from term to term is the core of our instrument. As an alternative, we also instrument using the distance between students homes and the nearest campus. Our results suggest that online courses do less to promote student learning and progression than do in-person courses. In our most conservative specifications, online courses typically reduce the probability that a student gets an A by almost 6 percent, reduce the probability of passing by 4 percent, increases the probability of early withdrawal by 2.5 percent, and decreases the likelihood of enrolling in the next term and one year later. Additionally, we find that student achievement outcomes are more variable in online classes. By contrast we find that professors contributions to student outcomes are less variable in online classes, even among professors who teach in both. A number of mechanisms could lead students to perform better (worse) in online college classes. Online courses substantially change the nature of communication between students, their peers, and their professors. First, on a practical dimension, students can participate at any hour of 2

the day from any place. That flexibility could allow for better allocation of students time and effort. That same flexibility could, however, be a challenge for students who have not yet learned to manage their own time. Second, online asynchronous interactions change the implicit constraints and expectations on academic conversations. For example, a student does not need to respond immediately to a question her professor asks, as she would in a traditional classroom, instead she can take the time she needs or wants to consider her response. More generally, students may feel less oversight from their professors and less concern about their responsibilities to their professors or classmates in the online setting. 2 Third, the role of the professor is quite different in online classes. Notably, eliminating differences in professors lectures through standardized video lectures should reduce the between-professor variation in student outcomes. Indeed such standardization is a necessary condition for the economies of scale promised by online education. However, how professors choose to use the time saved by not lecturing could expand that between-professor variation. Evidence from K-12 schools consistently shows large and important between-teacher variation in student outcomes (for example Hanushek, Rivkin and Kain, 2005; Chetty, Friedman, and Rockoff, 2013a, 2013b), yet we largely ignore such variation in higher education (see exceptions by Carrell and West, 2010; Bettinger and Long, 2012). All these potential mechanisms suggest heterogeneous effects of online classes from student to student. Students skills, motivations, and work- and home-life contexts may be better or worse served by online classes. While some existing work estimates average effects, few studies have assessed differences across students, and none that we know of systematically assessed differential effects due to different course characteristics (e.g. different professors). 2 In a separate paper we study peer effects in DeVry s online college classes (Bettinger, Loeb, and Taylor 2014). 3

Our research contributes to three strands of literature. First, the study provides significant evidence of the impact of online courses in the for-profit sector across hundreds of courses. Existing evidence with much smaller samples shows negative effects on course completion and course grades at community colleges (Xu and Jaggers, 2011; Streich, 2013; Hart, Friedman and Hill, 2014) and student exam scores at public four-year institutions (Figlio et al. 2013, Bowen, Chingos, Lack, and Nygren 2014). 3 Students who attend for-profit colleges are distinct from other student populations in higher education, and our estimates are the first to focus on the impact of online courses in this population. Second, our paper contributes to the small literature on professor contributions to student outcomes. Work by Carrell and West (2010) examines between-professor variation at the Air Force Academy, and finds somewhat less variation than similar estimates in primary and secondary schools. We build on this finding demonstrating that in online settings and at muchless selective institutions, the variance might even be lower. Third, our paper adds to the new and growing literature on private for-profit colleges and universities. Research on DeVry University and its peers is increasingly important to understanding American higher education broadly. The for-profit share of college enrollment and degrees is large and growing; nearly 2.4 million undergraduate students (full-time equivalent) enrolled at for-profit institutions during the 2011-12 academic year, and the sector granted 3 Xu and Jaggars (2013) use a distance-to-school instrument for Washington state community colleges and find online courses cause a 6.0 percentage point drop in completion as well as reduction in grades. Streich (2013) using an instrument of the percent of seats for a course offered online finds negative effects on the probability of passing of 8.3 percentage points. Hart, Friedman and Hill (2014) study California community colleges using individual and course fixed effects and also find negative effects of approximately the same size (8.4 percentage points) on course completion. The only indication of positive effects comes from Streich (2014) that finds some evidence of positive employment effects though largely in years immediately following initial enrollment when student may still be enrolled. Online courses may provide greater opportunities for students to work. Working at a public four-year institution, Figlio et. al. (2013) randomly assigned students in an economics course to take the course either online or in-person and found negative effects of the online version on exam scores especially for male, Hispanic and lower-achieving students. Bowen et al. (2014) also study students at four-year public institutions, comparing online and hybrid courses. They find no differences in student outcomes. 4

approximately 18 percent of all associate degrees (Deming, Goldin, and Katz 2012). Additionally, the sector serves many non-traditional students who might be a particularly important focus for policy. Thirty percent of for-profit college students have previously attended college and 25 percent having attended at least two other institutions before coming to the forprofit sector. Approximately 40 percent of all for-profit college students transfer to another college (Swail 2010). While the results suggest that students in online courses do not perform as well as other students in traditional in-person courses, our results have some limitations which we address. In particular, we examine online courses at a point in time when they are still developing. Further development and innovation could alter the results. Moreover, a full welfare analyses is not possible given our data as we cannot quantify to what extent online courses may have created new enrollments or reduced costs. The remainder of the paper is organized as follows. Section 2 reviews the data and methodology. Section 3 presents our results, and Section 4 discusses the paper including making some conclusions based on our analysis. II. Data and Methodology We address three empirical research questions in this paper: 1. How does taking a course online, instead of in a traditional in-person class, affect average student academic performance: course completion, course grades, and persistence in college? 2. How does taking a course online affect the variance of student performance? 3. How does teaching a course online affect the variance of professor performance, as measured by professors contributions to student outcomes? 5

In this section we first describe the specific students, professors, and courses on which we have data, and second describe our econometric strategies. Data and Setting We study undergraduate students and their professors at DeVry University. While DeVry began primarily as a technical school in the 1930s, today 80 percent of the University s students are seeking a bachelor s degree, and most students major in business management, technology, health, or some combination. Two-thirds of undergraduate courses occur online, the other third occur at one of over 100 physical campuses throughout the United States. In 2010 DeVry enrolled over 130,000 undergraduates, or about 5 percent of the for-profit college market, placing it among the 8-10 largest for-profit institutions that combined represent about 50 percent of the market. DeVry provided us with data linking students to their courses for all online and in-person sections of all undergraduate courses from Spring 2009 through Fall 2013. These data include information on over 280,000 students in more than 175,000 sections of 831 different courses. About one-third of the students in our data, took courses both online and in-person. Only the data from Spring 2012 through Fall 2013 contain information on professors, so part of our analysis is limited to this group of approximately 120,000 students and 5,800 professors, of which half of students took both online and in-person classes and 12 percent of professors taught both online and in-person classes. Table 1 describes the sample. Just under the half of the students are female and while they average approximately 31 years of age, there is substantial variability. Students in online courses are more likely to be female (54 percent vs. 37 percent) and older (32.5 years vs. 28.3 years). 6

The focus of this paper is on level and variance of student outcomes. The data provide a number of student outcomes including course grades, whether the student withdrew from the course, whether the student was enrolled during the following semester and how many units her or she attempted, whether the student was enrolled one year later and the number of units attempted in that semester. As shown in Table 1, the average grade was 2.8 (approximately B-), on the traditional zero (F) to four (A) scale. There is substantial variation in grades: the standard deviation is 1.3 (more than a full letter grade). Approximately 11 percent of students failed any given course, while 41 percent receive an A- or higher in the courses we observe. Just over 80 percent were still enrolled at DeVry University in the following semester or had completed their degree. Over 50 percent were enrolled one year later or had completed their degree. These outcome means are consistently higher in the in-person classes than in the online setting, even when we limit the sample to students who took both types of classes. In the next section we discuss our strategies for controlling for selection bias and for estimating causal effects of online courses. While we focus on one large university in the for-profit sector, our results are likely generalizeable beyond DeVry University. Table 1c shows average characteristics of students from various sectors of education. The first four columns use data from the National Postsecondary Student Aid Study from 2012. The final column uses the DeVry data. DeVry has a similar profile to other for-profit colleges. The for-profit colleges differ from traditional sectors, including two-year colleges, given their focus on non-traditional students and African- American students. Empirical Methodology Effects on the Mean of Student Outcomes 7

We first estimate the effect of taking a course online, instead of a traditional in-person classroom, on the average student s success in the course and persistence in college after the course. We measure student success using course grades. We measure student persistence in the subsequent semester and the semester one year later. Calculating the difference in means is straightforward, but the decision to take a course in an online section or traditional section is likely endogenous driven by unobservable information about each student s chances of success in the online versus traditional options. Our identification strategy is to use only variation in online versus traditional course taking that arises because of idiosyncratic changes over time in each local campus s in-person course offerings. Our first student outcome is the grade,, received by student in course during term. Each professor assigns traditional A-F letter grades, which we convert to the standard 0-4 point equivalents. 4 To estimate the mean difference in course grades,, between students in online and traditional classes we first specify the following statistical model:, (1) In addition to the variable of interest, the indicator, specification 1 includes several relevant controls: students prior grade point average (GPA),,, measured in all terms before term, observable student characteristics, (gender and age), course fixed effects,, for each of the 800+ courses, and a non-parametric time trend,, over the 27 terms (4.5 years, May 2009 through the end of 2013) in our data. Specification 1 also includes fixed effects for each student s home campus represented by. DeVry University operates over 100 local campuses throughout the United States. We assign each student to a home campus,, based on the physical distance between the student s home address and the local campus addresses; 4 An A is 4 points, A- is 3.7, B+ is 3.3, B is 3, etc. 8

selecting as home the campus with the minimum distance. 5 By limiting estimation to the within-campus over-time variation, we account for fixed differences between local campuses in the scope of course offerings. Fitting specification 1 by least squares will provide an unbiased estimate of under the assumption: (2) in words, students taking online classes are no more likely to score higher (lower) than students taking traditional in-person classes, conditional on included controls. While the available controls, especially prior GPA and home campus, are helpful, we remain concerned about unobserved student-level determinants of the online versus in-person decision that would bias our estimates. In response to these concerns we propose an instrumental variables strategy. Specifically, we instrument for in specification 1 with an indicator variable for whether student s home campus offered course on campus in a traditional class setting during term. Combined with the home campus fixed effects,, this limits the identifying variation to between-term variation within-campuses in the availability of an in-person option. This strategy produces a local average treatment effect (LATE) estimate, where is interpreted as the effect of taking a course online for compliers: students who would take a course in-person if it was offered at the nearest DeVry campus but otherwise take the course online. With this setting never takers deserve special consideration. A never taker is a student who would always take the given course in-person. Notably, if the course is not offered at their 5 Some students home addresses do change during their enrollment at DeVry. We calculate home campus separately for each term. 9

home campus in the term they would prefer to take the course, these never takers may wait until next term to take the course or take the course earlier than they would prefer in order to find an in-person section. This potential inter-temporal behavior motivates a modification to our instrument. Specifically, we instrument for in specification 1 with an indicator variable 1 if student s home campus offered course on campus in a traditional class setting during term any of terms, 1, or 1. 6 We also present results using an alternative instrument: the distance in miles between student s home address and the nearest local campus. The motivation for this second instrument is that the more a student must travel for an in-person course the more likely she is to take a course online. In addition to student grade points, we use the same specification and strategy to estimate for other student outcomes. First, we examine whether the effect on grade points is concentrated in a particular part of the grade distribution. We replace grade points with binary outcome variables: receiving an A- or higher, a B- or higher, a C- or higher, and simply passing the course. Second, we turn to student persistence outcomes. We estimate the difference in the probability of enrolling at DeVry in the very next semester, and the difference in the number of units a student attempts the next semester. We repeat the same analysis for enrollment during the semester one year after term. 7 Effects on the Variance of Student Outcomes 6 This instrument assumes that never taker students do not shift their course taking by more than one term before or after their preferred term. In results not presented we find that expanding the instrument to include 2 or 3 does not change the pattern of results. 7 DeVry s academic calendar divides the year into six terms, with two consecutive terms equivalent to a semester in a more traditional calendar. Following the University s approach, we define enrollment the next semester as enrollment during either term 1 or 2 or both. We define enrollment one year later as enrollment during either term 6 or 7 or both. 10

Our second objective is to estimate the effect of taking a course online, instead of a traditional in-person classroom, on the variance of student outcomes. If students were randomly assigned to take a course online or in-person, the parameter of interest could be estimated straightforwardly by 1 0. We estimate the variance effect,, using an instrumental variables approach that builds our approach for estimating the mean effect. First, we obtain the fitted residuals,, after twostage least squares estimation of specification 1. Second, we repeat the identical two-stage least squares regression used to estimation specification 1, except that the dependent variable,, is replaced with the squared residuals. This is analogous to the familiar steps in FGLS or a White test for heteroskdasticity. Unbiased estimates of require the same assumptions as the IV estimate for mean effects. As an alternative approach we estimate using a linear mixed model extension of specification 1:, 1, with the assumption ~,,. (3) The new terms and are student random effects, specific to online and traditional class settings respectively. We estimate 3 by maximum likelihood and obtain. We limit the estimation sample to students who have taken at least one course online and one course in-person. This approach allows us to estimate the difference in variances under alternative assumptions. 11

Effects on the Variance of Professor Contributions to Student Outcomes Last we estimate the effect of teaching a course online on the variance of professors contributions to student outcomes. These estimates provide important context for understanding potential mechanisms behind the other effects we estimate. These estimates are also of interest per se; the estimates provide information on how professor job performance varies, and how that variability is affected by moving a class online. We estimate the difference in the variance of professor effects using a linear mixed model approach analogous to the specification in 3. In this case we replace the student random effects with random effects for professors,, and for course sections,., 1 1, with the assumption ~,, 0 0 0 0,. (4) The section random effects,, capture shocks common to all students in student s class,, but not common to all classes taught by professor. We estimate 4 by maximum likelihood and obtain. We limit the estimation sample to professors who have taught at least one course online and one course in-person. Additionally, the estimation sample is also limited to nine terms from May 2012 forward; the data system used before May 2012 does not record the professor for each course section. III. Results 12

Effects on Average Student Outcomes For the average student, taking a course online, instead of in a traditional in-person classroom setting, reduces student learning, as measured by course grades, and lowers the probability of persistence in college. Table 2 reports our instrumental variables estimates. These estimates are consistent with the negative effects reported in other settings and papers. Nevertheless, our estimates provide new information on an important but unstudied population: students at private for-profit colleges and universities. Our most compelling identification strategies instrument for taking a course online with an indicator 1 if the student s local campus offered the course in-person either during the term the student took the course or the term just before or after (panels A and B). Using that strategy the local average treatment effect on achievement is a 0.2 grade point drop in course grade, approximately a 0.15 standard deviation decline. Put differently, complier students taking the course in-person earned a grade of 2.8 on average while their peers in online classes earned a grade of 2.6. Results presented in the appendix suggest this effect is true across the grade distribution, not simply at the top or bottom. The LATE estimates for persistence show a 3-7 percentage point drop in the probability of remaining enrolled (or graduating within) one year later. The complier students taking the course in-person had a 70 percent probability of remaining enrolled while their peers in online courses had a 63-67 percent probability. While our setting is quite different, it is useful to consider other effects in the higher education literature. For example, the literature on financial aid often finds that a $1000 in financial aid increases persistence rates by about three percentage points (Bettinger 2014) and college mentorship increases persistence rates by five percentage points (Bettinger and Baker 2013). 13

These are estimates of effects among compliers; in this case compliers are students who, for a given course, take the course in-person if it is offered in-person at the nearest DeVry campus but otherwise take the course online. Table 3 provides some descriptive information about observable characteristics of these compliers compared to all DeVry undergraduates. Compliers are older, less likely to be female, and have higher prior GPAs on average. An alternative, and more common, instrument uses the distance between each student s home address and the nearest DeVry campus (we exclude students who live more than 30 miles away). Estimates using this alternative strategy (panel C and D) also show negative effects, though of somewhat different magnitudes: a 0.09 grade point reduction and 7-13 percentage point drop in the probability of persistence. While the estimates are similar, as discussed above, students who live closer may differ in unobserved ways from those who do not so we are less confident about those estimates. Additionally, the definition of compliers under this alternative instrument also captures potentially different students. In addition to assessing the outcomes in Table 2, we also estimated the effects of online course taking on a range of other outcomes (see Appendix Tables A1-A4). Just considering the estimation strategy based on availability and including home campus fixed effects (as in table 2 panel A), we find that online students were less likely to pass the course by 4.0 percentage points and less likely to receive an A minus or above by 7.1 percentage points. After taking an online course, the students are 2.5 percentage points less likely to enroll in the following semester and, if they do, take fewer credits. Effects on the Variance of Student Grades We also find evidence that, in addition to reducing average performance, online courses also increase the variance of student academic performance. Table 4 provides estimates of from 14

the model in equation 3 as well as the IV approach. When students take courses in traditional inperson settings the between-student standard deviation of grades 0.30 grade point lower than when students take courses online. Similarly, the IV approach estimates a difference in variance that is.11 grade points higher for students in online settings, and when restricted to only students taking both online and in-person courses, the estimate is.28 grade points. This increase in variance is large but perhaps not surprising given our discussion in the introduction of differences between online and in-person practices and technologies. Taken together our estimates for mean and variance suggest some students are better off taking a course online, while other students are worse off. Additionally, in results presented in the appendix, we find that online courses also increase the variance of the number of course credits taken in future terms, conditional on enrolling in a future term. Professor Effects on Student Outcomes Differences in professor performance between online and in-person settings are one important potential mechanism generating differences in student performance. Yet while the variance of student achievement increases in online classes, we find the opposite for professor performance, as measured by professor contributions to student achievement. In online classes between-professor variation in student outcomes is significantly smaller (a standard deviation of 0.25 grade points) than between-professor variation in in-person classes (0.31 grade points). Table 5 provides these results. Among traditional in-person classes, students assigned to a topquartile professor post grades 0.40 points higher than their peers assigned to a bottom-quartile professor. By contrast, in online classes, the top- versus bottom-quartile gap is 0.35 points. These professor differences could be the result of intentional efforts to reduce the variation across courses by providing clear structure for professors interactions with students. 15

In student standard deviation terms, these variances are roughly 0.20 and 0.23. That places these estimates at the upper end of similar estimates from K-12 teachers, measured with student test scores. And both the online and in-person professor differences are noticeably larger than estimates from the more-selective, in-person Air Force Academy (Carrell and West 2010). By contrast, we do not find an effect of online teaching on professor performance as measured by student persistence in college. The difference in point estimates is negative, but not statistically significant. In the appendix we report similar professor effects estimates for other outcomes and alternative specifications, all of which consistently show smaller variance when professors teach online instead of in-person. IV. Discussion and Conclusion This study is the first that we know of to estimate the effects of online courses at broadaccess 4-year institutions of higher education or at for-profit institutions, both of which serve a substantial proportion of students. Our analyses provide evidence that students in online courses perform substantially worse than students in traditional in-person courses. Moreover, we find that the variance attributable to students increases while the variance attributable to professors decreases. The results are in line with prior studies of online education in showing that in-person courses yield better outcomes than online courses (e.g. Figlio et al 2013, Streich 2013). Our estimates of lower grades and lower persistence in online courses are similar to those in Xu and Jaggars (2013) and Hart, Friedman, and Hill (2014). Most prior studies focused on particular courses or settings where students already enrolled were assigned to online or in-person courses. In our study, students opted into online programs, and our instrument focuses on predicting the likelihood that students took specific courses online. While we find that online courses lead to worse student outcomes, our results do not give the full welfare 16

analysis. For example, the availability of online courses might have enabled some students to take courses that otherwise would not have done so. Additionally, if online courses significantly reduce costs, they could impact the overall costs that students face. We do not have the data to fully perform such an analysis. Nonetheless, our findings are an important start to such a welfare analysis. In terms of our findings on the variance of outcomes, there are several potential explanations for the increases. Figlio et al (2013) argues that procrastination in online settings was much more prevalent and led to the worsening of outcomes. While we have no measure of procrastination, we find the worsening of outcomes is especially large at the bottom of the distribution students who either withdraw or fail the course. These students generally fail to complete the course requirements and may have procrastinated in their course responsibilities. We find fewer differences in outcomes at the top of the grade distribution suggesting that online courses may not be worse for high achieving students. If indeed procrastination is driving many of the results, courses could be redesigned to improve student participation. As for the difference in the variance attributable to professors, we can also hypothesize about their potential causes. While professors in the two settings share the same syllabus, materials, and objectives, professors may follow scripts more closely in online settings where they do not need to respond to students as quickly. Online courses also share online lectures, allowing for no differences in the lecture portion of the courses. Moreover, online interactions may be more restrictive, limiting variation across professors. The decrease in variance could be both good and bad. It might eliminate extreme outcomes from bad instruction, yet it might also stifle good teaching. Overall, the results lower student performance, greater student variation, and lesser professor variation while not necessarily surprising provide evidence that online courses are not yet as effective as in-person courses. This current state, however, is clearly not a necessary end state. The greater homogeneity of online courses points to the potential to scale good courses and reduce the 17

variability in experiences that students have within the same course taught by different professors in traditional classrooms. 18

Table 1a. Student characteristics and outcomes of analytic sample All Online In person (1) (2) (3) Observations Student course session 1,947,952 1,163,563 784,389 Students 197,158 160,198 99,636 Professors 5,287 2,588 3,366 Courses 744 555 649 Sections 158,458 60,463 97,995 Student characteristics Female 0.464 0.541 0.349 Age 31.238 33.102 28.474 Online 0.597 1 0 Student prior academics Cumulative GPA 2.927 2.966 2.870 (0.931) (0.935) (0.923) Failed any course 0.279 0.271 0.291 Withdrew from any course 0.261 0.258 0.264 Student course outcomes Grade (0 4) 2.818 2.801 2.842 (1.316) (1.343) (1.275) A or higher 0.403 0.408 0.394 B or higher 0.693 0.689 0.701 C or higher 0.833 0.825 0.846 Passed 0.888 0.880 0.901 Student persistence outcomes Enrolled next semester 0.887 0.880 0.897 Credits attempted 9.756 9.110 10.677 next semester (4.745) (4.646) (4.734) Enrolled semester one year la 0.708 0.703 0.715 Credits attempted 7.555 6.719 8.750 semester one year later (5.765) (5.498) (5.925) Note. Authors' calculations. Columns 1 3 report means (standard deviations) for DeVry University undergraduate course enrollments from May 2009 to November 2013. 19

Table 1c. Student characteristics, Institution types and DeVry University (percentages) Public 4-year Private not-forprofit 4-year Public 2-year Private for-profit DeVry University (1) (2) (3) (4) (5) Age 23 or younger 69.6 71.2 49.1 31.6 28.1 24-39 24.8 19 36.4 50.7 52.5 40 or older 5.6 9.9 14.4 17.7 19.4 Female 53.9 56.6 55.7 64.1 44.5 Race/Ethnicity African-American or Black 12.8 13.4 16.4 25.6 25.5 Asian 6.9 6.9 5 2.9 4.1 Hispanic or Latino 13.8 10.1 18.6 18.5 15.5 Other, More than one 4.4 4.5 4.3 4.6 4.0 White 62.2 65.1 55.8 48.5 38.3 Columns 1-4: 2011-12 National Postsecondary Student Aid Study (NPSAS:12) as reported by NCES QuickStats. Column 5: 2012 DeVry University administrative data. Race/ethnicity imputed for 15 percent of students missing data assuming missing at random. 20

Table 2. Effect of taking a course online, instead of in person, on student achievement and persistence Dependent variable Course grade Enrolled Next Enrolled One Year (A = 4... F = 0) Semester Later (1) (2) (3) (4) (5) (6) A. Instrument = Course available in person at home campus Local average treatment effect 0.230*** 0.214*** 0.027*** 0.043*** 0.031*** 0.066*** (0.011) (0.013) (0.004) (0.005) (0.005) (0.008) First stage: coefficient on 0.277 0.248 0.264 0.233 0.264 0.233 excluded instrument (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) Sample mean (st. dev.) for dep. 2.818 0.851 0.671 (1.316) B. Instrument = Distance to home campus (excluding students more than 30 miles away) Local average treatment effect 0.034 0.098*** 0.048*** 0.072*** 0.074*** 0.128*** (0.030) (0.023) (0.007) (0.006) (0.010) (0.010) First stage: coefficient on 0.007 0.010 0.007 0.010 0.007 0.010 excluded instrument (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) Sample mean (st. dev.) for dep. 2.823 (1.304) 0.857 0.681 Course fixed effects Term fixed effects Home campus fixed effects Note. The first row of each panel reports the estimated local average treatment effect from a separate two stage least squares regression. Row two reports the coefficient on the excluded instrument from the first stage. Row three reports the sample mean (standard deviation) of the dependent variable. Dependent variables described in column headers. The specification includes one endogenous treatment variable, an indicator = 1 if the student took the course online. In panel A, the excluded instrument is a binary variable = 1 if the course was offered inperson at the student s home campus (defined as the nearest campus) in any of the current term, the previous term, or the following term. In panel B, the excluded instrument is the distance in miles from the student's home address to her home campus. All specifications also included controls for (i) prior cumulative grade point average, (ii) an indicator for student gender, and (iii) student age. The estimation sample is limited to students who have address information. For panel A there are 1,947,952 student course observations. Panel B is further limited to students whose home address is within 30 miles of a campus; there are 1,481,132 observations. Standard errors allow for clustering within sections. 21

Table 3. Observable characteristics of compliers Instrument = course available in person at home campus Mean prior Proportion Mean GPA female age (1) (2) (3) Compliers: will take the course in person 2.999 0.441 31.572 if offered, otherwise will take the course online Always takers: always take the course online 2.871 0.542 32.673 Never takers: always take the course in person 2.953 0.338 28.074 Note. Assuming no "defiers", compliers make up 28.4 percent of the sample, always takers 47.2 percent, and never takers 24.3 percent. Always and never taker means are observed. Complier means estimated assuming no "defiers". The treatment indicator is = 1 if the student took the course online. The excluded instrument is a binary variable = 1 if the course was offered in person at the student s home campus (defined as the nearest campus) in any of the current term, the previous term, or the following term. The estimation sample is limited to students who have address information: 1,947,952 student course observations. 22

Table 4. Table 4. Effect of taking a course online, instead of in person, on the variance of student achievement Dep. Var. = Course grade (A = 4 F = 0) Sample st. dev. course grade Online in person diff. P value test diff. = 0 Studentcourse observations (1) (2) (3) (4) Instrumental variables estimate 2.818 0.296 0.000 1,947,952 Random effects estimate 2.818 0.106 0.000 1,947,952 Random effects estimate (only students who 2.844 0.282 0.000 1,023,610 took courses both online and in person) Note. Column 2 reports the estimated effect of taking a course online on the standard deviation of student course grade. The depenednet variable scale is 0 4, A = 4... F = 0. Column 1 reports the sample standard deviation of course grade. The estimate in row 1 is an instrumental variables estimate obtained in two steps. Step 1 obtain the fitted residuals from the two stage least squares regression used to estimate the mean effect on course grades. Step 2 repeat the identical two stage least squres regression except that the dependent variable course grade is replaced with the squared residuals from step 1. Just as in table 2 panel A column 1, the 2SLS specification includes one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument is a binary variable = 1 if the course was offered in person at the student s home campus (defined as the nearest campus) in any of the current term, the previous term, or the following term. The specification also includes controls for (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, and (iv) separate fixed effects for course, term, and home campus. The estimation sample is limited to students who have address information. The test in column 3 allows for error clustering within sections. The estimates in rows 2 and 3 are random effects estimates from a linear mixed model. The specification includes a random student effect which is allowed to be different for online and in person classes. The fixed portion of the model includes the same regressors (i) (iv) as the 2SLS model. The test in column 3 is a likelihoodratio test where the constrained model requires the online and in person variance parameters to be equal. 23

Table 5. Variance of professor effects in online and in person classes A. Professors who teach both inperson and online Course grade (A = 4... F = 0) Enrolled Next Semester Enrolled One Year Later Course grade (A = 4... F = 0) B. All professors Enrolled Next Semester Enrolled One Year Later (1) (2) (3) (4) (5) (6) Standard deviation of professor effects In person classes 0.310 0.024 0.044 0.330 0.023 0.044 Online classes 0.294 0.018 0.030 0.270 0.020 0.019 In person = online test p value [0.244] [0.255] [0.026] [0.000] [0.354] [0.000] Student course observations 116301 71723 71723 470201 303390 303390 Note. Each column reports estimates from a separate linear mixed model. In person and online variances are estimated random effects parameters. Dependent variables are listed in the column headers. Additional fixed controls include (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, and (v) session (time) fixed effects, and (vi) home campus fixed effects. The estimation sample is limited to May 2012 to November 2013, the time period in which we can identify the specific sections of courses professors taught. 24

Table 6. Heterogeneity of effect by prior achievement Dependent variable Course grade Enrolled Next Semester Enrolled One Year Later (1) (2) (3) (4) (5) (6) A. Instrument = Course available in person at home campus Online 0.229*** 0.214*** 0.027*** 0.043*** 0.031*** 0.066*** (0.011) (0.013) (0.004) (0.005) (0.005) (0.008) Online * prior grade point avg. 0.091** 0.101*** 0.012 0.001 0.008 0.002 (centered at sample mean) (0.032) (0.027) (0.009) (0.008) (0.006) (0.006) B. Instrument = Distance to home campus (excluding students more than 30 miles away) Online 0.014 0.121*** 0.048*** 0.075*** 0.075*** 0.132*** (0.030) (0.024) (0.007) (0.006) (0.011) (0.010) Online * prior grade point avg. 0.183*** 0.195*** 0.003 0.027*** 0.013 0.030*** (centered at sample mean) (0.025) (0.019) (0.008) (0.006) (0.009) (0.007) Course fixed effects Term fixed effects Home campus fixed effects Note. Each column, within panels, reports estimates from a separate two stage least squares regression. Dependent variables described in column headers. The specification includes two endogenous treatment variables, an indicator = 1 if the student took the course online, and the interaction of that indicator and students' cumulative prior grade point average. Prior GPA has been centered at the sample mean, so the main coefficient for "online" represents the effect at the mean of prior GPA. In panel A, the excluded instruments are a binary variable = 1 if the course was offered in person at the student s home campus (defined as the nearest campus) in any of the current term, the previous term, or the following term; and the interaction of that binary variable with prior GPA. In panel B, the excluded instruments are the distance in miles from the student's home address to her home campus; and the interaction of that distance with prior GPA. All specifications also included controls for (i) prior cumulative grade point average main effect, (ii) an indicator for student gender, and (iii) student age. The estimation sample is limited to students who have address information. For panel A there are 1,947,952 student course observations. Panel B is further limited to students whose home address is within 30 miles of a campus; there are 1,481,132 observations. Standard errors allow for clustering within sections. 25

Appendix Tables. Table A1. Table A1. Effects of Online Coursetaking on all Outcomes (course, home campus and session fixed effects) Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Credits Next Semester Enrolled Next Semester Credits Next Year Enrolled Next Year Panel A. OLS Estimates Online 0.199*** 0.050*** 0.057*** 0.050*** 0.042*** 0.026*** 1.195*** 0.035*** 1.391*** 0.033*** (standard error) (0.005) (0.002) (0.002) (0.001) (0.001) (0.001) (0.032) (0.002) (0.048) (0.002) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel B. IV using Availability (current session) Online 0.201*** 0.059*** 0.057*** 0.047*** 0.038*** 0.024*** 1.127*** 0.029*** 1.516*** 0.029*** (standard error) (0.008) (0.003) (0.003) (0.002) (0.002) (0.002) (0.042) (0.003) (0.061) (0.004) First stage F statistic 236.51 236.51 236.51 236.51 236.51 229.33 183.14 164.71 209.38 164.71 First stage coefficient 0.394 0.394 0.394 0.394 0.394 0.388 0.392 0.382 0.405 0.382 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel C. IV using Availability (current, next, prior sessions) Online 0.230*** 0.071*** 0.067*** 0.052*** 0.040*** 0.019*** 1.150*** 0.027*** 1.521*** 0.031*** (standard error) (0.011) (0.005) (0.004) (0.003) (0.002) (0.002) (0.054) (0.004) (0.082) (0.005) First stage F statistic 146.15 146.15 146.15 146.15 146.15 142.12 107.17 98.23 121.14 98.23 First stage coefficient 0.277 0.277 0.277 0.277 0.277 0.271 0.273 0.264 0.282 0.264 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 26

Panel D. IV using Availability (current, next 1 or 2, prior 1 or 2 sessions) Online 0.241*** 0.074*** 0.071*** 0.054*** 0.042*** 0.017*** 1.245*** 0.033*** 1.566*** 0.040*** (standard error) (0.013) (0.006) (0.005) (0.004) (0.003) (0.003) (0.068) (0.004) (0.088) (0.006) First stage F statistic 122.15 122.15 122.15 122.15 122.15 118.55 87.39 81.23 98.65 81.23 First stage coefficient 0.394 0.394 0.394 0.394 0.394 0.388 0.392 0.382 0.405 0.382 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel E. IV using Availability (current, next 1, 2 or 3, prior 1, 2, or 3 sessions) Online 0.240*** 0.075*** 0.071*** 0.053*** 0.041*** 0.016*** 1.298*** 0.031*** 1.633*** 0.042*** (standard error) (0.014) (0.006) (0.005) (0.004) (0.003) (0.003) (0.077) (0.004) (0.099) (0.006) First stage F statistic 111.12 111.12 111.12 111.12 111.12 108.07 79.36 74.31 89.08 74.31 First stage coefficient 0.219 0.219 0.219 0.219 0.219 0.214 0.214 0.208 0.222 0.208 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel F. IV using Distance (limited to students within 30 miles) Online 0.034 0.055*** 0.013 0.015* 0.019** 0.018** 1.817*** 0.048*** 2.232*** 0.074*** (standard error) (0.030) (0.013) (0.009) (0.007) (0.006) (0.006) (0.214) (0.007) (0.220) (0.010) First stage F statistic 437.18 437.18 437.18 437.18 437.18 415.91 330.57 328.84 309.90 328.84 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1481132 1481132 1481132 1481132 1481132 1651846 1273063 1485971 1012533 1485971 Note. The top row of each panel reports a local average treatment effect from a separate regression, estimated using OLS (panel A) and standard instrumental variables methods (Panel B-E). Dependent variables are those listed in the column headers. The specifications include one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument in each of Panel B-D is a binary variable = 1 if the course was offered in-person at the student s home campus in (i) the current session, (ii) any of the current session, previous session, or following session, (iii)any of the current session, previous two sessions, or following two sessions, and (iv) any of the current session, previous three sessions, or following three sessions, respectively. Panel E uses distance from the student s home to the nearest DeVry campus as the instrument. Other included regressors include: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, (v) session (time) fixed effects, and (vi) home campus fixed effects. The estimation sample is limited to those students who have address information. Standard errors are clustered at the section level. 27

Table A2. Table A2. Effects of Online Coursetaking on all Outcomes (course and session fixed effects) Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Credits Next Semester Enrolled Next Semester Credits Next Year Enrolled Next Year Panel A. OLS Estimates (course and session fixed effects) Online 0.198*** 0.047*** 0.056*** 0.051*** 0.044*** 0.026*** 1.256*** 0.039*** 1.416*** 0.043*** (standard error) (0.005) (0.002) (0.002) (0.002) (0.001) (0.001) (0.037) (0.002) (0.056) (0.003) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel B. IV using Availability (current session) Online 0.192*** 0.046*** 0.050*** 0.051*** 0.046*** 0.016*** 1.307*** 0.042*** 1.510*** 0.057*** (standard error) (0.010) (0.004) (0.004) (0.002) (0.002) (0.002) (0.076) (0.004) (0.084) (0.006) First stage F statistic 438.16 438.16 438.16 438.16 438.16 398.65 289.92 292.85 295.68 292.85 First stage coefficient 0.349 0.349 0.349 0.349 0.349 0.342 0.339 0.334 0.348 0.334 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel C. IV using Availability (current, next, prior sessions) Online 0.214*** 0.051*** 0.055*** 0.057*** 0.051*** 0.007* 1.331*** 0.043*** 1.443*** 0.066*** (standard error) (0.013) (0.006) (0.005) (0.003) (0.003) (0.003) (0.099) (0.005) (0.122) (0.008) First stage F statistic 308.94 308.94 308.94 308.94 308.94 284.69 207.66 202.91 215.17 202.91 First stage coefficient 0.248 0.248 0.248 0.248 0.248 0.242 0.238 0.233 0.246 0.233 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 28

Panel D. IV using Availability (current, next 1 or 2, prior 1 or 2 sessions) Online 0.221*** 0.051*** 0.056*** 0.059*** 0.055*** 0.003 1.401*** 0.050*** 1.445*** 0.075*** (standard error) (0.014) (0.006) (0.005) (0.004) (0.003) (0.003) (0.121) (0.006) (0.147) (0.009) First stage F statistic 255.88 255.88 255.88 255.88 255.88 237.46 172.46 168.64 179.15 168.64 First stage coefficient 0.215 0.215 0.215 0.215 0.215 0.209 0.204 0.200 0.212 0.200 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel E. IV using Availability (current, next 1, 2 or 3, prior 1, 2, or 3 sessions) Online 0.219*** 0.051*** 0.055*** 0.059*** 0.055*** 0.001 1.435*** 0.049*** 1.477*** 0.077*** (standard error) (0.015) (0.006) (0.005) (0.004) (0.004) (0.004) (0.132) (0.006) (0.164) (0.009) First stage F statistic 228.02 228.02 228.02 228.02 228.02 213.10 155.36 151.87 161.38 151.87 First stage coefficient 0.199 0.199 0.199 0.199 0.199 0.194 0.190 0.186 0.196 0.186 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel F. IV using Distance (limited to students within 30 miles) Online 0.098*** 0.003 0.022** 0.038*** 0.035*** 0.003 1.631*** 0.072*** 1.615*** 0.128*** (standard error) (0.023) (0.011) (0.007) (0.005) (0.004) (0.004) (0.159) (0.006) (0.167) (0.010) First stage F statistic 309.79 309.79 309.79 309.79 309.79 316.22 226.40 257.44 196.08 257.44 First stage coefficient 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.009 0.010 N (student by course) 1481132 1481132 1481132 1481132 1481132 1651846 1273063 1485971 1012533 1485971 Note. The top row of each panel reports a local average treatment effect from a separate regression, estimated using OLS (panel A) and standard instrumental variables methods (Panel B-E). Dependent variables are those listed in the column headers. The specifications include one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument in each of Panel B-D is a binary variable = 1 if the course was offered in-person at the student s home campus in (i) the current session, (ii) any of the current session, previous session, or following session, (iii)any of the current session, previous two sessions, or following two sessions, and (iv) any of the current session, previous three sessions, or following three sessions, respectively. Panel E uses distance from the student s home to the nearest DeVry campus as the instrument. Other included regressors include: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, and (v) session (time) fixed effects. The estimation sample is limited to those students who have address information. Standard errors are clustered at the section level. 29

Table A3. Table A3. Effects of Online Coursetaking using Various Distance thresholds (course, home campus, and session fixed effects) Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Credits Next Semester Enrolled Next Semester Credits Next Year Panel A. Limited to students within 10 miles Online 0.167* 0.089** 0.066* 0.011 0.002 0.035* 1.987*** 0.009 1.273* 0.004 (standard error) (0.075) (0.027) (0.027) (0.024) (0.020) (0.017) (0.381) (0.020) (0.528) (0.032) Enrolled Next Year First stage F statistic 276.65 276.65 276.65 276.65 276.65 294.76 216.71 245.58 205.05 245.58 First stage coefficient 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 N (student by course) 963178 963178 963178 963178 963178 1077158 831945 969651 662680 969651 Panel B. Limited to Students within 20 miles Online 0.070 0.038* 0.021 0.007 0.003 0.035*** 1.401*** 0.021* 2.015*** 0.031* (standard error) (0.043) (0.017) (0.015) (0.011) (0.009) (0.009) (0.204) (0.010) (0.266) (0.014) First stage F statistic 564.84 564.84 564.84 564.84 564.84 511.12 415.22 429.00 402.18 429.00 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1361105 1361105 1361105 1361105 1361105 1519013 1171590 1366770 932257 1366770 Panel C. Limited to Students within 30 miles Online 0.034 0.055*** 0.013 0.015* 0.019** 0.018** 1.817*** 0.048*** 2.232*** 0.074*** (standard error) (0.030) (0.013) (0.009) (0.007) (0.006) (0.006) (0.214) (0.007) (0.220) (0.010) First stage F statistic 437.18 437.18 437.18 437.18 437.18 415.91 330.57 328.84 309.90 328.84 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1481132 1481132 1481132 1481132 1481132 1651846 1273063 1485971 1012533 1485971 Panel D. Limited to Students within 40 miles Online 0.006 0.035** 0.000 0.014** 0.015*** 0.015*** 1.559*** 0.039*** 2.078*** 0.058*** (standard error) (0.024) (0.011) (0.008) (0.006) (0.004) (0.005) (0.214) (0.005) (0.196) (0.009) First stage F statistic 310.73 310.73 310.73 310.73 310.73 308.00 241.86 242.93 230.02 242.93 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1559818 1559818 1559818 1559818 1559818 1738573 1339282 1563754 1064858 1563754 30

Panel E. Limited to Students within 50 miles Online 0.030 0.022* 0.009 0.021*** 0.022*** 0.012** 1.630*** 0.039*** 2.012*** 0.074*** (standard error) (0.021) (0.010) (0.008) (0.004) (0.003) (0.004) (0.226) (0.005) (0.183) (0.010) First stage F statistic 196.32 196.32 196.32 196.32 196.32 197.79 155.54 156.71 149.10 156.71 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1614168 1614168 1614168 1614168 1614168 1798347 1384925 1617770 1100272 1617770 Panel F. Limited to Students within 60 miles Online 0.056** 0.014 0.018** 0.026*** 0.026*** 0.010* 1.722*** 0.035*** 2.134*** 0.067*** (standard error) (0.019) (0.009) (0.006) (0.004) (0.003) (0.004) (0.223) (0.005) (0.175) (0.011) First stage F statistic 133.19 133.19 133.19 133.19 133.19 134.81 106.79 108.16 103.35 108.16 First stage coefficient 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 N (student by course) 1660271 1660271 1660271 1660271 1660271 1849329 1423605 1663970 1130163 1663970 Panel G. Limited to Students within 70 miles Online 0.060** 0.008 0.017** 0.026*** 0.026*** 0.005 1.887*** 0.039*** 2.201*** 0.079*** (standard error) (0.019) (0.009) (0.006) (0.004) (0.003) (0.003) (0.238) (0.004) (0.184) (0.012) First stage F statistic 108.11 108.11 108.11 108.11 108.11 109.69 88.05 88.67 85.65 88.67 First stage coefficient 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 0.006 N (student by course) 1691204 1691204 1691204 1691204 1691204 1883535 1449427 1695118 1149734 1695118 Panel H. All Students Online 0.135*** 0.007 0.038*** 0.046*** 0.043*** 0.000 2.710*** 0.086*** 3.029*** 0.130*** (standard error) (0.019) (0.008) (0.007) (0.005) (0.004) (0.004) (0.340) (0.011) (0.291) (0.019) First stage F statistic 51.74 51.74 51.74 51.74 51.74 52.68 42.64 44.32 40.89 44.32 First stage coefficient 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.000 0.001 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Note. The top row of each panel reports a local average treatment effect from a separate regression, estimated using standard instrumental variables methods. Dependent variables are those listed in the column headers. The specifications include one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument in all panels is the distance from a student s home to the closest DeVry campus. Other included regressors are: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, (v) session (time) fixed effects, and (vi) home campus fixed effects. The estimation sample is limited to those students who have address information, and each Panel is limited to students within the listed distance threshold. Standard errors are clustered at the section level. 31

Table A4. Table A4. Effects of Online Coursetaking using Various Distance thresholds (course and session fixed effects) Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Credits Next Semester Enrolled Next Semester Credits Next Year Enrolled Next Year Panel A. Limited to students within 10 miles Online 0.290*** 0.138*** 0.091*** 0.039* 0.021 0.002 2.330*** 0.039** 1.296*** 0.086*** (standard error) (0.054) (0.023) (0.019) (0.017) (0.013) (0.010) (0.334) (0.013) (0.392) (0.023) First stage F statistic 123.50 123.50 123.50 123.50 123.50 126.55 97.97 103.50 99.70 103.50 First stage coefficient 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 N (student by course) 963178 963178 963178 963178 963178 1077158 831945 969651 662680 969651 Panel B. Limited to Students within 20 miles Online 0.113*** 0.037* 0.027** 0.027*** 0.021*** 0.003 1.502*** 0.063*** 1.410*** 0.120*** (standard error) (0.031) (0.015) (0.010) (0.008) (0.006) (0.006) (0.154) (0.008) (0.188) (0.013) First stage F statistic 148.55 148.55 148.55 148.55 148.55 151.38 110.54 123.73 98.83 123.73 First stage coefficient 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.010 N (student by course) 1361105 1361105 1361105 1361105 1361105 1519013 1171590 1366770 932257 1366770 Panel C. Limited to Students within 30 miles Online 0.098*** 0.003 0.022** 0.038*** 0.035*** 0.003 1.631*** 0.072*** 1.615*** 0.128*** (standard error) (0.023) (0.011) (0.007) (0.005) (0.004) (0.004) (0.159) (0.006) (0.167) (0.010) First stage F statistic 309.79 309.79 309.79 309.79 309.79 316.22 226.40 257.44 196.08 257.44 First stage coefficient 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.010 0.009 0.010 N (student by course) 1481132 1481132 1481132 1481132 1481132 1651846 1273063 1485971 1012533 1485971 Panel D. Limited to Students within 40 miles Online 0.097*** 0.004 0.025*** 0.035*** 0.032*** 0.000 1.431*** 0.058*** 1.584*** 0.105*** (standard error) (0.018) (0.009) (0.006) (0.004) (0.003) (0.003) (0.171) (0.005) (0.145) (0.008) First stage F statistic 417.05 417.05 417.05 417.05 417.05 423.85 314.88 343.71 269.13 343.71 First stage coefficient 0.009 0.009 0.009 0.009 0.009 0.009 0.009 0.009 0.009 0.009 N (student by course) 1559818 1559818 1559818 1559818 1559818 1738573 1339282 1563754 1064858 1563754 32

Panel E. Limited to Students within 50 miles Online 0.104*** 0.003 0.027*** 0.037*** 0.036*** 0.000 1.510*** 0.055*** 1.644*** 0.107*** (standard error) (0.016) (0.008) (0.006) (0.004) (0.003) (0.003) (0.187) (0.005) (0.142) (0.008) First stage F statistic 346.08 346.08 346.08 346.08 346.08 356.83 277.01 289.28 241.02 289.28 First stage coefficient 0.009 0.009 0.009 0.009 0.009 0.008 0.008 0.009 0.008 0.009 N (student by course) 1614168 1614168 1614168 1614168 1614168 1798347 1384925 1617770 1100272 1617770 Panel F. Limited to Students within 60 miles Online 0.132*** 0.009 0.036*** 0.044*** 0.043*** 0.001 1.599*** 0.051*** 1.796*** 0.100*** (standard error) (0.013) (0.007) (0.004) (0.003) (0.003) (0.003) (0.185) (0.005) (0.141) (0.008) First stage F statistic 254.71 254.71 254.71 254.71 254.71 262.84 209.05 214.02 188.92 214.02 First stage coefficient 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 N (student by course) 1660271 1660271 1660271 1660271 1660271 1849329 1423605 1663970 1130163 1663970 Panel G. Limited to Students within 70 miles Online 0.134*** 0.011 0.035*** 0.045*** 0.043*** 0.003 1.764*** 0.055*** 1.887*** 0.109*** (standard error) (0.012) (0.006) (0.004) (0.003) (0.003) (0.002) (0.192) (0.004) (0.139) (0.009) First stage F statistic 212.19 212.19 212.19 212.19 212.19 220.11 178.50 179.88 164.64 179.88 First stage coefficient 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 0.007 N (student by course) 1691204 1691204 1691204 1691204 1691204 1883535 1449427 1695118 1149734 1695118 Panel H. All Students Online 0.124*** 0.005 0.033*** 0.045*** 0.041*** 0.014*** 2.427*** 0.091*** 2.557*** 0.137*** (standard error) (0.014) (0.007) (0.005) (0.004) (0.003) (0.003) (0.222) (0.008) (0.193) (0.014) First stage F statistic 60.21 60.21 60.21 60.21 60.21 62.28 50.21 51.95 47.58 51.95 First stage coefficient 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Note. The top row of each panel reports a local average treatment effect from a separate regression, estimated using standard instrumental variables methods. Dependent variables are those listed in the column headers. The specifications include one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument in all panels is the distance from a student s home to the closest DeVry campus. Other included regressors are: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, and (v) session (time) fixed effects. The estimation sample is limited to those students who have address information, and each Panel is limited to students within the listed distance threshold. Standard errors are clustered at the section level. 33

Table A5. Table A5. Variance in Student Outcomes for all Continuous Outcomes Grade Credits Next Semester Credits Next Year Panel A. IV model (all students) S.D. Difference Online vs. Brick 0.296 0.723 0.401 Standard error [0.000] [0.328] [0.846] N (student by course) 1947952 1661933 1310420 Panel B. Random Coefficients Model (all students) S.D. Difference Online vs. Brick 0.106 0.167 0.062 Likelihood Ratio Test: Difference=0 (p value) [0.000] [0.000] [0.000] N (student by course) 1947952 1661933 1310420 Panel C. Random Coefficients Model (only students taking both) S.D. Difference Online vs. Brick 0.282 0.085 0.341 Likelihood Ratio Test: Difference=0 (p value) [0.000] [0.000] [0.000] N (student by course) 1023610 913559 752289 Note. Estimated difference between online and in-person variance in student outcomes is estimated from (i) an instrumental variables model, and (ii) a linear mixed model. For the instrumental variables model in Panel A, the dependent variable is the squared residuals from an IV model with the following specifications. Dependent variables are those listed in the column headers. The specification includes one endogenous treatment variable, an indicator = 1 if the student took the course online. The excluded instrument is a binary variable = 1 if the course was offered in-person at the student s home campus in any of the current term, the previous term, or the following term. Other included regressors are: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) session fixed effects, (v) course fixed effects, and (vi) home campus fixed effects. Standard errors are clustered at the section level. In panels B and C, the linear mixed model contains random effects for students that vary by online and in person status. The standard deviation of online and in-person effects is estimated directly from the model, and the p-value for the difference in effects is estimated using a likelihood ratio test of the null model in which the standard deviation for online and in-person effects are the same. Included regressors are the same as those in the IV model. For all panels, the estimation sample is limited to those students who have address information; in panel C, the sample is further limited to students who took both online and in person courses. 34

Table A6. Table A6. Variance in Professor Effects for all Outcomes Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Credits Next Semester Enrolled Next Semester Credits Next Year Enrolled Next Year Panel A. Professors teaching both in person and online Brick (standard deviation) 0.310 0.127 0.102 0.062 0.041 0.025 0.588 0.024 0.563 0.044 Online (standard deviation) 0.294 0.111 0.086 0.064 0.049 0.025 0.371 0.018 0.385 0.030 Likelihood Ratio Test: Difference=0 (p value) 0.244 0.002 0.001 0.562 0.006 0.988 0.000 0.255 0.021 0.026 N (student by course) 116301 116301 116301 116301 116301 127141 61968 71723 49985 71723 Panel B. All Professors Brick (standard deviation) 0.330 0.131 0.106 0.070 0.049 0.028 0.589 0.023 0.665 0.044 Online (standard deviation) 0.270 0.105 0.082 0.055 0.039 0.025 0.309 0.020 0.346 0.019 Likelihood Ratio Test: Difference=0 (p value) 0.000 0.000 0.000 0.000 0.000 0.011 0.000 0.354 0.000 0.000 N (student by course) 470201 470201 470201 470201 470201 519350 256391 303390 203906 303390 Note. Professor effect estimates are estimated in a random effects model with professor and section random effects that vary by online and in-person status. The standard deviation and correlation of online and in-person professor effects are estimated directly from the model, and the p-value for the difference in effects is estimated using a likelihood ratio test of the null model in which the standard deviation for online and in-person effects are the same. Dependent variables are those listed in the column headers. Additional controls include (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, and (v) session (time) fixed effects, and (vi) home campus fixed effects. The estimation sample is limited to May 2012 to November 2013, the time period in which we can identify the specific sections of courses professors taught. We also limit the estimation sample to courses with professors who taught both online and in-person courses during this time period. 35

Table A7. Table A7. Heterogeneity of Effects of Online Coursetaking by Prior GPA on all Outcomes Grade Grade: At least A Grade: At least B Grade: At least C Passed Withdrew Note. Each panel reports local average treatment effects from a separate regression, estimated using OLS (panels A and B) and standard instrumental variables methods (Panel C and D). Dependent variables are those listed in the column headers. The specifications include two endogenous treatment variables, an indicator = 1 if the student took the course online and the indicator interacted with prior cumulative GPA. The excluded instruments are a binary variable = 1 if the course was offered in-person at the student s home campus in any of the current session, previous session, or following session, and this instrument interacted with prior cumulative GPA. Other included regressors include: (i) prior cumulative grade point average, (ii) an indicator for student gender, (iii) student age, (iv) course fixed effects, and (v) session (time) fixed effects. Panels A and C additionally include home campus fixed effects. The estimation sample is limited to those students who have address information. Standard errors are clustered at the section level. 36 Credits Next Semester Enrolled Next Semester Credits Next Year Enrolled Next Year Panel A. OLS Estimates (course and session fixed effects) Online 0.196*** 0.047*** 0.055*** 0.050*** 0.043*** 0.025*** 1.262*** 0.039*** 1.421*** 0.043*** (standard error) (0.005) (0.002) (0.002) (0.001) (0.001) (0.001) (0.037) (0.002) (0.056) (0.003) Online*Prior GPA 0.112*** 0.010 0.030*** 0.036*** 0.036*** 0.023*** 0.296*** 0.017*** 0.350*** 0.002 (standard error) (0.010) (0.005) (0.004) (0.002) (0.002) (0.001) (0.025) (0.003) (0.032) (0.002) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel B. OLS Estimates (course, session, and home campus fixed effects) Online 0.197*** 0.049*** 0.056*** 0.049*** 0.042*** 0.026*** 1.200*** 0.035*** 1.395*** 0.033*** (standard error) (0.005) (0.002) (0.002) (0.001) (0.001) (0.001) (0.033) (0.002) (0.048) (0.002) Online*Prior GPA 0.107*** 0.008 0.030*** 0.035*** 0.034*** 0.023*** 0.281*** 0.014*** 0.323*** 0.001 (standard error) (0.010) (0.005) (0.004) (0.002) (0.002) (0.001) (0.024) (0.003) (0.032) (0.002) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel C. IV using Availability (course and session fixed effects) Online 0.214*** 0.051*** 0.055*** 0.057*** 0.052*** 0.007* 1.331*** 0.043*** 1.441*** 0.066*** (standard error) (0.013) (0.006) (0.005) (0.003) (0.003) (0.003) (0.100) (0.005) (0.121) (0.008) Online*Prior GPA 0.101*** 0.012 0.022 0.045*** 0.046*** 0.003 0.580*** 0.001 0.600*** 0.002 (standard error) (0.027) (0.014) (0.012) (0.005) (0.005) (0.005) (0.055) (0.008) (0.066) (0.006) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752 Panel C. IV using Availability (course, session, and home campus fixed effects) Online 0.229*** 0.071*** 0.066*** 0.051*** 0.040*** 0.019*** 1.153*** 0.027*** 1.522*** 0.031*** (standard error) (0.011) (0.005) (0.004) (0.003) (0.002) (0.002) (0.055) (0.004) (0.082) (0.005) Online*Prior GPA 0.091** 0.011 0.034* 0.039*** 0.030*** 0.003 0.402*** 0.012 0.389*** 0.008 (standard error) (0.032) (0.016) (0.013) (0.006) (0.005) (0.005) (0.062) (0.009) (0.078) (0.006) N (student by course) 1947952 1947952 1947952 1947952 1947952 2167719 1661933 1953752 1310420 1953752