Perspectives on Economic Education Research 9(1) 86-99 Journal homepage: www.isu.edu/peer/ Determinants of Success in Economics Principles: Online vs. Face-to-Face Courses 1 David Switzer a,2, Kenneth Rebeck b a Department of Economics, St. Cloud State University, Stewart Hall 386, St. Cloud, MN 56301, United States b Department of Economics, St. Cloud State University, United States Abstract With an increase in the number of online courses being offered by economics departments across the United States, it is important for both faculty and students to be aware of the factors that determine success in face-to-face vs. online courses. Students may enroll in online courses for which they are not prepared, and would perform better in a face-to-face course. This study examines 126 students taking the same course with the same professor, same assignments and exams, with half in an online course and half in a face-to-face course. A variety of demographic and student characteristics are included in the analysis, including survey responses in regards to study habits. Comparison of characteristics across online and face-to-face students indicates significant differences between the two groups. This research indicates that students who selfdescribe as being less able to push through difficult material and being more likely to procrastinate are more likely to enroll in the online course, as are students who work full time and commute from further away. In regards to performance, we found no disadvantage for the typical student to taking a principles of macroeconomics course online when accounting for GPA and other factors and controlling for instructor and course characteristics. Evidence indicates that the differences in study habits might play an important role influencing achievement in online courses. We conclude with suggestions for both students in online courses and faculty teaching them. Key Words: online, face-to-face, study habits, student performance JEL Codes: A20, A22 1 The authors are grateful for helpful comments provided by an anonymous referee. 2 Corresponding author.
1. Introduction While online education has been around for decades, its use became more widespread at the turn of the millennium. Since 2002, the number of courses taken online by students has more than tripled (Allen and Seaman, 2011). Prompted by potential to cut costs, increase enrollment, and satisfy student demand for more flexibility in how and when they learn, online education seems like a great solution to university administrators and departments. Even more, initial analysis of the effectiveness of online education was positive: learning outcomes in online courses appeared to be better than, or at least not worse than, those in traditional face-to-face (F2F) courses. But much has changed in the last decade. Research increasingly indicates that learning outcomes in online courses are now worse than those of F2F courses. Whether this is due to a failure to maintain quality standards as online education has spread, a change in the pool of students who now take online courses, or both, the fact is that we are at a crossroads when it comes to online education. If online education is to be the future of education, its quality must be on par with what it is replacing. Only by investigating the causes of this disparity can we hope to address it. This research investigates students enrolled in Principles of Macroeconomics in Fall 2010 and Spring 2011 with the same instructor, half of whom were in an online section and half of whom were in a face-to-face section. We first explore issues determining whether a student enrolls in the F2F section or the online section. Our research adds to the literature by specifically addressing different study habits that students have, to see how face-to-face and online students differ. We find that students that give themselves low scores on measures relating to their ability to complete tasks, not be discouraged, and avoid procrastination, are more likely to enroll in online courses. Unfortunately, online courses are often those in which these poor study habits are most likely to be detrimental to one s performance. This may be why previous research has consistently found that dropout rates are significantly higher in online courses than in F2F courses. We then examine the factors that determine students performance in the course, as measured by their final grade on three exams and online homework and quizzes. We find that performance by online students is roughly the same as the performance by the F2F students after controlling for GPA and other characteristics. We find that a student s ability to concentrate and stay on task is more important in online courses than in F2F courses, which is an intuitive result. Combining our results from the two different sections of the paper, we conclude the following: students with poor study habits are more likely to take online courses, and it is these study habits (either directly observed or observed in the form of a lower GPA) that cause them to perform worse than their peers in the F2F course. After a brief overview of the relevant literature, we detail the student population and the data collected. We then present our analysis of the determinants of online enrollment and then course performance. Finally, we conclude with some additional observations and suggestions not just for future researchers but for economics faculty or departments who are teaching or considering teaching online principles courses. 2. Literature review If there is one thing about which the literature on online education agrees, it is that the perceived or estimated effectiveness of online education depends on the study. For example, the Babson Survey Research Group found that one third of chief academic officers at institutions of higher education believe that online education is inferior to F2F instruction (Allen and Seaman, 2011). However, it also found that faculty those delivering instruction - on average have less confidence in the effectiveness of online instruction, with two thirds believing online instruction is inferior (Allen et al., 87
2012). At the same time, students are increasingly enrolling in online programs and taking online courses. Online instruction is attractive to traditional college students looking for a more flexible schedule, and is instrumental in reaching nontraditional students who may otherwise be unable to attend classes. Online education experienced a fair amount of research in the late 1990s and throughout the first decade of the 2000s. As Glanz (2012) reports, only a small amount of research was specific to the instruction of economics, but the results seemed to conform to the pattern found in online education research overall: early work indicated that outcomes in online education were positive and comparable to F2F courses, while more recent work indicates that online education s outcomes are inferior to F2F courses. Trawick, Lile and Howsen (2010) find that students perform worse in online courses than in F2F, and caution that conversion of F2F courses to online courses may cause some students to receive worse grades than they would have otherwise received. Harmon and Lambrinos (2006) find online performance to be better than or equal to F2F performance, but notes that online students also tended to be significantly older. In its original application, online education was designed for nontraditional students, so the average age of the online student was in fact older, and these students were likely more mature. The more recent finding of decreased effectiveness of online courses relative to F2F courses may be because online education is now being applied more broadly to university education, and more freshmen and sophomore students take these courses without the discipline and maturity necessary to complete the work. Some younger students may not be aware of the difficulty they have in online courses until they take one. In fact, even in studies that showed no difference in performance between online and F2F students, like Euzent et al. (2011), it is noted that the withdrawal rate is higher in online courses. Much of the early research in online economic education suffered from one of two flaws: the inability to adequately control for different instructors across online and F2F courses when comparing the outcomes of each, and the use of online assignments in a F2F course as a proxy for online instruction instead of actual online course enrollment. Some studies are still conducted at the department level and compare the performance of all students who took principles of economics online to those who took it in a F2F format. While some of these use common exams or identical TUCE questions to compare performance, they often involve different professors teaching these courses. For example, Gratton- Lavoie and Stanley (2009) found that students in online courses perform better and attributed this to these students being older and having higher GPAs. However, the professors were not consistent across delivery format, so it may also be that the instructors teaching online courses were those that were more dedicated to student performance. Since the quality of instruction is important to education, it is difficult to make a truly apples-to-apples comparison when the professors teaching the material differ. In regards to the second flaw, some studies analyze the use of web-based components and conclude that online instruction is beneficial. For example, both Schuhmann and McGoldrick (2001) and Coates and Humphreys (2001) investigate the use of some web-based components that supplement lectures. Both studies found that students that were more actively involved in the web-based practice quizzes and discussion questions received a better grade. However, these studies are not examining the differences between a standard F2F course and a completely online course. It should come as no surprise that students who do more practice questions receive better final grades whether those practice questions occur online or on paper may not be relevant. This paper adds to the existing literature by addressing the factors that influence a student s decision to enroll in an online course, and whether these factors have a significant effect on the student s performance in the course. In doing so, we hope to provide students, professors and academic advisors alike with a set of criteria that can be used to help guide students into the appropriate format for their economic education, where their chance of success is highest. 88
3. Data Data consists of grades and attributes of students enrolled in Principles of Macroeconomics (Econ 205) with Dr. David Switzer at St. Cloud State University in either Fall 2010 or Spring 2011 who completed the course and completed a survey. Students knew whether the course was lecture or F2F before enrolling, and enrolled in the course of their choice, so this was not a randomized experiment. In Fall 2010, there were 63 students who completed the survey and course in the F2F format and 29 students who completed the survey and course in the online format. An additional 34 students completed the survey and course in the online format in Spring 2011. Thus, there are 63 F2F and 63 online students included in this study. It should be noted that this analysis is only for students who completed the course. The survey of study attributes was created at the end of Fall 2010 and administered to students still in the course. Thus, the attributes of students who failed to complete the course are not observed which is an important extension for future work in determining which students are more likely to drop out of F2F vs. online courses. A brief discussion of preliminary findings about these students is included at the end of the Results section of the paper. Students were given the same three exams at the same times during the semester, and the same weekly homework and quiz assignments using the online homework site Aplia. Lectures given in the F2F course were from the same PowerPoint presentations used to create the lecture-capture videos watched by the online students, with an effort to make the lectures as identical as possible for both groups. The exams and Aplia assignments account for 888 out of 1,000 course points. The remainder differed slightly, with F2F students receiving points for completing in-class exercises and online students receiving points for weekly discussion participation. Scores on these differing assignments are excluded from this analysis, and the final grade analyzed here includes only exams and Aplia assignments. 4. Survey Data The survey was administered online using the class s course management system, Desire2Learn. A printed copy of the survey is included in the Appendix. The survey includes 21 questions about the student s own study habits, where students indicate whether they do certain activities or have specific attitudes almost never, rarely, sometimes, frequently, or almost always. Almost never is assigned a value of 1, while Almost always is assigned a value of 5. 3 The questions are worded such that higher scores on each question indicate better study habits. The 21 questions consist of three questions each in seven different categories: attitude, concentration, testing, motivation, note taking, reading, and time management. Additional survey questions pertained to demographic characteristics that are not available through the university s registration system. This research includes responses such as the student s employment (do not work, work part time, work full time), average commute time to campus and age. Information on student GPAs was obtained through the university and matched to each student in the course. The GPA used is the GPA at the start of the semester in which the course was taken. Since some students were freshmen or transfer students and had no university GPA on file, these students are not included in any regression that includes GPA data. There are 26 students for which no GPA data was available, reducing the sample size of these regressions to 100 students. 4 3 This Study Skills Inventory survey has been used by various Academic Support offices of universities such as Louisiana Tech University and St. Catherine College. 4 Analysis of these 26 students reveals they are similar to the other 100 students in terms of class performance. For example, the Exam Total average for this group is 72.67%, while the average for the other 100 students is 72.37%. 89
Table 1 below provides a list of the variables used, along with a brief description of each variable and its range. Table 1: Summary of Variables GPA GPA for course work at St. Cloud State University at the beginning of the semester in which the student took Econ 205 ONLINE Dummy variable equal to 1 if the student is in an online section of the course, 0 if the student is in lecture FEMALE Dummy variable equal to 1 if the student is female, 0 if the student is male AGE Student's self-reported age at time of completion of the survey ATTITUDE The sum of all three questions on ATTITUDE. The minimum value possible is 3 and the maximum is 15. The sum of all three questions on CONCENTRATION. The minimum CONCENTRATION value possible is 3 and the maximum is 15. STUDYHABITS The sum of all 21 questions on study habits. The minimum value possible is 21 and the maximum is 105. HOURSSTUDY Self-reported average hours per week spent studying for the course. COMMUTE Student's self-reported commute time between their residence and SCSU WORKFULL Dummy variable equal to 1 if the student self-reported working full time, 0 if working part time or not working EXAM1 Score received on exam 1 (20% of total course grade), adjusted so that the maximum score is 100 EXAM2 Score received on exam 2 (20% of total course grade), adjusted so that the maximum score is 100 EXAM3 Score received on exam 3 (20% of total course grade), adjusted so that the maximum score is 100 APLIA Score received on Aplia quiz and homework assignments (28.8% of total course grade), adjusted so that the maximum score is 100. EXAMTOTAL Total score received on all three exams, adjusted so that the maximum score is 100 Total score received on all three exams and all Aplia assignments FINALGRADE (88.8% of total course grade), adjusted so that the maximum score is 100. Table 2 below shows sample means and standard deviations for all of the variables, separated between F2F students and online students. For each variable, the mean is the number on top and the standard deviation is the number on bottom. Asterisks indicate whether the means (continuous variables) or proportions (categorical variables) of the two samples are significantly different at the 5% and 1% levels. Again, note that for the GPA numbers, the sample contains 40 F2F students and 60 online students. All other variables contain the full sample of 126 students, 63 in each format. 90
Table 2: Variable Comparisons Between F2F and Online Students Variable Lecture Online t-stat Demographic GPA 2.918 2.701 1.935 0.529 0.632 FEMALE 0.365 0.429-0.724 0.485 0.499 AGE 20.48 22.21-2.313 ** 3.252 4.968 Survey on Study Habits ATTITUDE 12.25 11.73 1.661 1.866 1.668 CONCENTRATION 10.29 10.84-1.418 1.971 2.404 STUDYHABITS 77.63 77.22 0.236 9.290 10.321 HOURSSTUDY 3.415 4.845-2.452 ** 2.558 3.846 COMMUTE 11.06 14.23-3.743 ** 14.23 29.16 WORKFULL 0.063 0.246-4.400 ** 0.246 0.485 Course Performance EXAM1 73.496 71.387 0.815 15.006 13.904 EXAM2 74.464 68.849 2.098 * 15.226 14.808 EXAM3 74.722 70.595 1.545 14.436 15.533 APLIA 76.433 74.381 0.935 11.809 12.813 EXAMTOTAL 74.228 70.644 1.537 13.141 12.921 FINALGRADE 74.943 72.064 1.376 11.682 11.715 * indicates means are significantly different at the 5% level. ** indicates the 1% level It appears that, compared to the students in F2F, students taking the course online work full time much more often, have slightly lower GPAs (significant at the 10% level), study more, and are older. With respect to GPAs, this is in direct contrast to some studies, like Gratton-Lavoie and Stanley (2009), who found that online students had higher GPAs. 5. Results The goal of this research is to answer two questions: what factors affect a student s choice to take the course online instead of in F2F format, and do these factors in turn have any effect on a student s performance in the course format they chose? To answer the first, we use ONLINE as the 91
dependent variable and perform probit analysis. Table 3 below shows the results from the two regression estimations. Model 1a includes GPA, FEMALE, and AGE for demographic variables. It also includes ATTITUDE and CONCENTRATION as two measures of study habits. Finally, it includes COMMUTE and WORKFULL, since students who live far from campus and work full time are likely to be more attracted to online courses. Model 1b is identical to Model 1a except that it excludes GPA in an attempt to include more students in the analysis. Coefficient values in bold indicate significance at the 5% level. Table 3: Models 1a and 1b Results Dependent Variable: ONLINE Model 1a Model 1b Variable Coefficient Std Error Coefficient Std Error Constant 1.949 1.260366-0.306 1.0962 GPA -0.340 0.2592 FEMALE 0.452 0.305236 0.471 0.279707 AGE 0.003 0.0474 0.030 0.038843 ATTITUDE -0.307 0.104802 ** -0.292 0.0879 ** CONCENTRATION 0.200 0.076893 ** 0.218 0.072571 ** COMMUTE 0.024 0.009818 * 0.024 0.007827 ** WORKFULL 1.205 0.440768 ** 1.130 0.35124 ** McFadden R 2 = 0.253 McFadden R 2 = 0.276 Observations: 100 Observations: 126 * indicates statistical significance at the 5% level; ** indicates 1% level. Several different models were attempted in a test-down approach before the two models in Table 3 were decided upon. All seven study habits variables were initially included, but only ATTITUDE and CONCENTRATION were significant. Similarly, one specification included instead the aggregated STUDYHABITS variable but it was insignificant. That the other five categories of study habits were not significant is informative. It means that online and F2F students have similar note-taking, testing, reading, and time management skills in our sample. But where they appear to differ is in their ability to persevere through difficult assignments or a poor exam grade, and not procrastinate (ATTITUDE), and their ability to concentrate and stay on task (CONCENTRATION). These regression results have several important implications. First, prior expectations were consistent with our findings on the influence of commute time to campus and hours working on students online enrollment decisions. Online courses eliminate the need to drive to campus (except to take exams in this course, which were proctored on campus) and allow students more flexibility in their schedules. In this sense, online courses appear to be serving the student population for which it is designed: those with inability to attend lectures in traditional F2F courses. Second, none of the other demographic variables is statistically significant. At least in these two courses, one s gender, age, or GPA had no significant impact on one s decision to enroll in the online section. While some research finds that GPAs of F2F students and online students are different (and the sample means in this study are significantly different at the 10% level), when accounting for other factors, GPA does not seem to be a determinant of online enrollment. Third, and most important to our study, is the significance of the coefficients on ATTITUDE and CONCENTRATION. Referring back to Table 2, we find that the online students gave themselves 92
significantly lower ATTITUDE scores and had significantly lower GPAs (both at the 10% level of significance). The three questions for ATTITUDE and CONCENTRATION are listed below: ATTITUDE I continue with studying through difficult assignments and poor academic performance. I attend class, arriving punctually prepared to enthusiastically participate. I keep up with assignments, readings and test preparations, avoiding procrastination. CONCENTRATION I study effectively without the distraction of negative thoughts and worrying. I study where I can be most alert and attentive, eliminating noise distractions. I keep study time a priority, saying no to social demands and extracurricular events. It should be noted that the correlation between ATTITUDE and CONCENTRATION is roughly 0.5, so it is not the case that students rating themselves highly in one also rate themselves highly in the other. But even when they are correlated, and students have similar scores on both, the coefficients from the regression indicate that good students will be less likely to enroll in an online section. This is because the magnitude of the ATTITUDE coefficient is approximately 50% greater than the CONCENTRATION coefficient, and has the opposite sign. For example, consider a female student, age 22 who lives 14 minutes from campus and does not work full time. If she has ATTITUDE and CONCENTRATION scores of 15 each (the maximum possible), she has a 52.75% probability of enrolling in the online course. Now take that same student and drop her down to average ATTITUDE and CONCENTRATION scores of 9 each, and she now has a 69.68% probability of enrolling in the online section. Drop her scores even further to the minimums of 3 each, and she has an 83.18% chance of enrolling in online. While online courses are designed to give students flexibility in their schedules and increase access, it appears that they are also attracting a caliber of student that may not be as disciplined as their F2F peers. This finding appears to be consistent with Harmon and Lambrinos (2006), but they conclude this based on the coefficient on student age, inferring that older students are more mature. 6. Performance Now that we have determined that students with low ATTITUDE scores and high CONCENTRATION scores are more likely to enroll in online courses, the next question is: do these students do well in online courses? After all, low ATTITUDE scores mean the student gets frustrated when she does not perform well, does not participate much and fails to keep up with assignments, instead often procrastinating. Given the self-paced nature of many online courses, these types of students might be expected to not perform well in these courses. To answer this question, we measure performance using the final grade: the sum of all three exams and all Aplia assignments, normalized to a maximum of 100. Table 4 presents models 2a and 2b, the difference being that 2b excludes GPA in order to include more students in the analysis. 93
Table 4: Determinants of Final Grade Dependent Variable: FINAL GRADE Model 2a Model 2b Variable Coefficient Std Error Coefficient Std Error Constant 70.411 11.987 ** 74.964 9.444 ** GPA 5.947 1.928 ** ONLINE -39.098 15.044 ** -35.708 13.613 ** ATTITUDE 0.291 1.130 1.831 0.861 * ATTITUDE*ONLINE 0.975 1.464-0.025 1.290 CONCENTRATION -1.318 0.935-1.926 0.813 * CONCENTRATION*ONLINE 2.492 1.141 * 3.449 1.049 ** WORKFULL -2.949 2.684-4.704 2.539 HOURSSTUDYING -0.423 0.307-0.687 0.305 * Adj. R-squared 0.246 0.165 Observations 100 126 * indicates statistical significance at the 5% level; ** indicates 1% level. In Model 2a, results suggest that there exists a negative but very small influence associated with taking the principles of macroeconomics course online as opposed to F2F. Using the sample averages of 11.99 for the ATTITUDE score and 10.57 for the CONFIDENCE score, a student is expected to earn about 1.08 fewer points (on a 0 to 100 scale) if taking the course online (-39.098 + [11.99 x 0.975] + [10.57 x 2.492]). Although recent research suggests a larger negative influence attributed to the online format for economics courses, results from this study suggest that a careful research design that aims to control for instructor and course differences across online vs. F2F course type can affect results. When comparing course type using the same instructor with a closely-matched online version of the F2F class, the estimated difference in achievement between the two course types can be better identified as due to the nature of taking a course online, and not to differences in instructors effectiveness or other course content. Although there appears to be little difference in average estimated achievement across the two course types, these results do suggest that study habits and traits of students might play an important and greater role influencing achievement when students choose the online course. Using Model 2a s estimated coefficients for ONLINE, ATTITUDE, CONCENTRATION and the two interaction terms, we can compare the estimated effect of an increase in attitudes and concentration across course type. Following the finding noted above that there is a significant (and expected) positive correlation between ATTITUDE score and CONCENTRATION score, we expect the regression to have some trouble attributing achievement to one or the other (with statistical significance attributed to CONCENTRATION in the online course only in Model 2a), so we conducted a general F-test of the null hypothesis that the coefficients on ATTITUDE*ONLINE and CONCENTRATION*ONLINE are jointly equal to zero. This null hypothesis was rejected at the 1% level of significance. Due to this finding and the correlation between the two variables, we increase both simultaneously to estimate the influence of study traits on achievement across course type. To accomplish this, we can estimate the influence of an increase in each score from 9 (an average rating for each question of 3) to 15 (an average rating of each question of 5). For students taking the principles of macroeconomics course in the F2F format, this change in study characteristics is 94
associated with a decrease of 6.162 course points. Since the null hypothesis that the coefficients on ATTITUDE and CONCENTRATION are jointly equal to zero could not be rejected, we interpret this finding as a lack of estimated influence of these two study traits in the F2F course. For students taking the course online, this increase in study characteristics is associated with an increase of 14.640 course points, or roughly a letter-and-a-half grade increase. The benefit of higher concentration levels (greater ability to work through distractions and keep study time a priority) and better attitudes toward studying (more likely to work through difficult assignments and avoid procrastination) is predicted to be substantial for students taking the online course relative to students taking the F2F course. This finding is also consistent with research by Asarta and Schmidt (2013), who note that the increased use of online materials in a blended course requires an increased need for self-motivation on the part of students. They find that the more students keep up with the material throughout the semester (as determined by their pace variable and consistent with a student giving themselves high ATTITUDE scores in our study) the better they performed in the course. Of the other factors included in Model 2a, a student s GPA was found to be an important factor that significantly influences achievement. A 2-point increase in GPA, from 2.0 to 4.0 is expected to increase one s grade by 12 percentage points. Model 2b reports the findings when GPA is not included as a control, as has been the case in some studies estimating achievement in online courses. The estimated coefficients on three of the four variables that include study traits are now larger in absolute value, and three are now statistically significant at least at the 7% level, as opposed to one when GPA is included. Thus omitting GPA appears to lead to bias in the estimated coefficients on variables theoretically (and statistically) correlated with GPA such as the study trait variables in this study. 7. Additional observations While not included in this paper, there are a few other pieces of analysis that may be instructive for the reader. First, if the first exam score is included in models 2a and 2b, the only thing that is significant is the first exam score (value of 0.727) and the adjusted R-squared increases to 0.75. Once a student s performance is realized on that first exam, ATTITUDE, GPA, and everything else appears to be irrelevant. Thus, students should be advised that if they do poorly on the first exam, they have some serious changes to make in order to do well in the course. Second, additional dummy variables not included may provide more clues as to what it takes to succeed in economics. One regression included dummy variables indicating whether the student had already taken or was currently taking Principles of Microeconomics. Controlling for all variables included in Model 2a, students who had already taken principles of microeconomics had, on average, a 6% higher course grade, and students who were taking principles of microeconomics in the same semester had a 10.6% higher course grade. This latter finding is consistent with Terry and Galchus (2003), who used a sample of 870 students at the University of Arkansas at Little Rock, and concluded that students who took both principles classes concurrently performed better than students who took them sequentially. Analysis showed no correlation between students taking both courses together and having a higher GPA, so one is left to conclude that students are performing better in both courses because the two courses reinforce each other, not because smarter students believe they can handle both courses simultaneously. This has important implications for undergraduate advising. Third, it should also be noted that the actual course grade was also higher in F2F courses than in online. This course grade includes the remaining 12.2% of the course that remains after including exams and Aplia assignments. One potential issue in comparing online vs. F2F course effectiveness is precisely this difference in activities. These different assignments, based on what works best at encouraging 95
student engagement in each particular course delivery format, may have an impact on exam grades across format. Students in the F2F course were given in-class exercises, which are designed to have students working in small groups to apply the material they just learned in lecture. In-class exercises typically have one correct answer for most of the work and then a follow-up question at the end that calls for students to provide normative analysis. These assignments cannot be implemented practically in an online self-paced format where some students watched lecture videos on Monday while others waited until Thursday. Online students were instead given discussion questions and required to post regularly throughout the course for credit. To be effective at encouraging discussion, discussion questions are frequently open-ended and can have multiple answers depending on students normative judgments on the concept. These differences in assignments may have helped the F2F students perform better in the course, as the open-ended exam questions were more similar to the in-class exercises than the more open-ended discussion questions. One way of reducing the disparity between the performance of F2F students and online students would be to at least make these in-class exercises available to online students, if not require them to be completed and submitted before being discussed in online groups. Fourth, while not shown here, performance on all three exams was analyzed similar to Model 2a. The ONLINE variable was not significant for the first or third exam, but it was for the second exam. Students in the online course did significantly worse than students in the F2F course on the second exam. We are interested in using more data in the future on when students drop to determine if lowerachieving students dropped the course, which then eliminated the disparity in scores on the third exam. Finally, the lack of data on students who failed to complete the course in Fall 2010 prohibits us from analyzing factors that may cause students to drop courses. However, the survey was administered to all students in Spring 2011 at the beginning of the semester. Thus, we observe not only characteristics for the 34 students who completed the course that semester, but an additional 8 students that did not complete the course (although we do not have drop dates). A comparison of sample populations of online students including and excluding these students indicate that the students who dropped the course were more likely to work full time, were younger, and had lower ATTITUDE scores, all of which seem intuitive. This is clearly an area to investigate in the future as more data is gathered. 8. Conclusion and extensions This research aims to enhance the literature on online instruction of economics by investigating specific student characteristics that impact their enrollment in online courses and, later, their performance in those courses. Surveys given to students provided information about students attitudes along seven different study criteria: attitudes, concentration, note-taking, testing, time management, reading, and motivation. We found that students with poor study attitudes (inability to push through difficult assignments, tendency to procrastinate, etc.) but strong concentration traits (ability to work through distractions) are more likely to enroll in online courses. When comparing performance across course format, we find that a carefully-designed research study shows no large difference in achievement across course types when controlling for student, instructor and course characteristics, and it appears that students with high scores on concentration and study attitudes are at an advantage in online courses. Unfortunately, the population that takes online courses tends to score lower on study attitudes, so they end up earning lower grades. This information is important for both students and faculty alike. Ultimately, our goal is to create a short survey or questionnaire that a student could complete before signing up for a course. After entering the information, probit analysis would determine their likelihood of passing the course. Some 96
students with poor study habits who opt in to online courses may, after receiving this information, reconsider this decision and enroll in a F2F section instead. Future work will have students who enroll in the course (both face-to-face and online) take the more expansive survey used in this study at the start of the course. We will then monitor when students drop out of the course. This information will then be used to create a tool that students can use to determine their likelihood of dropping out of an online course before they even enroll, or after taking the first midterm exam. The hope is that students have a more accurate idea of their chances of success in an online economics course before they spend more of their time and money. References Allen, E. and J. Seaman. 2011. Going the Distance: Online Education in the United States, 2011. Babson Survey Research Group. Allen, E., J. Seaman, D. Lederman, and S. Jaschik. 2012. Conflicted: Faculty and Online Education, 2012. Babson Survey Research Group. Asarta, C., and J. Schmidt. 2013. Access Patterns of Online Materials in a Blended Course. Decision Sciences Journal of Innovative Education 11 (1): 107-123. Coates, D., and B. Humphreys. 2001. Evaluation of Computer-Assisted Instruction in Principles of Economics. Educational Technology and Society 4 (2). Euzent, P., T. Martin, P. Moskal, and P. Moskal. 2011. Assessing Student Performance and Perceptions in Lecture Capture vs. Face-to-Face Course Delivery. Journal of Information Technology Education 10. Glanz, S. 2012. Student Performance in Traditional and Online Sections of an Undergraduate Macroeconomics Course. International Journal of Social Sciences 1 (2): 71-82. Gratton-Lavoie, C., and D. Stanley. 2009. Teaching and Learning Principles of Microeconomics Online: An Empirical Assessment. Journal of Economic Education 40 (1): 3-25. Harmon, O.R., and J. Lambrinos. 2006. Online Format vs. Live Mode of Instruction: Do Human Capital Differences or Differences in Returns to Human Capital Explain the Differences in Outcomes? Working Paper. Schuhmann, P.W. and K. McGoldrick. 2001. Integrating Web-Based Student Competitions into Principles Courses. Journal of Effective Teaching 5 (1). Terry, A. and K. Galchus. 2003. Does Macro/Micro Course Sequencing Affect Student Performance in Principles of Economics Courses? Journal of Economics and Finance Education 2 (2): 30-37. Trawick, M., S. Lile, and R. Howsen. 2010. Predicting Performance for Online Students: Is it Better to be Home Alone? Journal of Applied Economics and Policy 29 (1): 34-46. 97
Appendix ECON 205 BACKGROUND QUESTIONNAIRE By writing your student ID below, you are asserting that you have read and responded to all items honestly. Student ID: Would you like me to send to you the class-average results of this survey? Yes No Directions: Rank your academic performance in each of the 7 study skill categories using the designated criteria. 1=Almost Never 2=Rarely 3=Sometimes 4=Frequently 5=Almost Always ATTITUDE I continue with studying through difficult assignments and poor academic performance. I attend class, arriving punctually prepared to enthusiastically participate. I keep up with assignments, readings and test preparations, avoiding procrastination. CONCENTRATION I study effectively without the distraction of negative thoughts and worrying. I study where I can be most alert and attentive, eliminating noise distractions. I keep study time a priority, saying no to social demands and extracurricular events. TESTING I go into an examination prepared and certain of what I ve studied. Before a test, I find out what will be on the exam and what kind of test will be given. I am relaxed, confident, and without test anxiety during an examination. MOTIVATION I see college as an investment in my future. I trust myself and have confidence in my ability to succeed in college. I motivate myself, showing initiative and taking personal responsibility for learning. NOTE TAKING I take notes that are legible, organized, creative and personalized for best recollection. I review my notes as soon as possible after each class or study session. I copy key information from the board or power point or emphasized by the professor. READING I am satisfied with my reading speed and characterize myself as a good reader. I use the SQR3 reading system survey, question, read, recite, and review. I remember most of what I read because I comprehend and concentrate well. TIME MANAGEMENT I study at approximately the same time each day or night, establishing routine. I schedule the most difficult studying first while the mind and body are most alert. I prioritize by putting studying first and then balancing recreation, work, and sleep. [PLEASE CONTINUE ON THE BACK SIDE] 98
Directions: Please check or provide the answers to ALL of the following items. 1) Gender: Male Female 2) Are you a U.S. citizen? Yes No 3) What was your ACT or SAT score? ACT SAT Did not take 4) How many credits are you taking this semester? 5) Approximately what is your overall GPA? 6) Are you a student athlete at St. Cloud State University? Yes No 7) Are you involved with SCSU student government? Yes No 8) Are you involved with (non-student-government) SCSU student organization(s)? Yes No 9) Are you a member of a sorority or fraternity? Yes No 10) Do you WORK during this semester? No Yes, part time Yes, full time 11) On average, about how many hours per week do spend STUDYING for this class? 12) Have you taken ECON 201 (Introduction to Economics)? Yes No 13) Have you taken ECON 206 (Principles of Microeconomics)? No Taking currently Already taken 14) Why did you take this course? Required for College of Bus. Required for economics major/minor Satisfies gen. ed. requirement Other (briefly explain) 15) Do you live on or off campus, or with family? On Off, not with family Off campus, with family 16) If you live off campus, approximately how many minutes does it take you to commute one way to campus? 17) Are you married? Yes No 18) How many children do you have? 19) What is your current age? 20) Are you a veteran of the U.S. military? Yes No 21) In your own opinion, how much TV do you watch (personally, both researchers believe we watch too much TV). I seldom watch TV Some, but not a lot More than I think I should 22) In your own opinion, how much time do you devote to socializing during the semester? Too little About right Too much 23) Calculus is not required for the economics major. Pretend you are considering economics as a major. IF calculus was required, would having to take calculus discourage you from the economics major? Yes No 24) Have you taken calculus? Yes No, and I probably won t No, but I plan to take calculus Thank You 99