Does Attendance Matter? Evidence from an Ontario ITAL

Transcription

1 Does Attendance Matter? Evidence from an Ontario ITAL Pierre-Pascal Gendron and Paul Pieper The Business School Humber Institute of Technology & Advanced Learning Toronto, Canada Draft for Discussion May 15, 2005 Abstract Academic administrators and faculty often take for granted the positive impact of classroom attendance on student achievement. The evidence is usually anecdotal but nevertheless well accepted. As a result, there have been few studies focussing on the separate effect of attendance on achievement. This study examines the separate impact of classroom attendance on student achievement using a repeated cross section of studentand classroom-specific data. Attendance is measured very precisely and the data come from an introductory microeconomics course we taught at the Humber Institute of Technology & Advanced Learning in Toronto, Ontario, Canada. Preliminary OLS results using our baseline model shows that attendance has a strong positive impact on final course grade. Our fixed-effects model (with class-specific effects) shows again a strong impact of attendance on final grade. In both models, the relationship is non-linear in a way that suggests diminishing returns to attendance. Unlike some of the previous literature, however, we do not find in our sample a threshold beyond which attendance would negatively affect achievement. Key words: student attendance, achievement, performance JEL Classification: I21 Acknowledgements: We wish to thank several members of Humber s Office of Research: Dr. Peter Dietsche for comments, and Gillian Brenning and Chali Chen for expert research assistance. We gratefully acknowledge financial assistance through an Institutional Research Grant from the Office of Research.

2 Contents 1. Introduction The Literature Non-Behavioural Determinants of Student Performance Student Absences and Academic Performance Synthesis The Institutional Environment Institution and Student Body The Business School The Data Empirical Results Baseline Model Fixed-Effects Model Sensitivity Analysis Conclusions Appendix A: Scatter Plot of Grade on Attendance Ratio Appendix B: Tabular Summary of Literature Review References i

3 1. Introduction Academic institution administrators and faculty intuitively believe that student attendance in the classroom under either the lecture, tutorial, or discussion group delivery systems matters for student achievement. In most institutions casual observation or common sense supports this belief. Unfortunately, the evidence remains anecdotal unless the impact of attendance on achievement can be quantified. The examination of the association between attendance and achievement brings forth additional benefits besides understanding the relationship between the two in isolation. Other student and classroom characteristics may also be examined in the process, especially in the context of their relationship with student achievement. Understanding the impact of attendance has implications for faculty and administrators. Instructors want learning to take place. Since the classroom environment evolves slowly and the lecture delivery method remains dominant, instructors may want to maximize attendance within this system using the various methods at their disposal, such as class discussions, class exercises, rewards, and so on. Administrators also have a stake in this: in order to minimize student attrition, they may find it desirable to clarify or strengthen existing academic rules in favour of attendance. The problem is generally more confined in programs where attendance is compulsory. The paper reviews studies of attendance and various other determinants of student performance from a variety of academic disciplines. It then presents the results of an empirical investigation of the relationship between final course grades and attendance as well as other student and classroom characteristics. We conducted the investigation at the Humber Institute of Technology & Advanced Learning (hereafter Humber ), a large community college in Toronto, Ontario, Canada. Moreover, we relied exclusively on primary student- and classroom-based data. 1

4 Section 2 reviews and summarizes the relevant literature, and outlines some of the directions for research that emerge from the literature. Section 3 describes Humber s institutional environment and student body, with an emphasis on The Business School, within which the study was conducted. Section 4 describes the data and collection methods, and provides results from univariate and bivariate statistical analyses. Section 5 describes the models used, the empirical analyses carried out, and their results. We conclude in Section 6. 2

5 2. The Literature The literature is consistent overall in showing a link between classroom attendance and academic performance (usually measured as a continuous percentage grade). In order to better understand the context of student performance, the first section of the literature review examines possible non-behavioural determinants (e.g. external learning environment effects) and student socio-demographic characteristics and their relation to academic performance. Next, we present a review of the literature as it pertains to classroom attendance and the impact of non-attendance on grades as an outcome of academic performance. We mention other determinants of academic performance when relevant. Not all studies have been able to find a statistically significant relationship between attendance and grades. In nursing, where classroom attendance was very much part of the program culture, the relatively low number of absences (an average of 1.4 classes missed by students with an 80 percent plus grade, and an average of 4.4 classes missed by students in the 50 to 60 percent) even for the weaker students might account for the mixed results by Brown et al. (1999) when correlating attendance with absences. Of the nine class sections studied, only four were found to display any significant negative correlation. Although the research population was drawn from the same college as the present study, the students under review in Brown et al. (1999) represent a unique population with a significantly different program-induced attitude toward student responsibility with respect to absenteeism. Although Thompson and Plummer (1979) noted a significant difference between successful and unsuccessful remedial English students, they were unable to show that attendance had any impact on the letter grades achieved by the successful students. Since Berenson et al. (1992) dealt with remedial students in mathematics, additional underlying factors other than attendance may explain their failure to find a positive correlation between attendance and grades. Hyde and Flournoy (1986) surprisingly concluded that mandatory classroom attendance policies could impede learning. However, the majority 3

6 of the literature reviewed would seem to support the intuitive and reasonable notion that attendance does improve academic performance Non-Behavioural Determinants of Student Performance In order to deal with the context of student absenteeism many studies focused on researching the potential link between the characteristics of the classroom, or schools, and student performance, whereby class attendance was only one of those characteristics. Besides measuring individual student characteristics such as gender, race, parental income, and other individual student characteristics, studies such as Betts and Morell (1999), Hanushek et al. (2003), Nichols (2003), and Sosin et al. (2004) all examined various student populations, ranging from kindergarten to university economics students, to determine which environmental (i.e. classroom or school) or student (sociodemographic background) characteristics best explained the variance in educational performance. Betts and Morell (1999) found significant differences in grades, as one outcome of educational performance, between different programs. Specifically, they found the GPAs were lowest in science and engineering and highest in the arts and sciences. Hanushek et al. (2003) were able to establish that the peer experience in the classroom had a significant effect. Because members of peer groups tend to have similar experiences over time through systematic neighbourhood and school choice, they argued, omitted peer characteristic factors would be common to the entire peer group, including contemporaneous inputs. Supporting the findings of Hanushek et al. (2003), Nichols (2003) observed in a large empirical study of high school graduation student data in Indiana that lower income students consistently had greater failure rates, sometimes double that of higher income student graduate groups, and that ethnic majority (i.e. white) females tend to have higher GPAs than other comparator groups. Interestingly, the results were not equally strong across all disciplines. Correlation analysis found a consistently strong negative 4

7 correlation among language, mathematics, and reading scores in relation to yearly average absenteeism rates. The relation between absences and reading was less clear, while for mathematics and English it was very clear. Sosin et al. (2004) also found the technology used in teaching, students overall GPA, and whether the student had taken a prior mathematics course, to have a significant impact on course performance. Didia and Hasnat (1998) modeled a standard production function approach using both questionnaire and institutional data. Among the 210 SUNY students enrolled in introductory financial management courses taught by four different faculty members, a strong positive statistical relation was found between students cumulative GPA score and their current course grade. That students overall average performance should be a good predictor of academic achievement in any one course is not surprising. What was unique was their finding that there appeared to be a marginally significant negative relationship between the number of hours of study and course grades. Gender or age (i.e. maturity) did not contribute to predicting grades. Emerson and Taylor (2004) also used a standard production function approach, but used it to determine whether how a Principles of Microeconomics was taught had an impact on outcome scores. They specifically investigated whether students learned better in a traditional lecture course or in a course with an emphasis on classroom experiments. Students in experiment-based classes seemed to achieve better TUCE and other exam scores than students following the more traditional pedagogy, but there were potential problems with positive selection, and teacher effects. Females seemed to benefit most, but not non-whites. In an econometric analysis of a large-scale experiment on the placement of kindergarten students, Krueger (1999) analyzed the impact of randomizing student placements into small and regular sized classrooms with or without teaching aides. He found that the average performance on standardized tests increased by four percentile points the first year that students attended a small class. This could be important for the present research in that our average college classroom size is significantly smaller than 5

8 that typically found in freshman introductory university courses. He also found that the class size had a larger grade impact on minority students, which again is a concern at the current college study site. Teacher characteristics (e.g. Master s versus Bachelor s degree or length of teaching experience) did not have much of an effect. Using an education production function, Krieg and Uyar (1997) found that university students enrolled in introductory business/economics statistics courses achieved student performance that was significantly better in the spring than fall semesters. Similar to other studies, they noted that the students overall GPA, mathematics scores, and a parental income surrogate measure were positive and significant factors in predicting course grades. On the other hand, student gender, being a transfer student, and examination schedules were insignificant. Living in a dorm, hours spent working and the percent of Friday classes missed also tended to lower course grades. Linking student postal codes to census data describing various socio-economic characteristics of the communities in which the students lived allowed Johnson (2005) to estimate that only 40 to 50 percent of the variation in school success rates can be ascribed to socio-economic characteristics Student Absences and Academic Performance Although background characteristics may have an influence on academic performance, as measured by grades, other studies developed research designs to directly examine the impact of individual student behaviour, specifically students decisions not to attend class. This work is close in spirit to optimization models that incorporate time allocation decisions. Siegfried and Walstad (1998) argued that student effort, study time and attendance have an important influence on students achieving higher grades. While study time was measured, neither the quality nor the intensity of the learning could be 6

9 quantified. Johnson et al. (2002) used ordinary least squares (OLS) regression to assess more directly the relation between effort and student performance. Instead of selfreported survey data on effort, the authors measured the number of attempts made on practice quizzes and the time spent on them. They found a positive and significant relationship between both measures of effort and grades. Again, they did not find gender to be significant but they did find a positive relation between course grades in an introductory financial management course and overall GPA. However imperfect attendance measurement data are as a proxy for effort or quality of time spent in the classroom, they can be viewed as a distinct input variable that at a minimum represents a level of effort and perhaps motivation by the student. In that vein, several studies specifically examined the relationship between class attendance (or lack thereof) and final course grades. Often these studies were set up as economic production function models. Empirical evidence of an important relationship between classroom attendance and academic performance has been established in many different learning and teaching environments. Durden and Ellis (1995) found that the attendance effect was non-linear and mattered only after a student missed more than four classes, with the size of the negative impact increasing with each additional absence beyond that threshold. They also found that in addition to absenteeism, students GPA and college entrance examination scores (MSAT, VSAT) are among the most important determinants of student academic performance. Having taken a calculus course had a significant positive effect on outcome grades. The finding of a non-linear relationship with a threshold effect before it negatively affects student performance was also the conclusion reached by Arce et al. (1996). Buckles and McMahon (1971) conducted one of the few random experiments that looked at the differential impact of having only a programmed text but not attending economics lectures versus a control group that had students attending lectures with the programmed text. They found that students gained little from attending lectures if the 7

10 lectures did no more than explain the material covered in the assigned reading. This study suggests that there must be a benefit to students learning if students are to be expected to attend classes and that it may not be enough to only measure attendance and not the impact of a lack of attendance. Marburger (2001) made an interesting observation on why the absences, or lack of attendance, may not be as easily detected. Consistent with Romer (1993), he concluded that students quickly determine which teacher is the better instructor from their perspective, and therefore provides the greatest value-added in terms of attendance. To the extent that Marburger (2001) found examination scores of students with relatively high attendance patterns to have a significant positive effect, it follows that classes missed in which quality of instruction was poor would have less of an impact. Moore et al. (2003) performed OLS regression on historical data and class attendance data collected by various science instructors and found a significant positive association between attendance and course grades. Shimoff and Catania (2001) explored the potential impact of encouraging attendance through having introductory psychology students sign-in. They found that having students sign-in did produce a significant impact on student outcomes, as measured by quiz marks. They also found that students in the B and C grade range experienced the most pronounced positive effect. The latter result may demonstrate that students do not always accurately estimate the extent of their absences. Van Blerkom (1992) noted that undergraduate psychology students self-reported that they attended classes more regularly early in the semester and less frequently later in the semester. Even with these self-reported data, the correlation between class attendance and course grades was significant. Sophomores reported missing more classes than freshmen or senior students, and gender was not significant. 8

11 In another health educational setting, Newman et al. (1981) found that third-year dental students did demonstrate a significant and positive correlation between the number of classes missed and numerical final grades. What surprises here is that the authors could establish a correlation since these senior professional students knew that attendance was required and it was recorded. 2.3 Synthesis As the review of the last few studies cited suggests, even in situations where students are aware of attendance expectations and the implications of absenteeism, some students still choose to miss classes. With few exceptions, most studies have been able to show that student class absenteeism has a predictable negative impact on academic performance as measured by course grades. Despite several attempts, research has not conclusively established yet the impact on grades of the following variables, among others: student motivation, student innate abilities, and the amount studying time spent by students. The potential difference in quality of instruction across teachers has also not been sufficiently examined in the literature. Since the correct model is not yet known, it appears important for pedagogical reasons and academic policy and to determine in each unique learning environment the extent to which student absenteeism is to be acceptable or accepted. 9

12 3. The Institutional Environment We briefly describe below the environment at the Humber Institute of Technology & Advanced Learning ( Humber ), its student body, and the profile of students from The Business School who are the subjects of this study Institution and Student Body In April 2003, the institution then known as Humber College obtained a designation as an Institute of Technology & Advanced Learning (ITAL). This designation was bestowed on only three of the 22 community colleges in Ontario by the provincial government. The designation was in recognition of the advanced level and high quality of academic programming that Humber College has provided since its creation in Humber is one of the largest colleges in Ontario, with over 16,000 full-time students, 1,200 of which are enrolled in the University of Guelph-Humber. The latter is a unique joint venture between the University of Guelph and Humber, whereby students graduate with both a degree and a diploma from the two respective educational institutions. Humber s students are enrolled in over 160 full-time programs including: diploma, certificate, apprenticeship, bachelor degree, and post graduate certificate programs. In addition to full-time offerings, Humber provides part-time courses to approximately 65,000 students who are enrolled in traditional evening courses, on-line courses, and off-site industry purchased programming that is customized to meet their specific needs. Finally, Humber is one of only 12 Vanguard Learning Colleges in North America and the only one in Canada selected for their excellence in education and training. 10

13 3.2. The Business School The Business School is the largest of six schools at Humber, with approximately 30 percent of Humber s total enrolment. The Business School mirrors Humber in that it offers high-quality advanced programming, both full- and part-time, and at the certificate, diploma, bachelor degree, and post graduate certificate programs. The School has several partnerships with various industries and is actively involved in several international projects, notably with Ningbo University (China), Université de Lyon (France), and projects in St. Vincent, Tanzania, and Zimbabwe to name a few. Within the Business School there are several distinct post-secondary program areas that include Accounting, Business Administration, Business Management, Court and Tribunal Agent, Fashion Arts, Law Clerk, Logistics, and Marketing programs. At the post-graduate level, the School offers Human Resource Management, International Marketing, International Project Management, Professional Golf Management, and Public Administration certificate programs. The School offers a Bachelor of Applied Business in e-business as well as a Bachelor of Applied Arts in Paralegal Studies. The students included in this study come from the following post-secondary programs: four- and six-semester Accounting diploma programs, a six-semester Business Administration diploma program, and two four-semester diploma programs in Business Management and Marketing. Accounting students take the introductory Microeconomics course in their third semester. The other three Business diploma programs have a common first year and it is during the second semester of that first year that students are scheduled to take the introductory Microeconomics course. In all four programs, Microeconomics and all other economics course are one-semester course. Only Accounting and Business Administration students are scheduled to take additional economics courses: Macroeconomics for the former, and Macroeconomics, Labour Economics, and Money, Banking and Finance for the latter. 11

14 4. The Data We collected the data for this study in the courses we taught during the normal day-time teaching activities at Humber in the academic year. The data come from 13 sections of the Microeconomics principles course for full-time students. The data were collected over the three semesters (fall, winter, and summer) over which the course is offered. We collected the data for almost all the sections we taught and focused on the Microeconomics course only since it is a mandatory course in all business diploma programs. As noted earlier, students usually take the course in their first year of study. We collected the data on student attendance throughout the semester, and compiled and coded student characteristics (including final course grade) and classroom characteristics at the conclusion of each semester. The review of the literature suggests that several factors affect student achievement beyond contemporaneous student and classroom characteristics. This is no doubt true, as the variety of data and methods surveyed in the previous section shows. We made a deliberate choice to limit ourselves to contemporaneous student- and classroom-specific data for several reasons. Firstly, we had complete control over the data collection and coding and, as a result, were able to avoid measurement error almost entirely. Secondly, we wanted to use all of the information available from the classroom at the individual section level, thus naturally setting the stage for a fixed (section) effects model. Finally, we wanted to understand all usable data prior to enriching the model with other entry characteristics such as high-school grade-point average. As a result of our choices, the data set consists solely of primary data. We note at the outset that attendance is not compulsory in Business School programs. We recorded attendance in each class and here we measure attendance for each student as a weighted average ratio by dividing the student s number of classroom hours attended over the entire semester by total possible hours over the semester. 1 1 In fact, each period lasts for 50 minutes with a five-minute break between each period. The approximation is made to simplify exposition only and does not affect the accuracy of the calculations. 12

15 Classes amount to a total of three hours per week (four in the summer semester) and those can be delivered in block times of one hour plus two hours, or 3 straight hours (two plus two hours in the summer). Our weighted ratio uses this extra information in that it accounts for each hour attended and hence acknowledges that missing a straight threehour block is more serious than missing a single period or a two-hour block. The measure also picks up variation across sections by accounting for classes not taking place due to holidays, teacher absences, or two-hour tests or exams taking place during a threehour block, for example. This measure thus maximizes the variation in attendance at the student level by picking up more variability than measures normally used in the literature. One further advantage is that the absence ratio is simply one minus the attendance ratio. The class sizes varied between 27 and 39 students per class and so we knew the students well and therefore had a high degree of confidence in the accuracy of these ratios. To our knowledge, this way of measuring attendance has not been used yet in the literature. Table 1 provides descriptions of the variables employed in the baseline OLS model, along with their means, standard deviations (Std Dev), and minimum and maximum values. Table 1 Description of Variables (n = 429) Variable Description Mean Std Dev Min Max GRADE Student course final grade ARATIO Student attendance ratio ARATIO2 Student attendance ratio squared DGR Dummy: 1 if female, 0 if male DPG Dummy: 1 if Business Administration program, 0 otherwise CSZ Class size D30 Dummy: 1 if three-hour block, otherwise DSEM Dummy: 1 if fall semester, otherwise DPR Dummy: 1 if professor A, 0 otherwise

16 The scatter diagram shown in Appendix A allows for a quick view of the key relationship here. It suggests a fairly strong positive association between GRADE and ARATIO, with a mass of observations near the upper right-hand corner of the quadrant and a fairly steep and sparse relationship near the origin. The scatter plot suggests a nonlinear relationship which justifies our addition of the square of ARATIO to allow for a quadratic curve shape. We coded student and classroom characteristics based primarily on data availability, and secondarily on the literature. We comment below on each variable and our expectations, if any. We refer to Table 1 for variable descriptions. GRADE, ARATIO, and ARATIO2. The foregoing discussion and the scatter diagram in Appendix A point to a positive and non-linear association between attendance and achievement. DGR. We do not have a precise expectation with respect to the association of gender with achievement. Many studies have found it not to be a significant factor either way. DPG. The Business Administration diploma program is the most comprehensive diploma program offered by The Business School in that it lasts for six semesters and features significant breadth and depth. We would expect the program to attract the strongest students and hence being registered in the program to be associated with higher achievement, everything else equal. CSZ. Class size is believed by many to affect achievement; witness current elementary school reforms in Ontario where reductions in class size have been the centerpiece of the reformers arguments. The small variation in class sizes in our sample makes any expectation unlikely to be fulfilled. 14

17 D30. Students and teachers frequently express the view that consecutive three-hour blocks are less conducive to good learning than broken blocks. The reasons commonly cited include fatigue and saturation with material. We therefore expect that variable to be associated with lower achievement, everything else equal. DSEM. Stronger students usually take the Microeconomics course in the fall so we expect fall results to be stronger than for other semesters, everything else equal. DPR. The professor (or teacher) dummy accounts for the fact that the authors each taught a share of the sections in the sample. Given that there are only two teachers in the sample, the variable is of limited interest at this time. In a similar study with more instructors, however, one could effectively model different instruction methods and their effectiveness and examine their impact on grades as well as attendance. This is left for future work. 15

18 5. Empirical Results Table 2 reports the results of two OLS regressions with GRADE used as the measure of student performance and hence as dependent variable. All the other variables listed in Table 1 are used as independent variables. Table 2 Determinants of Student Performance in Introductory Microeconomics (Dependent Variable: GRADE; n = 429) Independent variable Model 1 Coefficient Model 1 t value Model 1 P value Model 2 Coefficient Model 2 t value Model 2 P value Constant ARATIO < < ARATIO < <0.001 DGR DPG CSZ < D DSEM DPR < SECTION SECTION SECTION < SECTION SECTION SECTION SECTION SECTION < SECTION < SECTION SECTION SECTION Adjusted R ESS b Notes: a The P value is the probability, if the test statistic really were distributed as it would be under the null hypothesis, of observing a test statistic no less extreme than the one actually observed. If it is less than α [say 5%], then we would reject [the null hypothesis that the coefficient is zero] at the α [=5%] level. See Davidson and MacKinnon (1993), pp ; b Estimated Sum of Squares. 16

19 5.1. Baseline Model Model 1 is estimated by regressing GRADE on all the independent variables listed in Table 1. This is our baseline model (Model 1). We comment on the SECTIONi variables shown in Table 1 further below. The estimates from Model 1 reveal a statistically significant and quantitatively large relation between attendance and performance, even when taking other student and classroom characteristics into account: the t value on attendance is almost eight. As suspected, the coefficient on ARATIO2 is negative and statistically significant so the relationship is non-linear. This confirms our initial interpretation of the scatter diagram in Appendix A: the impact of attendance is strongest for students who are close to the origin but becomes weaker as ARATIO increases. This suggests the presence of diminishing returns to attendance. Turning the result around, this suggests that missing a few hours will have a small negative effect on achievement while missing a significant number of hours will have a negative and significantly larger effect on achievement. This is consistent with the findings of Durden and Ellis (1995), and Romer (1993), for example, in the context of economics classes. The impact of the professor could not be determined reliably at this point, as there were too few professor-sections. The Microeconomics course has a common course outline, outlining the evaluation scheme and schedule to be used so each course should be very similar in content and evaluation. However, the two professors did use slightly different approaches to classroom attendance management, and so this aspect of the study will be examined further as more data are collected. One of the weaknesses of the baseline model is that it is the restricted version of a more comprehensive model. Firstly, it is likely that there exist other characteristics at the section level that influence success in the course. Those variables are not included in the equation due to data unavailability and hence represent omitted variables. Secondly, the sample used in estimating Model 1 is a pooled set of repeated cross sections with each class section (there are 13 of them) representing a cross-sectional unit. The assumption implicit to the pooling procedure is that both the intercept and slope are constant across 17

20 sections; that is not necessarily reasonable. A typical procedure with cross-section models is to specify cross-section fixed effects whereby there are 12 section (=13 1) dummies that take the value 1 for the individual section, 0 otherwise. Given that the data are repeated cross sections without a time-series component, there is no reason to believe that random effects would serve any purpose. We turn to that model next Fixed-Effects Model Results from the estimation of the fixed-effects model (Model 2) are shown in the columns labelled Model 2 in Table 2. As is often the case in models where dummy variables account for a significant share of the independent variables, complex patterns of multicollinearity can emerge. We performed sensitivity analyses on the models and based on the results, determined that the four independent variables CSZ, D30, DSEM, and DPR jointly caused the problem in the presence of the section dummies. The first variable is continuous but the remaining three are dichotomous. We eliminated the problem by dropping those variables from the fixed-effects model. Results for Model 2 in Table 2 reflect this elimination. As shown in Table 2, section dummies are statistically significant at conventional levels in half the sections. This is not a bad result considering the relatively small total number of sections (13). Even with the presence of fixed effects, ARATIO continues to be large and highly significant: its t value is 7.51, slightly below the value of 7.98 in the baseline model. The quadratic term ARATIO2 is still negative and significant, implying a non-linear relationship between GRADE and ARATIO. As in the baseline model, DGR and DPG are not significant. Finally, we perform an F test to evaluate the restrictions imposed by the baseline model in comparison to the fixed-effects model. The baseline OLS model is the restricted model while the fixed-effects model is unrestricted. The test evaluates whether the difference in the error sums of squares (ESS) is sufficiently large. The test statistic is equal to [(ESS restricted ESS unrestricted )/r] / [ESS unrestricted )/(n k)] with r = 12 restrictions or 18

21 degrees of freedom on the numerator, and (n k) = = 412 degrees of freedom on the denominator. Using the sums of squares reported at the bottom of Table 2, the F test statistic is 3.41, which exceeds the critical value of 2.18 with an α = 1% size test. The test rejects the null hypothesis that the equal-intercept restrictions are correct and thus supports the fixed-effects model. Correcting the test statistic to account for the fact that four variables (CSZ, D30, DSEM, and DPR) were dropped in the unrestricted model does not change the test s conclusion Sensitivity Analysis The results are in agreement with some parts of the literature in that student characteristics such as gender do not exhibit a significant relationship with student achievement. What is perhaps more unusual is the strength of the attendance effect even when taking account of available student and classroom characteristics and class section effects. In one of our sensitivity analyses, whose results are not detailed here, we estimated restricted and fixed-effects versions of a model where the dependent variable is instead ARATIO and the independent variables consist of the variables listed in Table 1, excluding ARATIO and ARATIO2. The qualitative results are rather similar to those obtained from Model 1 and Model 2 with the exception of gender, which is significant. The results suggest that females attend more. We strongly qualify this by noting that the fit in models using ARATIO as a dependent variable is very poor relative to that of the models using GRADE as dependent variable. One explanation for those results is that while contemporaneous student and classroom characteristics fail to capture the determinants of attendance per se, they capture a significantly larger share of the determinants of achievement. Even though the models are not strictly comparable due to the different dependent and independent variables, we note that the fixed effects model with GRADE as dependent variable explains 2.7 times the variance of the fixed effects model with ARATIO as dependent variable. 19

22 6. Conclusions The literature generally suggests that attendance matters for academic achievement. There is less unanimity with respect to the other variables that contribute to achievement, however. The identity of such variables depends on studies features such as samples, levels, academic disciplines, and so on. In this study, we chose to restrict ourselves to data specific to student activity in introductory Microeconomics classes. The results of this study indicate that attendance does matter for academic achievement in the Microeconomics course, even after considering other student- and classroom-specific characteristics. The evidence suggests that the effect is non-linear: the effect of attendance on the grade is stronger at lower levels of attendance but levels off at higher levels of attendance. There are several potential extensions to this work. Firstly, we are currently in the process of adding data for the academic year. In addition to increasing the reliability of the results, this will allow us to test a (double-cohort) year effect. Secondly, it is straightforward to convert our continuous attendance ratio into discrete absence categories following Durden and Ellis (1995) to establish the threshold past which lack of attendance begins affecting achievement in a negative and significant fashion. Finally, we intend in later work to match student characteristics at entry (e.g. prior achievement variables such high school GPA) and possibly socio-economic characteristics with the student records that currently make up our expanding data set to see how the model and results change. This will also allow us to examine the effect of prior characteristics on grades as well as attendance itself. 20

23 Appendix A: Scatter Plot of Grade on Attendance Ratio Plot of GRADE*ARATIO. Legend: A = 1 obs, B = 2 obs, etc. 1.0 ˆ A A A A A B BA A AA C B B A A A A AA B A A A A A AD A A A A A AA B B C B 0.8 ˆ A AA A A A B A ABAC E CA C F A A A B A A AA A ACA BDAA F A A i A AB A AB AACDA AA A AC C B n A A A D E A A E B B AB AB I B A a A A A A A A AA AA B AAB B l AA A A C CAABAABA C AB B CBA AAAA DAB B A ABAA AAB AAA A CCA B C A A AAAB B C 0.6 ˆ B A A A C AA A B BE CA A BAA B o B A A A A u A B EA AB B B A AAA A ABAA B A r A A B A B B BA A A AA A A s AA A AA AA A A e B A B A AA A G 0.4 ˆ A A A A A r A a A A d A B A A A e A A A A A A A A 0.2 ˆ A B A A AAB A A A A A A A A A AA A A 0.0 ˆ Šƒƒˆƒƒƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒƒƒƒƒƒƒˆƒƒ Attendance Ratio 21

24 Appendix B: Tabular Summary of Literature Review 22

25 Table B1: Summary of Literature Documenting Association between Attendance and Grades Study/ Details Arce et al. (1996) Berenson et al. (1992) Betts and Morell (1999) Type of Study Study Population Dependent Empirical controlled experimental study Tinto theoretical predictive model measuring the strength of both students goal and the respective institutional commitments to understanding the factors influencing failure OLS Regression analysis of GPA to measure relative effectiveness of high school resources (model GPA as function of characteristics of the students schools) First semester freshman college students enrolled in a remedial math course during Fall 1988, N=263 Students who enrolled at U. of California between 1991 and 1993, who had previously attended California public high schools, N=5,000 Final grade Cumulative university Grade Point Average on a scale of 0 to 4 Independent SAT-Verbal and Math scores, high school grade point average, Group Assessment of Logical Thinking (GALT), an attitude to math score, and attendance High school profile data, including average student and family demographic characteristics associated with their high school Methodological Issues Conscientious students are more likely to attend classes but would do well even if they did not attend Stepwise regression of predictor variables for two years, where attendance was required in 1988 and not required in 1987 High school environmental effects were measured as well as student characteristics, vs. direct effects of their behaviour (e.g. class attendance) Key Relevant Findings Attendance conferred a positive effect on learning. Relationship between attendance and performance is probably non-linear and has a threshold before it negatively affects student performance. Results indicated no differences in final grade between the two years, where attendance was required in one year and not in the other. High school grade point average had the highest correlation with the dependant variable, but only accounted for 8% of the variance. The SAT-M accounted for an additional 2% of the variance and was not significant. Most family background variables were highly significant, including: males and ethnic minorities tend to have significantly lower GPA than females or whites; parental income below $50,000 neg. while above $200,000 tapered off. GPA was lowest in engineering and sciences, and highest in arts and humanities 23

26 Table B1: Summary of Literature Documenting Association between Attendance and Grades (continued) Study/ Details Brown et al. (1999) Buckles and McMahon (1971) Type of Study Study Population Dependent Descriptive study Students in nursing Course grades. examined the courses at Humber All students relationship College. N=342 within a year between class nursing students of wrote the same attendance in a which 80 were first test, not withstanding nursing course year and 262 were having and its effect on second year different grades achieved teachers Experimental design with random allocation of students to one of two groups: one with, and the other without required class attendance Two introductory microeconomics course sections, taught by two different instructors at Vanderbilt University Test score on the final exam, based on the content of a programmed course text. Independent Class attendance. TUCE pre-test, number of hours studied, course load, high school rank, SAT score, GPA, educational status, dummy variables for gender, section, major, and lecture attendance Methodological Issues 4 teachers collected data from nine class sections involving three different nursing courses. A correlation coefficient was calculated for each of the 9 class sections. The impact of the different courses, instructors, and year, were not analyzed. Classes missed were recorded vs. hours missed Final regression equations included only 6 independent variables as those variables that were found to be unrelated to the dependant variable were removed Key Relevant Findings 4 of the 9 class sections found a significant negative correlation between absenteeism and grades, but 5 groups found no significant relationships. Range of absenteeism was 0-5 for students with 80+% (mean of 1.4 classes missed); students with grades 50% to 60% missed an average of 4.4 classes while students with grades 40% to 50% missed and average of 5.7 classes) Lectures which do no more than explain the material covered in assigned reading do not significantly improve student academic performance 24

27 Table B1: Summary of Literature Documenting Association between Attendance and Grades (continued) Study/ Details Didia and Hasnat (1998) Durden and Ellis (1995) Durden and Ellis (2003) Type of Study Study Population Dependent Modelled a 210 SUNY students Final course standard enrolled in 7 sections letter grade production of an intro financial function management course approach using offered in Fall 1994 questionnaire and and Spring 1995, institutional data taught by 4 faculty OLS regression study of survey questionnaire data Empirical study using an OLS Regression model on the experience of two different instructors Multiple instructor data from several introductory economics courses Principles of Economics classes N = 252 Student grades were normalized to a 10-point grading scale to minimize grading effects across different instructors Overall class average grade Independent Prerequisite grades, age, class standing, hours of study, gender, instructor dummy, and transfer status Self-reported data on absences entered as continuous data and as a dichotomous variable Class attendance, GPA and SAT scores Methodological Issues Used OLS regression as well as ordered-probit estimation (because of discrete dependant variable), and got similar results with each technique Used dummy variables to investigate possible impact of: MSAT, VSAT, race, calculus course taken, prior economics exposure, gender, high school preparatory program, extra-curricular activities, credit hours taken per semester, hours worked, and if from N. Carolina See Table B2 Key Relevant Findings Found strong positive relation between cumulative GPA and course grade. Also true for prerequisite courses. Weak relation between age and course grade Gender played no role. Surprisingly, found a marginally significant negative relation between hours studied on grades (1) Attendance effect is non-linear and mattered only after a student missed a threshold of four classes, with the size of the negative impact increasing with each additional absence. (2) As number of absences increased beyond 4 the negative impact on grades increases. (3) GPA and college-entrance exam scores (MSAT, VSAT) are among most important determinants of student academic performance; also calculus course had significant positive effect. Results suggest, rather strongly that motivation is an independent factor with regard to average scores earned. Some weak evidence that classroom attendance may serve as a proxy for the effects of internal motivation 25

28 Table B1: Summary of Literature Documenting Association between Attendance and Grades (continued) Study/ Details Emerson and Taylor (2004) Hanushek et al. (2003) Type of Study Study Population Dependent Regression of a 9 sections of a Used 33 production university questions on function microeconomics economics approach, course (900 students) portion of the modelling student of which 2 sections TUCE on the learning as (59 students) relied first and last f n (aptitude, on classroom day of classes educational experimental background, learning vs. educational traditional lectures environment, and instructor effects) Econometric study of peer group data. Tried to overcome problems of omitted variables and simultaneous equations biases through use of a fixed effects framework and lagged measures of peer achievement Makes use of a unique matched panel data set on students and schools to identify the impacts of specific peer group characteristics on academic achievements. Individual grades not specifically examined. Regressed average per achievement in a grade Independent Used measures of aptitude (e.g. GPA and SAT scores), major, student s prior high school economics course, gender, ethnicity, also used a dummy variable for the experimental learning group Separated peer influence into endogenous (behavioural) effects and exogenous or predetermined (contextual) effects, e.g. Family (race, socio-econ) and school variables Methodological Issues Both dependent variables are potentially subject to censoring problems. Also a potential positive selection bias exists. Pre-course TUCE score is considered as a proxy for pre-course aptitude, so used 2SLS vs. OLS. Instructor level effect difficult to measure Because members of peer groups tend to have similar experiences over time through systematic neighbourhood and school choice, many omitted historical factors will be common to the peer group. Many poorly measured, or omitted, contemporaneous inputs will also tend to be common to the group Key Relevant Findings Students in the experimental sections (59) did significantly better on TUCE economic scores than did the 241 students attending traditional lectures. The results also indicate that certain student characteristics, including gender, major, and grade point average, can be used to predict a student s likely success when choosing between courses that rely on experiments vs. traditional pedagogy Most important finding is that peer average achievement has a highly significant effect on learning across the test score distribution. 26

29 Table B1: Summary of Literature Documenting Association between Attendance and Grades (continued) Study/ Details Jaggia and Kelly-Hawke (1999) Johnson et al. (2002) Jones and Field (2001) Type of Study Study Population Dependent Econometric Grades 4, 8, and MEAP production students taking the test scores, function study Massachusetts placing using an ordered Educational students in one logit model Assessment Program of five (MEAP) outcome categories, analogous to letter grades OLS regression study to measure relation between student performance and effort ANCOVA-based study to evaluate impact of supplemental instruction on final grade performance. 70 students in an intro financial management course at a Mid Western business school, over two semesters 1,359 students in 9 classes of Principles of Accounting Final course grade Final course grades Independent 1) School input measures: teacher-pupil ratio, per pupil and administrative expenditures 2) socioeconomic measures: percent of single mothers, professionals/ managers, rental units, and crime rate 3)a dummy variable if an urban area College GPA, gender, ACT score Attendance in supplemental instruction offerings, gender, student s major and instructor. Methodological Issues Even though the underlying dependant variable is continuous, only the discrete responses are observed. Instead of self-reported data on student effort, measured effort by number of attempts and time spent on computer quizzes (maximum of 9) Controlled for selfselection by students. Two covariates (students SAT and prior GPA) were included to capture students aptitude and prior academic performance. Key Relevant Findings Results strongly suggest that higher levels of spending do not have any consistent or systematic relation with student performance. Smaller class sizes (teacher-pupil ratios) lead to better student performance only in the early stages of education Positive relation between performance and both measures of effort number of attempts and log time. No relation for gender. Noted a positive relation natural aptitude and ability (ACT score, GPA) and performance. Participation in both voluntary and mandatory supplemental sessions was found to positively associate with final grades. Notably, level of supplemental instruction attendance was a factor 27