What Determines Academic Exam Scores in Accounting & Finance. Angela M. Cairns. Department of Law, Economics, Accountancy & Risk

Transcription

1 What Determines Academic Exam Scores in Accounting & Finance by Angela M. Cairns Department of Law, Economics, Accountancy & Risk Glasgow Caledonian University Glasgow, Scotland G4 0BA. Tel: August 2015 Key Words: supplemental instruction, prior accounting knowledge Abstract The relative impacts of each of six determinants of academic first year degree exam scores in financial accounting are compared holding the others constant. The results indicate that class seminar attendance, supplemental instruction, and prior accounting knowledge show the largest effects on exam scores. Address for all correspondence: Angela M. Cairns, Department of Law, Economics, Accountancy & Risk, Glasgow Caledonian University, Cowcaddens Road, Glasgow, G4 0BA, U.K. Tel Fax

2 Introduction Neoclassical economic theory informs us that the efficient production of any particular good or service requires the different input types to be combined in proportions such that the marginal physical product of each input (the change in output resulting from employing one more unit of a particular input type) divided by the unit cost of that input type is equalised over each input type used in its production. Stated more succinctly, efficient production requires the change in output to be the same for the last pound spent on each input. The problem of efficient input resource misallocation for private firms is simply, if painfully, resolved by the impartial selection process of the market. Private firms that stray far from efficient resource allocation for long periods in a competitive market environment are eliminated by take-over or bankruptcy. However, the public sector may be less exposed to market forces. For example, universities do not have private shareholders, whilst the government is a guaranteed customer. The university sector is characterised by a complex system of internal coordination and administration as well as internal and external market forces that interact via central administration, faculties, departments and individuals. Taken together, these features suggest greater latitude for long term non-optimal allocation of resources within the university sector, and correspondingly larger rewards from any low hanging fruit revealed by the identification of resource misallocation in this sector. The production function for university teaching output (exam scores) has multiple inputs. Basic economic theory indicates that efficient resource allocation requires the inputs to be combined in proportions that equate the marginal physical product divided by the input price between inputs. In other words there are likely high returns to universities that are able to identify the marginal cost per incremental exam score for each factor input and adjust the combinations accordingly. The empirical study in this paper attempts to identify, measure and compare the absolute and relative contribution of six determinants of first year academic exam scores in financial accounting at Glasgow Caledonian University over the three academic years 2011/12, 2012/13 and 2013/14. Prior research on academic performance in accounting There is a substantial literature on the effect of pre-university experience of accounting education on performance (for example, Rankin et al, 2003; Doran et al, 1991; Farley & Ramsay, 1988). Specific to Scotland, Mitchell (1985) found that first year students with a Higher grade in Accounting performed better in their quantitative exams than those without. A number of studies have examined relationships between single independent variables and university exam performance (Baldwin & Howe, 1982; Bergin, 1983; Loveday, 1993; Rohde & Kavanagh, 1996; Byrne & Flood, 2008). This group of studies, not discussed here, has general issues associated with missing variables and collinearity, such that it is not possible to isolate the effect of each independent variable holding other independent variables constant. Multivariate analyses include Alcock et al (2008) who examine the determinants of exam success in Business Studies but not in Accountancy, while studies by Eskew & Faley (1988); Gul & Fong (1983); Auyeung & Sands (1994); Tho (1994); Koh & Koh (1999) and Guney (2001) document multivariate analyses of accounting academic performance. However, their explanatory variables do not include class attendance hours and additional teaching hours offered. Schroeder (2001) uses pairwise comparisons and also checks for interactions using multiple regression to show the effects of each independent variable controlling for other independent variables. Bartlett et al (1993) carry out a

3 three year analysis, establishing that first year exam results are the best predictor of degree success but they do not consider student attendance. Keef (1988) carried out stepwise regression of high school accounting, mathematics and economics, identifying that prior exposure to these subjects is beneficial. Jackling & Anderson (2010) use ordinary least squares regression to consider students varying entry paths to tertiary education, including study mode, finding that part-time students outperform full-time. Duff (2004) focuses on a number of independent variables with particular emphasis on the learning approach. He concludes that prior academic achievement remains the best predictor of academic performance. However the effects of supplementary classes and actual student attendance are not considered. The Model As stated earlier, the empirical study in this paper attempts to identify, measure and compare the absolute and relative contribution of six determinants of first year academic exam scores in financial accounting at Glasgow Caledonian University over the three academic years 2011/12, 2012/13 and 2013/14. Exam scores for i th student, TOTAL i are modelled as linear functions of the following six variables: 1. UCAS i Student ability measured by UCAS score 2. ATT i Student attendance 3. PREVACC i Prior student exposure to accounting and finance 4. PSUPP i Proportion of supplementary classes attended offered to students without prior exposure to accounting and finance 5. TQUAL y Teaching quality for year y 6. EDIFF y Exam difficulty for year y The Data Children in Scottish secondary schools typically study standard grades for two years, sitting their exams at the end of fourth year. After a further year of study, the majority of children undertake Highers in fifth year and occasionally some children take Advanced Highers in sixth year. Some children in sixth year of school crash their Higher, which involves sitting the exam after one year of study, often skipping the knowledge learned at standard grade. TOTAL - During the semester students are required to undertake three elements of coursework. The marks from these were not considered when calculating the final student exam score which was used for the dependent variable TOTAL. UCAS - Standard UCAS points tariffs were used, with an A at Higher counting for 180 points, a B, 165 and a C, 150. Students with A levels or other qualifications were allocated with the appropriate UCAS pointage. Glasgow Caledonian University gives credit for creative Higher subjects, such as music and art, as well as standard academic subjects such as maths or English. ATT - There are usually around 100 students in the first year accounting course. Three years of student data yielded 291 observations. No lecture attendance records were kept as lecture attendance was not deemed to be compulsory. Therefore the register of weekly seminar attendance was used to construct the variable ATT. The maximum number of seminar attendances was 10 hours. However, 47 of the 291 seminar attendance records were unobtainable. In order to avoid reducing the number of observations, the seminar attendance variable ATT was measured as a categorical variable scored as 0 for less than or equal to 50% seminar attendance, and 1 for more

4 than 50% seminar attendance. The 47 missing values were assumed to have attended more than 50% of the seminars and ATT was set equal to unity for these students. PREVACC - During the three academic years in question, students prior accounting experience (PREVACC) was classified by those who had studied standard grade accounting, those who had crashed a Higher in accounting and those with no prior accounting qualifications from school. Students who did have prior accounting knowledge but not Scottish school qualifications were allocated a PREVACC score according to their length of study and subject coverage. Students with no prior accounting knowledge were allocated a PREVACC score 0, students with a crash higher in accounting a 1, i.e. with some prior knowledge and students with a standard grade in accounting were allocated a 2 for a significant level of prior accounting exposure. PSUPP - Students with either no prior accounting knowledge or a crash Higher in accounting were invited to attend supplementary classes in accounting. These were generally one-hour weekly classes that were held in addition to three weekly lectures and a seminar. Additional questions to be covered were detailed in the module handbook, with students required to attempt some questions before attendance at the supplementary class. The lecturer circulated around the class seeking to resolve student areas of difficulty. If there appeared to be a general problem, then the class was given guidance as to how to commence the problem. There are twelve teaching weeks in a semester, with the maximum possible supplementary class attendance being ten weeks. These classes were timetabled and class registers were kept. Students who had a standard grade in accounting were advised that they were not required to attend but were welcome to if they so desired - some students took up this invitation. PSUPP for each student in each of the three years is measured as the proportion of supplementary classes attended expressed as a proportion of the number of supplementary classes offered. TQUAL - Towards the end of teaching at the end of the semester, students were issued with a questionnaire that sought to evaluate the students learning experiences, with one of the questions asking students to grade the lecture and seminar performance on a five-point Likert scale. EDIFF - The same lecturer taught the course over the three years in question and tutored three seminar classes from five per annum, as well as running all the supplementary classes. An experienced financial accounting seminar tutor was used for the first two years but a different seminar tutor, again with reasonable financial accounting experience, was used in the third year due to the death of the previous tutor. An important implication of this was that the moderator for the third academic year s exam paper was different from the moderator of the first two. The change in the moderator is recognised by the exam difficulty variable EDIFF which is measured by a crude categorical variable set to zero for 2011 and 2012 but unity for 2013 when there was a change of exam moderator, because the 2013 exam was more difficult. Table 1 below shows the descriptive statistics summary used in this study.

5 Table 1 TOTAL UCAS ATT PREVACC PSUPP EDIFF TQUAL Mean Median Maximum Minimum Std. Dev Skewness Kurtosis Jarque-Bera Probability Observations 291 Results The multiple regression estimation equation was: TOTAL = a+b.ucas+c.att+d.prevacc+e.psupp+f.tqual+g.ediff+ɛ Parameter estimates are shown with T-statistics below in brackets. TOTAL = *UCAS+19.13*ATT+13.17*PREVACC+12.70*PSUPP-0.81*TQUAL-9.75*EDIFF (0.01) (3.4) (4.7) (10.7) (4.4) (0.1) (3.5) R 2 =0.45 N=291 Analysis Malm et al (2011) found that supplementary instruction (SI) had a very large positive impact on first year academic success. 79% of students with high SI attendance received a pass grade in the first academic year course compared with 39% of the students who did not attend any SI sessions. Our study concurs with respect to the importance of supplemental instruction. The parameter estimates for this study show the estimated effect of a unit increase for each independent variable on TOTAL exam score holding the other independent variables constant. The estimated effect of full attendance at all SI sessions offered was to increase the total exam score (TOTAL) by 12.7 percentage points. This is a very large impact considering that the mean exam score shown in Table 1 was A number of other studies on the effects of supplemental instruction in other subject areas, Wright et al (2002), Webster and Hooper (1998), McCarthy et al (1997), Congos and Schoeps (1993), Blanc et al (1983) all suggest that SI has a positive effect on student performance, no matter if in maths, chemistry or engineering. More specifically, Howitt and Harding s (2000) accounting study found that SI introduction into a first year accounting course in an Australian university had a positive effect on student performance. Ogden et al (2003) considered the short- and long-term impact of SI over time, finding that the effects of SI were significant in the short-term but not as long lasting as expected. However, it is important to note that all these studies, including the current study, highlight the problem of attribution bias caused by the correlation between supplementary instruction attendance and student motivation, given the self-selecting nature of students attending SI.

6 In our study, multiple regression analysis permits the examination of each separate effect holding the others constant, and also permits the calculation of equivalent swap rates between the separate effects. For example, full attendance at SI increased TOTAL exam score by The coefficient on UCAS score indicates that an equivalent increase in 12.7 could have been achieved by having students whose UCAS scores were 282 points higher. Attending less than 50% of the seminars lowered the TOTAL exam score by 19.13, suggesting that seminar attendance is on a par with SI. Prior knowledge of accountancy was also important. The students were given a score between zero (no previous experience) and two (previous training in accountancy). Thus students who had previously studied accountancy at school had TOTAL scores raised by over 25 percentage points. The returns to attracting students with previous accountancy experience are clearly very high. Teaching quality was rated on a five point Likert Scale from 1 (excellent) to 5 (poor). The regression coefficient suggests that good teaching alters TOTAL score by 4 percentage points. Exam difficulty was indicated by a dummy variable for 2013 when a new exam moderator changed the standard of the exam, resulting in TOTAL exam score being lowered by 9.75 percentage points. (This effect was treated as being unfair to the students, and exam grades were subsequently adjusted with an administrative note that future exams be carefully screened when there is a new moderator.) Discussion We have shown at least in principle how resource misallocation in the university teaching production function can be revealed. The extent of this misallocation may be calculated by dividing each parameter estimate by the marginal cost of one incremental unit of that factor input. The extent of input misallocation is revealed by the extent to which the marginal physical products divide by the marginal costs are dissimilar. The purpose of this paper is to describe a procedure to identify and quantify resource misallocation in the university teaching production function. The reasons for why we do not continue to produce actual quantitative estimates of misallocation are listed below. First we do not have the marginal costs for each of the independent variables. Second, there are some data issues that would obscure the calculation. For example, each student s score for teaching quality cannot be assigned to that student because the teaching quality survey is anonymous. The data used in the analysis for teaching quality was the mean teaching score over all students for that year, measured by a 1 to 5 Likert scale from excellent (1) to bad (5). The negative coefficient on TQUAL simply indicates that a lower score on the Likert scale indicates better teaching. However the insignificant t statistic on TQUAL is more likely to reflect the lack of individual student data for this variable. Third, the exam difficulty variable EDIFF is measured by a crude dummy variable set to zero for 2011 and 2012 but one for 2013 when there was a change of exam moderator, such that the 2013 exam was more difficult. Conclusion We commend the adoption of multiple regression techniques in order to minimise the problem of closely correlated confounding variables, and to identify and measure the separate effects of each type of input on the teaching output academic exam scores, holding other factors constant. This

7 analysis concludes that there are strong returns to insistence on high class lecture/seminar attendance possibly via a class register with a minimum attendance ratio requirement to be allowed to sit the degree exam, supplemental instruction being offered, and also making a strong effort to attract students with previous experience of learning accountancy. References Alcock, J., Cockcroft, S. and Finn, F. (2008) Quantifying the advantage of secondary mathematics study for accounting and finance undergraduates Accounting & Finance, 48, Auyeung, P.K. and Sands, D.F. (1994) Predicting success in first-year university accounting using gender-based learning analysis, Accounting Education: an international journal, 3, 3, Baldwin, B.A. and Howe, K.R. (1982) Secondary-level study of accounting and subsequent performance in the First College Course, The Accounting Review, 57 (3), July, Bartlett, S., Peel, M.J. & Pendlebury, M. (1993) From fresher to finalist: a three year analysis of student performance on an accounting degree programme, Accounting Education: an international journal, 2, 2, Bergin, J.L. (1983) The effects of previous accounting study on student performance in the First College-Level financial accounting course, Issues in Accounting Education, 1 (1), Blanc, R.A., DeBuhr, L.E., and Martin, D.C. (1983) Breaking the Attrition Cycle: The Effects of Supplemental Instruction on Undergraduate Performance and Attribution The Journal of Higher Education, Volume 54, Number 1, Byrne, M. & Flood, B. (2008) Examining the relationships among background variables and academic performance of first year accounting students at an Irish University Journal of Accounting Education, 26, Congos, D.H. and Schoeps, N. (1993) Does supplemental instruction really work and what is it anyway? Studies in Higher Education, Volume 18, Number 2, Doran, B.M., Boullion, M.L. and Smith, C.G. (1991) Determinants of student performance in accounting principles I and II, Issues in Accounting Education, 6 (1), Spring, Duff, A. (2004) Understanding academic performance and progression of first-year accounting and business economics undergraduates: the role of approaches to learning and prior academic achievement Accounting Education, 13, 4, Eskew, R.K. and Faley, R.H. (1988) Some determinants of student performance in the First College- Level financial accounting course, The Accounting Review, 63 (1), January, Farley, A.A. and Ramsay, A.L. (1988) Student performance in first year tertiary accounting courses and its relationship to secondary accounting education, Accounting and Finance 28 (1), May Gul, F.A. and Fong, S.C.C. (1993) Predicting success for introductory accounting students: some further Hong Kong evidence, Accounting Education: an international journal, 2, 1,

8 Guney, Y. (2011) Exogenous and Endogenous Factors Influencing Students performance in Undergraduate Accounting Modules Accounting Education: an international journal, 18, 1, Keef, S.P. and Hooper, K.C. (1991) Prior accounting education and performance in first level university course in New Zealand, Accounting And Finance, 31 (1), Koh, M.Y. and Koh, H.C. (1999) The determinants of performance in an accountancy degree programme, Accounting Education: an international journal, 8, 1, Jackling, B. and Anderson, A. (1998) Study mode, general ability and performance in accounting: a research note, Accounting Education: an international journal, 7, 1, Malm J., Bryngfors L. and Mörner L. (2011) Supplemental Instruction: Whom Does it Serve? International Journal of Teaching and Learning in Higher Education, Volume 23, Number 3, McCarthy, A., Smuts, B. and Cosser, M., Assessing the effectiveness of supplemental instruction: A critique and a case study Studies in Higher Education, Volume 22, Number 2, Ogden, P., Thompson, D., Russell, A. and Simons, C., Supplemental Instruction: Short- and long-term impact Journal of Developmental Education, Volume 26, Number 3, 2-4, 6, 8 Rankin, M., Silvester, M., Vallely, M. and Wyatt, A. (2003) An analysis of the implications of diversity for students first level accounting performance Accounting and Finance 43, Rohde, F.H. & Kavanagh, M. (1996) Performance in first year university accounting: Quantifying the advantage of secondary school accounting, Accounting and Finance, 36 (2) Schroeder, N.W. (1986) Previous Accounting Education and College-Level Accounting Exam Performance Issues in Accounting Education, 1, 1, Tho, L.M. (1994) Some evidence on the determinants of student performance in the University of Malaya introductory accounting course, Accounting Education: an international journal, 3, 4, Webster, T.J. and Hooper, L. (1998) Supplemental Instruction for Introductory Chemistry Courses Jounal of Chemical Education, Volume 75, Number 3, Wright, G.L., Wright, R.R. and Lamb, C.E. (2002) Developmental Mathematics Education and Supplemental Instruction: Pondering the Potential Journal of Developmental Education, Volume 26 Number 1, 30-35