USING AN ONLINE HOMEWORK SYSTEM IN AN INTRODUCTORY MANAGERIAL ACCOUNTING COURSE: SOME LESSONS LEARNED

Transcription

1 USING AN ONLINE HOMEWORK SYSTEM IN AN INTRODUCTORY MANAGERIAL ACCOUNTING COURSE: SOME LESSONS LEARNED John L. Haverty Associate Professor Accounting Department, Haub School of Business St. Joseph s University 5600 City Avenue Philadelphia, PA jhaverty@sju.edu

2 INTRODUCTION This paper reports on the introduction of an online homework system in a managerial accounting course, and tests whether or not the introduction of an online homework system has any impact on student evaluation of teaching. In addition, student and instructor reaction to the introduction are reported and some lessons learned are noted. The use of assigned homework problems and exercises in accounting has long been a part of most accounting courses in the curriculum (Rayburn and Rayburn 1999). Over the years, accounting instructors tried different methods to encourage and/or require students to do accounting homework. At times, homework would be collected and graded. At other times, homework was ungraded and simply reviewed in class. Sometimes solutions were provided in class, and sometimes solutions were made available to the students outside of class (Lindquist and Olsen 2007). While assigning homework was always valued by accounting instructors, the clerical effort involved in collecting and grading the homework and providing timely and meaningful feedback to the students became more burdensome as class sizes increased. Recent technology has provided instructors with a potentially attractive solution to this dilemma, the online homework system. The use of these systems has become more prevalent in accounting courses, particularly at the introductory level (Peng 2009), and are being simultaneously introduced in other disciplines as well (Fynewever 2008). Online homework systems allow professors to use internet technology to assign homework problems, coordinated with the assigned textbook, that students are able to complete online and are automatically graded. The instructor can control many aspects of the homework including Selection of problems Nature and timing of feedback Choice of algorithmic problems (each student has a similar problem with different numbers) Use of online quizzes and tests The use of an online homework system is a particularly attractive option given the current uncertain financial situation facing nearly all colleges and universities. Introductory accounting sections, both financial and managerial, are seeing increased class sizes as a result of financial strains. The potential cost savings in clerical support for instructors in the form of grading assistants and saving of instructor time appear to be quite large. Potential cost savings may be mitigated, however, by the instructor's reluctance to use this new technology due to perceived heavy start-up costs involved in the first-time use of an online homework system, and the possibility that student evaluations might suffer due to a possible negative student reaction to this new technology. TEST OF THE ONLINE HOMEWORK SYSTEM The online homework system was tested in three sections of a managerial accounting course taught in the spring semester of 2009 at a mid-sized urban private university. Increased

3 average class sizes in the first-year accounting course motivated the limited introduction of an online homework system. There were 18 sections of managerial accounting taught that semester, 17 of which served a traditional full-time day student population. There were 725 students registered in these 17 sections. At this university, managerial accounting is required for all business school students, and is normally taken during the second semester of a business student s first year, with its prerequisite, financial accounting, taken in the first semester. The section taught in the evening was excluded from this study since those students are normally older students working full-time and are chosen by a different admissions mechanism. The accounting department in this university has a long tradition of course coordination, consisting of a common syllabus for all sections with baseline homework exercises and problems assigned across all sections. Thus all 17 sections of managerial accounting were taught from a common syllabus and a common baseline set of homework problems. Of the 17 sections of managerial accounting, the three sections taught by the course coordinator used an online homework system as a mechanism to assign, check and grade homework. There were 135 students in these three experimental sections. Students selected the instructor and the section as part of their normal registration process. They were not aware that they would be using an online homework system when they registered for the course, but they were informed of it about a week before the course formally began. This was before the end of the add-drop period, when the students could change courses or sections as part of the normal process. No unusual add-drop activity was noticed during the course of this study. Other sections used the traditional ungraded homework system in which the homework was neither collected nor graded, but was reviewed in class. At this university, ungraded homework has been the traditional method of delivery of the managerial accounting course. ONLINE HOMEWORK SYSTEMS AND STUDENT EVALUATION OF TEACHING The standard student evaluations used in all courses at this university for a 4-year period ( ) were obtained. Students were asked to assess their agreement with a series of statements on a scale of 1 to 5, with 5 indicating the highest level of agreement. The four statements used in this analysis were: 1. The course appears to have been carefully planned. 2. Course objectives are being achieved. 3. Compared with other courses on this level carrying an equal amount of credit, the effort I put into this course is as much as in other courses. 4. Overall, I rate this instructor as a good teacher. The mean student responses on each of the four items were tabulated for the instructor teaching the experimental group, as were the mean student responses on the same four questions for all instructors in the accounting department, and are shown in Table 1.

4 Table 1 Student Evaluation of Teaching Results Experimental Group Compared to All Accounting Faculty Average Experimental Year 2009 The course appears to have been carefully planned. Experimental Scores Department Scores Course objectives are being achieved Experimental Scores Department Scores Compared with other courses on this level carrying an equal amount of credit, the effort I put into this course is as much as in other courses Experimental Scores Department Scores Overall, I rate this instructor as a good teacher Experimental Scores Department Scores NOTE: Students were asked to assess their agreement with a series of statements on a scale of 1 to 5, with 5 indicating the highest level of agreement.

5 The average means were calculated for the non-experimental years of the study, and shown in individual columns. The simple average of the years was calculated and is shown in the next to last column. The last column shows the average means for 2009, the year of the experiment. The effect of the use of the online homework system on student evaluations can be seen by comparing the average for the years when the online homework system was not used ( ) to the scores in 2009 when the system was introduced. These scores are shown in bold on Table 1. For all four items, the means in the year 2009, when the online homework system was used, are slightly higher than the average scores for the three years ( ) when the traditional non-graded homework system was used. This slight increase, however, should also be examined in the context of the student evaluations for the entire accounting department over the same time period. Table 1 also shows the evaluation scores for the entire accounting department on these same 4 student evaluation items. It can be seen that evaluation scores for the entire department were also improved in 2009 as compared to the average for the years This appears to be a history effect possibly resulting from a very stable core faculty who are gradually increasing their teaching skills over time. In summary, a slight increase in student evaluations was noted by comparing the average scores of the experimental instructor for three years prior to the introduction of the online homework system with that same instructor's evaluation scores the year of the introduction of the online homework system, on all evaluation items studied. This increase, however, was mitigated by an overall rise in similar measures during the same period for the overall accounting department. It is not possible to conclude that the introduction of the online homework system improved student evaluations of teaching, but no decline in student evaluations was noted. It appears that the savings in reduced clerical effort on the part of the instructor by using an online homework system can be achieved without suffering any reduction of student evaluation scores. IMPLEMENTATION OF THE ONLINE HOMEWORK SYSTEM: LESSONS LEARNED At the conclusion of the course, student responses to two open-ended questions were recorded. The questions were: 1. What did you like about the course? 2. What didn t you like about the course? No attempt was made to do a detailed quantitative analysis of the open-ended responses. Overall, student responses were overwhelmingly positive with a total of 88 positive comments and 19 negative comments. Many of the negative comments revolved around implementation issues, including the time the problems were due (students did not like a 7:00PM deadline, and frequently expressed a desire for a later submission deadline. Students were given three attempts on most assignments, and expressed a desire for more attempts). There were a few comments about the cost of the online homework system, the fact that it duplicated material already in the textbook, and a few negative comments about the complexity of the user interface, particularly the scrolling required between the data given in the problem and the place on the screen where they had to enter their responses.

6 A summary of selected student comments pertinent to the online homework system is shown in Table 2. These comments were selected because of their relative frequency, i.e. more than one student often made very similar comments. When reading this table, it is important to note that the number of positive comments far exceeded the number of negative comments. Table 2 Selected Student Responses to Open-Ended Questions in Student Evaluations of Teaching Experimental Group Selected Positive Responses Selected Negative Responses The homework manager was an The course cost a lot of money with excellent study tool for the tests and the book and so it was a lot more to a good way for students to stay on pay for it top of their work Homework manager difficult to The homework manager was good operate practice and really helped me boost The homework manager could be my grade confusing. I would rather not sit and I liked the homework manager do it on the computer because because it gave feedback right away everything is not always in front of instead of waiting for a teacher to you. have to grade the homework for I think the homework manager was everyone a total waste of money seeing that The homework manager is a nice all the questions were in the extra grade and forces you to textbook practice accounting The homework manager sucked It helped me practice the problems and learn what I was doing wrong right away. Homework manager problems were good review problems for tests and good grade boosters, especially since three tries were offered I never use the book, but I love the homework manager. It is great practice and the points help my final grade. Several issues arise from the last two positive comments in Table 2. Again, comments were only noted in this table if more than one student made them. Many students saw the online homework system as a grade booster, something like an extra credit assignment. It may be the grade boosting aspect of the online homework system that they liked, rather than the online homework system itself. With this in mind, instructors need to be careful to tie the online homework system assignments into a final grade, or else students will most likely not use it. Surprisingly, a negative aspect of the online homework system was evident in a number of the positive comments about the online homework system. A number of students indicated that the online homework system was so good in the practice and instant feedback it provided, that they did not read the book. The online homework system might be so attractive to the students and so tied into their final grade that they neglect other aspects of the course they feel

7 are less valuable. Instructors need to be aware of this possibility and insure that students who do read the book are rewarded in the short term via their final grade in the course. INSTRUCTOR EVALUATION OF THE ONLINE HOMEWORK SYSTEM Issues noted by the instructor teaching the course using the online homework system were that there was much up-front planning required, but that the publisher provided excellent technical support. There were no reported downtime issues with the online homework system. There were a few challenging times in the course, involving student registration with the online homework system s website. Students had to use exactly the same name as their official university name. Start-up was troubling, since students required much consultation to enter the first few problems. This problem was particularly difficult with a few of the lesser technically knowledgeable students. Instructors can also expect a flurry of last-minute s asking for help immediately before the problem is due. These can be difficult for a faculty member to respond to adequately. Overall, the instructor concluded that the online homework system was relatively easy to operate, and the extra time involved in the operation of the online homework system was less than the time spent reviewing the assigned homework in class. Also, it did eliminate the use of a teaching assistant. One negative aspect of the online homework system was that it did not support the use of Excel to solve the homework problems. A much-simplified spreadsheet that did not have any mathematical functionality and a calculator were provided as student support. This irritated students who were conversant with Excel. Since skills on Excel are considered a critical student skill at this university, this was considered a major shortcoming. ONLINE HOMEWORK SYSTEMS AND INSTRUCTIONAL COST Detailed cost data were not available to the researchers during the course of the study. A rough surrogate for the total cost of delivery was the cost of teaching assistants assigned to assist the instructor in the conduct of these classes. The instructor teaching the experimental group did not have a teaching assistant assigned. All other instructors were assigned a teaching assistant who was granted a small stipend for assisting these courses, largely in grading. While this does not provide conclusive evidence that the cost of delivering a course using an online homework system is less than the cost of the same course taught in the traditional fashion, it appears that it did successfully eliminate the need for a teaching assistant in these sections. In view of the fact that the instructor reported time-saving as a result of the use of the online homework system, the fact that no observable decline in student evaluations of teaching were noted, instructors should seriously consider the use of online homework systems in these constrained economic times. CONCLUSION This experiment showed that the use of an online homework system had no negative effect on student evaluation scores. It must be added that this experiment took place in a context of increasing class sizes from previous years. Also, the instructor introducing the online homework system had no prior experience with such systems. Student commentary was overwhelmingly positive, but this was not reflected in increased numerical scores on the

8 evaluation. Student comments indicated there were some implementation issues including the time the assignments were due, the cost of the system, the number of attempts permitted, and the necessity to do what they perceived to be too much scrolling during the completion of the homework. They appreciated the instant feedback and the opportunity to correct their mistakes. The instructor noted excellent technical support from the publisher, but some issues with student registration, immediately before an assignment was due, and noted the lack of Excel-like spreadsheet capability. Overall, the experience was positive, and the saving in clerical effort in the face of increasing class sizes was achieved without any decline in how students perceived the course. Instructors should consider the introduction of this technology.

9 REFERENCES Fynewever, H A Comparison of the Effectiveness of Web-based and Paper-based Homework for General Chemistry. The Chemical Educator 13 (4): Lindquist, T. M., and L. M. Olsen How much help, is too much help? An experimental investigation of the use of check figures and completed solutions in teaching intermediate accounting. Journal of Accounting Education 25 (3): Peng, J. C Using an Online Homework System to Submit Accounting Homework: Role of Cognitive Need, Computer Efficacy, and Perception. Journal of Education for Business 84 (5): Rayburn, L. G., and J. M. Rayburn Impact of Course Length and Homework Assignments on Student Performance. Journal of Education for Business 74 (6):325.