Issues in Accounting Education Vol. 26, No. 4 November 2011 pp

Transcription

1 The American Accounting Association is the largest community of accountants in academia. Founded in 1916, we have a rich and reputable history built on leading-edge research and publications. The diversity of our membership creates a fertile environment for collaboration and innovation. Collectively, we shape the future of accounting through teaching, research and a powerful network, ensuring our position as thought leaders in accounting. Vol. 26, No. 4 November 2011 pp Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction When Taking the Introductory Financial Accounting Course? American Accounting Association Publications The American Accounting Association is the copyright holder of this article and retains the worldwide rights to publish, reproduce, distribute, publicly display and/or license the Material in this article for any future editions and versions of this Work, derivative works based on this article (including translations and adaptations), related ancillaries and supplements, and in promotional materials in all print, digital and wireless and/or electronic formats. Further distribution of this article is strictly prohibited. Written application must be made to the American Accounting Association, 5717 Bessie Drive, Sarasota, Fl , for permission to reproduce any of the contents of this article, other than for use in courses of instruction. For additional information about the AAA, please visit where you can also browse the abstracts of AAA journal articles.

2 ISSUES IN ACCOUNTING EDUCATION Vol. 26, No pp American Accounting Association DOI: /iace Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction When Taking the Introductory Financial Accounting Course? Ronald F. Premuroso, Lei Tong, and Teresa K. Beed ABSTRACT: Teaching and student success in the classroom involve incorporating various sound pedagogy and technologies that improve and enhance student learning and understanding. Before entering their major field of study, business and accounting majors generally must take a rigorous introductory course in financial accounting. Technological innovations utilized in the classroom to teach this course include Audience Response Systems (ARS), whereby the instructor poses questions related to the course material to students who each respond by using a clicker and receiving immediate feedback. In a highly controlled experimental situation, we find significant improvements in the overall student examination performance when teaching this course using clickers as compared to traditional classroom teaching techniques. Finally, using a survey at the end of the introductory financial accounting course taught with the use of clickers, we add to the growing literature supporting student satisfaction with use of this type of technology in the classroom. As universities look for ways to restrain operating costs without compromising the pedagogy of core requirement classes such as the introductory financial accounting course, our results should be of interest to educators, administrators, and student retention offices, as well as to the developers and manufacturers of these classroom support technologies. Keywords: audience response systems (ARS); clickers; interactive pedagogy; learning; online homework manager systems (OHMS); satisfaction. Data Availability: Contact the first author for the data used in this study. Ronald F. Premuroso is an Assistant Professor, Lei Tong is a Masters of Accountancy Graduate, and Teresa K. Beed is a Professor, all at The University of Montana. We appreciate comments and suggestions received from workshop and conference participants of the School of Business Administration at the University of Montana and the School of Business at Montana State University. We thank Professor Terri Herron for her comments and suggestions on earlier versions of this manuscript. We also thank students at the University of Montana for participating in this study, the School of Business Administration at the University of Montana for financial support provided for this study, the Advising Department of the University of Montana School of Business Administration, including Sue Malek and Tara Kirkham, for their help in the data collection required for this study, and Lingqian Jiang and Jessica Trethewey for their assistance in the final preparation of the manuscript. The experiment using human subjects described in this manuscript was approved by the Institutional Review Board for the Use of Human Subjects in Research at the University of Montana. We also thank Bill Pasewark (editor) and three anonymous reviewers for comments on a prior version of this manuscript. Published Online: November

3 702 Premuroso, Tong, and Beed INTRODUCTION Many higher institution education budgets are under pressure (and possibly for the foreseeable future) from a combination of shrinking state revenues, the growth of online and/or distance learning programs, reduced donor contributions, and the inability (in the current economic environment) to raise student tuition to cover ever-increasing operating costs. Simultaneously, the introduction of course-related classroom technology systems potentially allows colleges to consider larger class sizes for certain required courses in the curriculum without reducing the quality and pedagogy of such courses. The use of new classroom technologies may also create a more interactive and dynamic teaching-learning environment compared to the traditional lecture-style classroom. Therefore, combined use of sound pedagogy and/or technology can potentially improve student learning experiences and enhance their understanding of the related course material for courses in certain academic disciplines (Chan and Snavely 2009). A relatively recent innovation in the classroom is the use by instructors of Audience Response Systems (ARS), otherwise known simply as clickers. Clickers allow students to respond to instructor-posed problem sets or questions in the classroom. The system software immediately summarizes the answers submitted by all students and displays a graphical summary of the results that the instructor can display in the classroom. In principle, clickers should be beneficial to students by engaging them actively in the learning process and providing them with immediate feedback to answers to the problem sets or questions. Additionally, clickers also should benefit instructors, providing them with instant feedback about the class s comprehension of various core course concepts tested by problem sets or questions. Clicker systems can be implemented at a relatively low cost to both students and universities. For students, the cost of the clicker ranges from $30 $50 (less, if used). Most clickers are reusable and powered by commonly available batteries that generally last at least one semester. Suppliers of clicker-type technologies normally provide the instructor module and a USB drive for recording student clicker responses to questions posed in the classroom without cost. Also, no special hardware or software investment is required by the university to operate the clicker in the classroom as most clicker systems use commonly available computer hardware and software technologies. 1 Most universities require business majors to take up to two introductory accounting courses: an introduction to financial accounting course and an introduction to managerial accounting course. Typically, introductory accounting courses are taught using traditional lecture methods covering textbook concepts, a series of examinations, quizzes, homework assignments, and/or homework using a web-based online homework manager system (OHMS). It is important for educators and instructors to understand the potential impact of using the clicker technology, from a student perspective including student performance, in teaching these introductory accounting courses. We study the effectiveness of using clicker technology in the classroom on student performance, specifically for the introductory financial accounting course. Prior research generally shows a positive relationship between student performance on pen-and-paper homework assignments and student final grade performance in many college courses, including both 1 The authors acknowledge there is a wide variety of clicker classroom solutions available in the marketplace, with varying choices of business models that can be employed by the university and the instructor. The pricing strategy employed by i. Clicker, as described in this paragraph, is the exception, rather than the norm, in that many educational institutions may face high investments in hardware, software, and technical support if they pursue a campus-wide, clicker-type solution (compared to an individual classroom application).

4 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 703 introductory and upper-level accounting courses (Beed and Evans 2008; Rayburn and Rayburn 1999; Ryan and Hemmes 2005). Prior research also shows a generally positive relationship between student performance in the introductory accounting course and the use of web-based OHMS in a variety of courses (Berry 2009; Gaffney et al. 2010). Also, no studies exist that specifically address the impact of using the clicker on student performance in the introductory financial accounting course as compared to or combined with traditional teaching pedagogy in a highly controlled experiment. We accomplish the first objective of studying the effectiveness of clicker technology by analyzing student performance in the introductory financial accounting course, first using in-class quizzes and graded homework assignments and a web-based OHMS in one semester, and then using clickers as a substitute for the in-class quizzes and graded homework assignments in a subsequent semester, to determine if the use of the clicker matters to student performance in this course. We undertake a highly controlled experiment, whereby the same instructor teaches the same introductory financial accounting course in two different semesters, using the same course textbook, instructor notes, course examinations, and OHMS homework questions. The only differentiating factor between the two classes is the use of the clicker in the classroom as a substitute for the other traditional methods of stimulating student interest in the course materials (in our case, for the quizzes and homework assignments). The goal is to estimate the potential impact the clicker makes on student performance in the introductory financial accounting course by comparing student examination results in the two classes. Also, accounting studies seldom use multivariate analysis, including controlling for other factors, in evaluating the effectiveness of technology on student course grade performance in the introductory financial accounting course. 2 Most prior literature focuses on the results from either univariate or survey methodologies to evaluate the technology impacts from the clicker on student performance. In general, prior literature shows several factors contribute to a student s performance in accounting courses. We control for these factors in the multivariate models included in our study, including grade point average (GPA), gender, the student s declared major at the time of taking the course, as well as other factors, to better isolate the impact of clicker use on student grade performance in the introductory financial accounting course. Second, this paper provides feedback on a series of survey questions asking students to rate their experience with the clicker in the introductory financial accounting course. It is important to understand student perceptions at the end of the semester after using the clicker in the introductory financial accounting course, as this will inform educators considering the use of clickers in such courses. This study will help deans and other administrators evaluate the use of technology such as clickers to restrain operating costs in these challenging economic times while simultaneously not compromising course quality, related delivery, and teaching pedagogy in the classroom. The Office of Student Retention on campuses also should be interested in our results if changes in teaching pedagogy result in lower student failure rates in core courses such as the introductory accounting courses. Further, our results should be of interest to the developers and manufacturers of classroom support technologies, as they consider new or incremental product developments that could increase the pedagogical content of the future classroom delivery of courses, such as the introductory accounting courses, across the nation. Finally, our results should be of interest to all accounting instructors who are looking for ways to stimulate student interest in the classroom when teaching 2 We discuss, compare, and contrast later in this manuscript literature with regards to the use of multivariate analysis in the introductory managerial accounting course, a course that differs in both course content and topical coverage from the introductory financial accounting course.

5 704 Premuroso, Tong, and Beed the introductory accounting courses required of most business majors, while simultaneously potentially increasing the instructor ratings 3 they receive from students. Accounting instructors also can benefit from a number of other direct and indirect benefits of using clickers in the classroom, including the reduction of either instructor and/or student assistant time spent grading traditional pen-and-paper based quizzes and homework assignments and a reduction in errors when grading such student work. This manuscript is organized as follows: the next section reviews the literature related to both business and accounting course studies regarding clicker use in college classrooms. This section includes a brief history of the use of the clicker, how the clicker works functionally in the classroom, student attitudes toward the clicker, and relevant literature to date. We then develop hypotheses related to usage of the clicker in the introductory financial accounting course. Next we discuss in detail our controlled experiment involving the use of the clicker compared to traditional classroom methods. A discussion of the results of surveys performed in the class using the clicker, including the descriptive statistics for the introductory financial accounting courses using and not using clickers, is included. This is followed by the discussion of the results of the multivariate models. Finally, we summarize our conclusions, discuss limitations of our study, and outline areas for future study related to the use of clickers in the introductory accounting courses. CLICKER BACKGROUND INFORMATION, LITERATURE REVIEW, AND HYPOTHESES DEVELOPMENT Brief History of the Use of Clickers in the Classroom The use of clickers in college classrooms has grown extensively since the technology was introduced in the 1980s. For example, at the University of Colorado, 19 departments, 80 courses, and over 10,000 clickers were in use during the Spring 2007 semester alone (Keller 2007). One of the major suppliers of clickers (i. Clicker: see estimates the company has sold approximately 3 million clickers for use in the U.S. educational market alone. Virtually every academic discipline, including the sciences, arts, and business, has introduced various clicker systems into their classrooms at colleges and universities across the nation and the world (Zhu 2007). Instructors use clickers in the classroom for a variety of purposes, depending on the learning objectives of the course and the teaching methods and pedagogy employed by the instructor, including the following: To assess student subject knowledge on a topic currently being covered or recently covered in the class before moving on to the next subject; To evaluate students understanding of new materials about to be covered or assigned by the instructor before a certain class meeting; To initiate classroom discussions on new or difficult course-related topics; To record student class attendance and participation; To gather student feedback on instructional teaching methods; and/or To administer tests and/or quizzes during a lecture (Zhu 2007). 3 The instructor (who is one of the co-authors of this manuscript) received either statistically significantly higher (ssh) or similar (s) ratings from students taking the introductory financial accounting course in the class using the clickers compared to the class not using the clicker, in the following categories contained in the university s standard Course and Instructor Evaluation Form: Student Interest Level (ssh); Intellectual Challenge (s); Student Effort Level (s); General Course Quality (ssh); Instructor Availability (s); Instructor Explanations (ssh); Instructor Preparedness (ssh); and Instructor Effectiveness (ssh). These instructor evaluation results are also a robustness test of the results comparing student satisfaction using a clicker compared to not using a clicker in the introductory financial accounting course later in the manuscript.

6 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 705 The above list is certainly not exhaustive and also includes, potentially, a combination that the instructor may choose from among any of the above-mentioned uses. The only limitation on the innovative application of clickers in the classroom is the creativity of the instructor (Zhu 2007). What Is an Audience Response System (ARS) and, Consequently, a Clicker? The general type of ARS system available in the market today consists of three components: The clicker itself (see Exhibits 1 and 2) is a wireless handheld transmitter approximately the size of a small TV control transmitter. It can be used individually by each student to transmit, for example, an answer choice using primarily radio frequency transmission technology; The instructor s receiver module (Exhibit 2) is a relatively small transportable device, including an antenna (receiver) that receives each student s individual response to a problem or question posed by the instructor. The receiver module records the response on a removable USB storage device; and The software included in the receiver module allows the instructor to record and display, in graphical form in real time in the classroom, the total class-related student responses to questions posed by the instructor immediately after a question is asked. The instructor, EXHIBIT 1 Illustration of Use of the Clicker by Students in the Classroom Answering Instructor Questions

7 706 Premuroso, Tong, and Beed EXHIBIT 2 Illustration of the Instructor Module (Left) and the Student Clicker (Right) therefore, can evaluate the entire class s comprehension of any particular course concept posed during class, quickly and efficiently. After class, the instructor electronically transfers the student data captured in the USB storage drive to the gradebook contained in a learning management system (LMS). The software included in the instructor module leaves a detailed trail, including the answer submitted by each student and the correct answer for each question posed by the instructor during any particular class, as well as a copy of the actual question itself. Students can review their clicker grade performance throughout the semester on the LMS on a daily basis. If a student misses a particular class meeting or does not answer a clicker question (see Exhibit 3) during a class session, most clicker systems record the student as AB, or absent. 4 The advantages to the instructor and the university of using clicker technology include: Green benefits (no paper is required to administer classroom quizzes); Time savings (the instructor or the instructor s grader can save time compared to manually grading and posting quiz grades); and Increased posting accuracy (there is improvement in the accuracy of recorded student grades on quizzes, especially compared to manually grading and recording quizzes in a gradebook). 5 Student Attitudes toward Using Clickers in the Classroom Various surveys and research studies performed over the past 20 years have found both strengths and weaknesses regarding student attitudes toward using clickers in the classroom. 4 The authors acknowledge there is a wide variety range of features and functionality in clicker product solutions available today in the marketplace from various suppliers. The clicker solution utilized in this study from i. Clicker is regarded in the industry as a low-end, low-cost system with somewhat limited features and functionality. Other clicker solutions provided by other suppliers provide the instructor with other features and functionality, including (1) minute-by-minute tracking of respondent feedback during an event or presentation; (2) spatial or location-based types of questions (in addition to the usage of standard, multiple-choice types of questions); (3) team scoring on a real-time basis; and (4) many other additional features. In fact, i. Clicker has introduced, since the completion of this study, i. Clicker 2 in June 2011, which includes additional entry types, such as: True/False; Yes/No; Text Entry; Fill in the Blank; Rankings; Short Answers of up to 15 Characters; and Multiple Correct Responses. 5 These advantages were observed by the authors in their use of clickers during this study.

8 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 707 EXHIBIT 3 Example Financial Accounting Clicker Question and Graphical Classroom Question Results Displayed to Students Panel A: Example Financial Accounting Clicker Question Panel B: Graphical Classroom Question Results Displayed to Students In general, research shows students enjoy using clickers in the classroom because it makes the instructor s lecture both fun and interesting (Conoley et al. 2006; Duncan 2006; Stuart et al. 2004). At the same time, research shows that students are able to improve their understanding of both the related course content and the instructor s course expectations when using clickers in the classroom

9 708 Premuroso, Tong, and Beed (Zhu 2007; Tomorrow s Professor Blog 2006). Students also are more likely to respond to instructor questions and participate in classroom discussion when clickers are used (Greer and Heaney 2004; Hoffman and Goodwin 2006). In general, clickers have been shown to facilitate and enhance a student s learning experience in a variety of college subjects, ranging from the sciences (including physics, chemistry, and geology) (Fies and Marshall 2006) to business (including accounting) (Edmonds and Edmonds 2008), history, mathematics, psychology, sociology, and journalism (Conoley et al. 2006; Uhari et al. 2003). Other studies show changes in classroom dynamics result from the use of clickers, including higher rates of student class attendance, less sleeping in class, and greater student involvement in classroom activities, especially if the clickerposed questions were stimulating (Keller 2007). On the other side, negative complaints about the clicker from students include the clicker cost (Greer and Heaney 2004), the occasional encountering of technical problems with the clicker during classroom usage (Zhu 2007), more noise in the classroom (as students converse when questions are being presented by the instructor in the process of using clicker technology) (Keller 2007), and use of the clicker ruining the flow of a particular traditional lecture (Zhu 2007). Students also disliked clickers if the instructor required and used clickers in the classroom solely to take attendance or to record their answers on questions posed during class without any corresponding grade credit (Zhu 2007). Studies Related to Effects of Clickers on Measures of Student Learning in Business Courses, Including Accounting Bryant and Hunton (2000) appealed to researchers to investigate the pedagogical benefits of using technology to deliver instruction in the accounting classroom. Since then, several studies using a variety of research techniques, including survey and archival data, have attempted to address the effects of clickers on student learning in accounting courses. Carnaghan and Webb (2007) studied student self-reported perceptions of the learning effects of clickers, including using and not using clickers in four different introductory management accounting courses during various parts of one semester, and found clickers helped students learn the material. In addition, the summarized class answers, in the form of histogram charts presented by the instructor in class after each clicker question was posed by the instructor, helped students in the Carnaghan and Webb (2007) study track their personal progress in the course. Finally, the response to clicker questions in the Carnaghan and Webb (2007) study related to learned behaviors showed students felt clickers encouraged them to work hard as well as to come prepared for class. Carnaghan and Webb (2007) also tested objective measures of the learning effects of using clickers. As a proxy, they used the performance on exam questions in a limited regression model and found mixed results. Within the course itself, Carnaghan and Webb (2007) found an overall improvement in exam performance on course concepts covered with clicker questions in class, especially for high ability students (those with high GPAs entering the class). However, when they used a repeated-measures ANOVA to analyze their data, with clicker usage as the within-subjects factor and with ability and order of usage as the between-subject factors, Carnaghan and Webb (2007) found improvement in student performance was limited to exam questions that were similar to those questions employed when using the clicker only. Similarly, Cummings and Hsu (2007) find clicker usage in the tax accounting course improves student exam scores only where the underlying tax concepts are both taught and discussed specifically by the instructor in class using the clicker. Our study is a more stringent test of the pedagogical effects of clicker usage in the accounting classroom compared to such prior research, and does not include the use of clicker questions that are purposely similar, in any significant

10 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 709 manner or to any significant degree, to the examination questions given to students during the introductory financial accounting course. Cunningham (2008) reports on the use of action research, including the use of a clicker, to align an instructor s classroom activities or strategies with course goals in an elementary accounting course taught to ( primarily) sophomore students (similar to this study). Despite the fact that this study encountered technological problems with the clicker (problems that the current study did not encounter), overall, the course evaluations and questionnaire comments indicated students believed clickers enhanced the classroom environment, helped maintain student interest and focus, and provided immediate feedback both to the students and the professor. Edmonds and Edmonds (2008), in a highly controlled experiment performed similar to the current study on courses including introductory managerial accounting students, found that students in the courses using clickers outperformed students in the courses not using clickers after controlling for age, gender, prior GPA, and ACT scores. A more recent study by Chan and Snavely (2009) in the finance classroom found students were satisfied with using clickers and believed using clickers helped improve their examination performance. However, using various multivariate models and controlling for other confounding factors, Chan and Snavely (2009) found no difference in student examination performance between students using and not using clickers in the classroom. To our knowledge, no such study comparing students examination performance in classes using and not using clickers in a highly controlled experiment, including controlling for other confounding factors, has been performed in the introductory financial accounting course, a void this study will help fill. Theoretical Rationale for ARS Effects on Student Learning and Related Hypotheses The Conversational Framework, an influential theory developed originally in 1993 and upgraded and expanded by Laurillard (2002), envisions the learning process as an iterative dialog between student and teacher and describes how technology positively affects student learning in higher education courses. Interactive teaching, using technology products like the ARS used in this study, also is likely to encourage active learning and, therefore, influence student motivation in the classroom positively, if used correctly by the teacher. Students active engagement in the classroom, including being engaged with ideas and applications, supports student learning according to several other studies summarized by Draper et al. (2002). At the same time, Cue (1998) argues timely feedback and reinforcement are vital to the synthesis and integration process of learning for a student, which is another feature of the ARS. We hypothesize that the ARS will improve student motivation and, therefore, learning and overall performance in the introductory financial accounting course for several reasons. First, the biggest learning gains from using the ARS are likely to come as a result of the quicker student feedback from the teacher, the method of teaching in which questions are used as part of the classroom lecture, discussions in class initiated by well-designed questions, and the commitment of each student to an initial position on each ARS question before receiving the answer to the question (Draper et al. 2002). Second, although interactive teaching using ARS may not result in an increase in active student cognitive experiences automatically, students have been found to be engaged more mentally when involved in the classroom in learning activities such as problem solving (Van Dijk et al. 2001). Finally, the types of interactive teaching techniques involved when using products like the ARS generally create an atmosphere where true engagement by students is encouraged and supported by the teacher (Martin 1999). To summarize, the appropriate use of ARS in the classroom should improve interaction between the student and the instructor in the classroom. At the same time, the attempt by students to

11 710 Premuroso, Tong, and Beed solve problems and questions posed by the instructor using the ARS, and the immediate feedback provided by the instructor in the classroom to such problems and questions, should both enhance student satisfaction with the course itself and improve student performance in the introductory financial accounting class. This paper, therefore, tests two hypotheses: H1: Using clickers increases student satisfaction with an accounting course. H2: Using clickers improves student performance in the introductory financial accounting course compared to not using clickers in the same course, assuming all other factors remain equal. RESEARCH METHODOLOGY The controlled experiment of teaching two sections of the introductory financial accounting course, one with and one without the use of clickers, involved one of the authors over two different semesters (Spring 2009 and Spring 2010) 6 at one university. The introductory financial accounting course is a required course taken by all students majoring in business; it also is an elective course for other students attending the university. The course typically is taken during the student s sophomore year. The same course textbook, 7 syllabi, course schedule, OHMS 8 (including use of the same homework questions and problems in the OHMS), instructor lecture notes prepared by the instructor teaching the course, weekly class meetings and times (Tuesday and Thursday mornings), and multiple choice examinations (three examinations during the semester plus one department-wide common comprehensive final examination) were used in both semesters. 9 The only difference between the two classes was that students taking the Spring 2009 course took various unannounced pencil and paper multiple-choice quizzes in class and submitted various pre-assigned homework assignments during the semester at the request of the instructor on an unannounced basis, while students taking the Spring 2010 course were quizzed in class using the clicker as a direct substitute for the aforementioned quizzes and hand-graded homework assignments. 10 The instructor also used the same multiple-choice quiz questions and homework problems from the Spring 2009 course (translating the homework problem into clicker-type, multiple-choice questions) when teaching the Spring 2010 course using the clicker Both semesters were equal in length (16-weeks long) and exactly one calendar year apart. 7 The course textbook used in both classes was titled Fundamentals of Financial Accounting, 2nd edition, by Phillips, Libby, and Libby (2008); published by McGraw-Hill Irwin. 8 The OHMS used for the classes is the companion online Homework Manager product offered with the abovementioned Phillips, Libby, and Libby course textbook by McGraw-Hill Irwin. 9 There was no student in the 2010 class who took the same class with the same instructor in In the 2010 class, there were five students who took the same class with another instructor in Our results remain unchanged when excluding these five students from our results shown in Tables 1, 3, 4, and 5. Also, students were not allowed to retain or obtain copies of either the homework solutions or the course examinations as a matter of departmental policy for their permanent records. 10 Another minor difference was the Spring 2009 class consisted of two class sections meeting in the mornings, back-to-back, with an equal number of students in each class, whereas the Spring 2010 was one class consisting of all the students taking the course with the instructor. As will be seen later, the larger size of the Spring 2010 class did not affect the examination performance of students taking the class compared to students in the Spring 2009 classes in a smaller class size situation. This is also important to our results in this study as, counterintuitively, one may believe the larger class size might adversely affect student performance in such a course. 11 The 200 clicker questions used by the instructor in the introductory financial accounting course are available by contacting the author whose name is listed first on the title page of this article.

12 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 711 Description of How Clickers Were Used in the Classroom in This Study During the first week of class, the instructor explained to the students how clickers 12 would be used in the course during the semester. This included: How clickers would be incorporated into the classroom lectures; How much the clicker responses throughout the semester would count toward the student s total course grade that semester (the percentage weight toward a student s final grade for the class using clickers was the same as for the class not using clickers but having unannounced quizzes and graded homework assignments); and Instructions on how both to use the clicker in the classroom and how to register the clicker online at the supplier s website. 13 A course syllabus and a detailed course schedule were provided to each student on the LMS, advising them of the materials (consisting of textbook chapter readings and exercises and/or problems assigned in each chapter) to be read and/or prepared in advance of each class meeting. The instructor used clickers in the classroom starting the second week of classes and used the clicker to pose various computational-type problems and theoretical questions to students during every class meeting for the remainder of the semester, except for the scheduled course examination dates. The instructor scattered clicker questions throughout each classroom lecture: some at the beginning of class, some during class, and some questions at the end of class. The instructor prepared the clicker questions in Microsoft t Word beforehand. On a random basis, the clicker questions covered materials from the previous classroom lecture, the present classroom lecture, and any new materials included in that day s scheduled classroom lecture or assigned homework exercises and problems shown on the course schedule. The majority of the clicker questions consisted of multiple-choice type questions and five different answer choices. After each clicker question was posed and the polling time period for student answers for the question was closed, the instructor immediately provided the students with the correct answer to the question, including a bar graph showing the total performance of the class on that particular question. When student performance was perceived by the instructor to be low or not acceptable on a particular question (for example, more than 50 percent of the students selected an incorrect answer), the instructor discussed the question again in detail with the class. In the case of computational-related questions, the instructor immediately provided a detailed explanation of the correct answer in the classroom (the same explanations were provided throughout the semester for the class not using clickers for the quizzes and homework problems collected in that course). This form of immediate feedback to students is an important feature of the clicker technology and related interactive pedagogy. In the case of a question showing poor class results, the instructor also occasionally presented a similar question on the fly, immediately after explaining the correct answer to the previous question, to retest the comprehension levels of the class on challenging course-related concepts. Perhaps this is one of the reasons that clicker use has sometimes led to improved student performance. 12 The instructor and the students used clicker technology supplied by i. Clicker ( 13 It is important to note the majority of students using clickers in the introductory accounting course included in this study had previous experience with using clickers in some other college-level course at the university. The instructor posted instructions on Blackboard on how to use the clicker in class to assist those students with no experience using clickers in the classroom and to answer any questions students might have on the operation of the clicker in the classroom. The combination of having previous experience with the clicker and the instructions provided by the instructor on its use leads the authors of this manuscript to believe students knew how to use the clicker correctly in the classroom throughout the semester under study.

13 712 Premuroso, Tong, and Beed Students received credit for each clicker question answered correctly with no credit given for incorrect answers; for the class not using clickers, the same rules were applied in the case of the quizzes and graded homework assignments. After class, the instructor downloaded the day s clicker results into the LMS. 14 Thus, the credit students received for correctly answering the clicker questions essentially mimics the credit students received in the course not using clickers for correct answers on the quizzes and the graded homework assignments. During the last week of the semester in the class using the clicker, the instructor distributed a one-page survey in the form of a questionnaire consisting of 18 questions, asking students to rate usage of the clicker during the course on a scale of 1 (1 ¼ totally disagree) to 5 (5 ¼ totally agree). In addition, at the end of the questionnaire there was an open-ended question requesting students to list any other comments they had regarding the clicker use in the course. The analysis of these survey results was used to test H1. In order to test H2, we used a multiple regression model to examine the examination outcomes from using and not using the clicker technology in the classroom. There were 98 students in the class using clickers and, coincidentally, there also were 98 students in the class not using clickers. During the semester, four students dropped (withdrew) from each class; we address this in the robustness tests section below. Therefore, the final number of students included in both classes in the regression models is n ¼ 94. The advising office of the School of Business Administration helped us collect the required student demographic and academic data. We used the LMS to record student grades in both classes on the course examinations, the OHMS, the clicker grades from each class meeting for the class using the clickers, and the quiz and homework grades in the course not using clickers. Using ordinary least squares (OLS), we use the following multiple regression model to examine the effectiveness of using clickers in the introductory financial accounting course: where: Exam average ¼ k 1 þ k 2 Clicker averageðhw=quiz averageþþk 3 Semester credit hours þ k 4 GPA þ k 5 Gender þ k 6 Major þ k 7 OHMS average þ e; Exam average ¼ the student s average percent score on the four examinations; Clicker average ¼ the student s average percent score on the clicker questions; HW/quiz average ¼ the student s average percent score on homework and quizzes; Semester credit hours ¼ the student s total semester hours enrolled for the semester; GPA ¼ the student s GPA as of the semester the student took the course; Gender ¼ a dichotomous variable of 1 if the student is female, 0 otherwise; Major ¼ a dichotomous variable of 0 if the student is a declared business administration major, 1 otherwise; 15 OHMS average ¼ the student s average score on the question and problem sets performed in the online homework manager system; and e ¼ the random error term. ð1þ 14 This consists of downloading the day s clicker data from the USB drive into a column in the course LMS, a procedure that takes about one minute for the instructor to perform. It also is free of any clerical or unintentional errors that might occur in the case of the manual grading and posting of the results of quizzes or homework assignments. 15 At the university where this experiment was performed, students are not required to declare their major formally until completion of a lower core of classes. Until that time, students can make a general declaration of their major in the university s advising system, which can include BADM (a business administration major) or other disciplines outside of BADM, as well as undeclared.

14 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 713 Exam average (the average of each student s three examination scores and the comprehensive final examination score during the semester) is the dependent variable in each of the regression models and the measure we use to proxy for student performance in the introductory financial accounting course, similar to Chan and Snavely (2009). First, we estimate Equation (1) for the class of n ¼ 94 students using clickers (including the related k 2 Clicker average variable); then, we separately estimate Equation (1) for the class of n ¼ 94 students not using the clicker (including the related amount for the k 2 HW/quiz variable in the model). We expect to find a positive relationship between the k 2 variables in the separate models and the student exam average. Finally, we estimate Equation (1) in a pooled regression model for both of the classes combined (a total of n ¼ 188 students), including for k 2 Clicker the average variable for the students using clickers and zero for the students not using clickers, following Chan and Snavely (2009). The explanatory variables included in the models are based upon factors previously found to be associated with student performance in the introductory accounting and other types of business courses in both clicker-related regression models (Carnaghan and Webb 2007; Cummings and Hsu 2007) and non-clicker related regression models (Beed and Evans 2008). We expect to find a positive relationship between the GPA and the OHMS independent variables and the dependent variable, exam average. We expect to find a negative relationship between the Semester credit hours variable and exam performance. We have no expectations for the signs on the coefficients either for the Gender or the Major 16 control variables. Students Using Clickers Survey Results RESULTS Table 1 shows the results of the questionnaire distributed to students during the last week of the semester in the class that used the clicker. Out of 94 students taking the course, 83 (88.3 percent) voluntarily filled out the questionnaire. As can be seen in Table 1, the mean response for many of the positive survey questions was in excess of 4 on a scale of 1 to 5. Students felt the clicker questions practiced in class increased their understanding of the material (mean ¼ 4.40), better prepared them for the exams (mean ¼ 4.26), and improved their learning experience when they could see the correct answer immediately after each clicker question was presented in class by the instructor (mean ¼ 4.74). It also is important to note students said they were more likely to attend class because of the daily use of the clicker (mean ¼ 4.53). Using the clicker in every class meeting during the semester also helped students stay more focused during class itself (mean ¼ 4.08). Overall, students believed using a clicker was very effective in this particular accounting course (introductory financial) (mean ¼ 4.38). Student responses to some of the negative survey questions showed students disagreed with whether the clicker actually helped them during the course. Students indicated they did not agree that they spent less time preparing and studying for this class than usual because of the use of the clicker (mean ¼ 2.08). Also, students disagreed with the statement their grades were worse than 16 At the university where this experiment was conducted, students are not able to declare their specific major field of study until they complete their required lower core of studies (generally two years of general education requirements), which includes the introductory financial accounting class (typically taken some time during the second year of studies). However, students can declare before completion of their lower core of classes if their general course of studies will be Business Administration, undeclared (unknown), or a major outside of studies related to the business school. We control for the Major declared or not declared by each student accordingly in our model.

15 714 Premuroso, Tong, and Beed TABLE 1 Survey of Students Taking the Introductory Financial Accounting Course (n ¼ 83) Survey Questions Mean Median Std. Dev. 1. The clicker questions practiced in class increased my understanding of the course material. 2. I believe the use of the clicker increased my learning experience compared to other classes where my instructor did not use the clicker. 3. Practicing problems using the clicker in class prepared me better for the exams. 4. Seeing the correct answer after each clicker question was important to my learning experience. 5. The clicker questions were too difficult I believe using the clicker to solve accounting-type problems every class meeting is important. 7. I was more likely to attend class because of the daily use of the clicker. 8. I spent less time preparing and studying for this class than usual I believe my grade was worse than expected due to the use of the clicker. 10 I was satisfied with the number of clicker questions posed by the instructor during each class meeting. 11. Using a clicker every class meeting takes up too much class time. 12. I retained more of the lecture material as a result of using a clicker. 13. Using a clicker in every class helped me stay more focused during class. 14. Overall, I believe using a clicker is very effective in this accounting course. 15. I wish all of my instructors used a clicker to pose questions and work problems during class. 16. Earning 15% of my grade during the semester using a clicker is reasonable. 17. I believe the use of the clicker improved my final grade in this course. 18. The use of the clicker increased my interest in accounting Rating Scale: 1 ¼ Totally Disagree; 2 ¼ Somewhat Disagree; 3 ¼ Neither Agree nor Disagree; 4 ¼ Somewhat Agree; 5 ¼ Totally Agree. they expected due to use of the clicker (mean ¼ 1.93), and that using the clicker every class meeting took up too much class time (mean ¼ 1.69). Overall, the survey results indicate students seemed to be satisfied with using a clicker in the introductory financial accounting course, and also were satisfied with the manner in which the clicker was used as part of the interactive learning pedagogy utilized by the instructor in teaching the course, adding to the literature in this area and confirming H1. Table 2 shows a sample of the student open-ended comments regarding use of the clicker in the introductory financial accounting course. In general, students supplied the instructor with positive

16 Does Using Clickers in the Classroom Matter to Student Performance and Satisfaction? 715 TABLE 2 Representative Sample of Student Open-Ended Comments Regarding Use of the Clicker in the Introductory Financial Accounting Course (n ¼ 83) (Survey Question: Please list any other comments you have regarding clicker use in this class in the space below.) Student Positive Comments I used the clicker at the University of Colorado, and I think it is very beneficial to our understanding of the material. Once I got used to the speed required in order to answer the questions, I began to like the clicker. The clicker definitely put my thinking into higher gear and focused my attention more. I thought the clicker problems were a very effective way of teaching. Go Clicker! First time I used clickers; I thought it is a great idea and really think it helps with the exams. I think you have a good system going. Accounting is very complicated and working through problems can be very helpful. I really like the clicker because it helped me learn the material. Doing examples in class with the clicker is extremely helpful! I feel I would not have had a chance passing this class without the participation of the clickers. Clickers were very important and helped my grade in the course. Clicker is a good way to keep the class engaged in the lecture and inspires us to know the material prior to class. Student Comments Containing Some Type of Prefacing Words Sometimes I felt rushed by the clicker. I do not believe it is necessary every class to use clicker, but using clickers in class is very useful. Clickers make class somewhat impersonal, as no one can share answers. Just another expense to an already expensive class. I like the clicker, but I am slow sometimes and need more time to work out the math in the problems. Please give us some more time to think. It is too fast. For the longer problems or questions using clickers, sometimes I felt rushed. I think you go too fast on some clicker questions. Otherwise, high five! written comments about the use of the clicker in this particular course. The positive comments shown in Table 2 include students believed the clicker helped them in terms of learning and understanding the course materials and the course examinations in terms of preparation, kept them engaged in the class lectures, and raised their expectations of the final grade they expected to earn in the course. It should be noted, however, that some students wrote that the clicker was too expensive and sometimes they felt rushed when using the clickers to respond to the questions posed by the instructor in class. Descriptive Statistics Table 3, Panel A separately shows student performance on each part of the introductory financial accounting course requirements, including the mean, median, and standard deviation for each part, for the class using clickers and the class not using clickers. Panel A is broken down