The Impact of Instruction Mode on Student Performance in Graduate Financial Management

Transcription

1 The Impact of Instruction Mode on Student Performance in Graduate Financial Management Neil Terry, West Texas A&M University, USA James K. Owens, West Texas A&M University, USA John W. Cooley, West Texas A&M University, USA ABSTRACT This paper presents empirical results concerning the effectiveness of campus, online, and interactive television (ITV) instruction modes in business education. The sample is derived from graduate students enrolled in a graduate financial management course at a midsized regional university in the Southwestern region of the United States. The results indicate grade point average, standardized admission test score (GMAT), effort measured by homework grade, and nationality are the primary determinants of student performance. No statistical difference in student performance on a final comprehensive exam was found between the campus, online, and ITV instruction modes. INTRODUCTION There is little doubt that technological changes have fundamentally altered the way higher education is delivered. Online and web-enhanced courses have become ubiquitous in higher education. In recent years, the efficacy of the online instruction mode has been debated in the literature (Lezberg, 1998; Okula, 1999; Bowman, 2003; Worly & Dyrud, 2003; Fortune, Shifflett, & Sibley, 2006; Terry, 2007). One alternative to online instruction is the resurgence of interactive television (ITV) as an instruction mode. The ITV mode has been around for over thirty years but technological advances in recent years has improved the clarity and interaction of the instruction mode. The purpose of this paper is to compare student performance in the campus, online, and ITV instruction modes in business education. The research is based on multiple sections of a graduate course in financial management at a regional university. This manuscript is organized as follows: First, an overview of concepts and definitions important to distinguish the three instruction modes is provided. The second section describes the data and model. The next section offers empirical results for the determinants of performance on a common comprehensive final exam. The final section offers a summary and conclusions. DEFINING CAMPUS, ONLINE, AND HYBRID MODES OF INSTRUCTION The fundamental characteristics of the campus, online, and ITV instruction modes are not universally agreed upon. A generic description of each instruction mode is put forth in this section in order to facilitate the research process. Campus-based or traditional instruction is probably the easiest to understand. The campus mode is characterized by student/faculty interaction via lectures, discussion, and exams on campus at scheduled times and days. There is approximately forty-five contact hours associated with a three credit hour course in most traditional campus courses. The personal interaction 148

2 between students and faculty associated with campus courses is often perceived as a characteristic that facilitates high quality learning. In addition, most professors were educated via traditional campus instruction and are very comfortable with the learning environment. The online mode of instruction replaces the walls of the classroom with a network of computer communication. Some of the benefits of online instruction are its temporal, geographic and platform independence, and its simple, familiar and consistent interface (Kearsley, 1998; Okula, 1999). Some of the drawbacks are: sophistication and creativity restricted by hardware and software compatibility; resistance to shift to new and alternative teaching and learning paradigms; privacy, security, copyright, and related issues; and a lack of uniform quality (McCormack & Jones, 1998). Online instruction is heralded for providing flexibility for students in that it reduces the often-substantial transaction and opportunity costs associated with traditional campus offerings. This flexibility in structure is countered by potential problems including lack of personal interaction (Fann & Lewis, 2001), the elimination of a sense of community (James & Voight, 2001), and the perception of lower quality. In addition, faculty often have reservations about preparing a new online course because of the large initial time investment involved (Terry, Owens, & Macy, 2000). Not all students can take campus courses and not all want online instruction. The general problem with campus courses for working professionals is the time and travel constraints. In contrast, the most common complaint about online courses is the lack of personal interaction between students and professor that is often needed to facilitate the learning process, especially for advanced coursework. Interactive television is a possible solution to the time and distance constraints associated with campus courses without compromising the personal interaction associated with online courses. Modern ITV delivers the same course in multiple locations with the instructor and students able to see and hear each other in real time with assistance of high-definition television. The sending location typically involves the instructor providing either a traditional lecture or a lecture accompanied by computer generated materials or by the use of a document camera, which can serve as a white board to write upon or to illustrate, for example, the use of a calculator. Students in the same room are able to view the identical illustrations being provided to the students in the remote classroom. Each student has a microphone, which they can press to ask a question that is heard by everyone in both classrooms. Some advantages of the ITV format include the following: (1) Student convenience in the sense of being able to attend class in either location; (2) The class is live, which is beneficial to auditory learners who may not find the typical online course appropriate for their style of learning; (3) The ability of the instructor to broadcast from any location, thereby providing more of a sense of class community for the students; (4) The need to be extremely organized in the preparation and presentation of material that is largely done in an electronic format; and (5) Exams are taken in the real rather than electronic world, which seems less intimidating to students. Some of the disadvantages of the ITV format include the following: (1) The need to be extremely organized in the preparation and presentation of material that is largely done in an electronic format that is not easily conducive to ad hoc materials; (2) The technology is such that instructors are limited in their ability to move around the room freely since they will quickly move out of camera range; (3) Some students simply feel better when the professor is physically present and have a tendency to move back and forth between locations as the instructor moves, if possible; (4) Offering courses in multiple locations makes it is possible to dramatically increase the size of the classes with resultant demands on faculty; and (5) The technology is relatively expensive and constantly being improved, as a result, ITV is a continuous investment in both learning time and finances. 149

3 All collegiate business programs are tasked with the ongoing need for assessment (Bagamery, Lasik, & Nixon, 2005; Martell & Calderon, 2005; Trapnell, 2005). Traditionally, accrediting bodies have focused primarily on input measures (Peach, Mukherjee, & Hornyak, 2007). Input measures should reflect characteristics of the students who attended the business program (Mirchandani, Lynch & Hamilton, 2001) or organizational factors such as the institution s reputation, faculty-student ratio, or number of faculty with terminal degrees (Peach, Mukherjee, & Hornyak, 2007). For collegiate business programs aspiring to meet or maintain the standards of accreditation, this requires the schools of business have program learning goals and utilize direct measures that reflect student demonstration of achievement of these goals (Martell, 2007; Pringle & Michel, 2007). It has been argued that graduate schools of business do a poor job of educating students in key managerial skills (Ghoshal, 2005; Pfeffer & Fong, 2002). This research contributes to the literature by assessing performance of students in the campus, online, and ITV instruction modes by focusing on student performance on a common comprehensive exam in a financial management course at a public institution in the Southwestern region of the United States in the years The financial management course is selected because it is a key course for most graduate business programs because of the applied concepts, which include time value of money, rate of return, cost of capital, firm valuation, mergers, and risk. DATA AND MODEL The purpose of this section is to develop an empirical model that can test student performance. Davisson and Bonello (1976) propose an empirical research taxonomy in which they specify the categories of inputs for the production function of learning. These categories are human capital (admission exam score, GPA, discipline major), utilization rate (study time), and technology (lectures, classroom demonstrations). Using this taxonomy, Becker (1983) demonstrates that a simple production function can be generated which may be reduced to an estimable equation. While his model is somewhat simplistic, it has the advantage of being both parsimonious and testable. A number of problems may arise from this research approach (Chizmar & Spencer, 1980; Becker, 1983). Among these are errors in measurement and multicollinearity associated with demographic data. Despite these potential problems, there must be some starting point for empirical research into the process by which business knowledge is learned. The choice as to what demographic variables to include in the model presents several difficulties. A parsimonious model is specified in order to avoid potential multicollinearity problems. While other authors have found a significant relationship between race or age and learning (Siegfried & Fels, 1979; Hirschfeld, Moore, & Brown, 1995), the terms are not significant in this study. A number of specifications are considered using race, age, work experience, and concurrent hours in various combinations. Inclusion of these variables into the model affected the standard errors of the coefficients but not the value of the remaining coefficients. For this reason they are excluded from the model. University academic records are the source of admission and demographic information (Maxwell & Lopus, 1994). The model developed to analyze student learning relies on a production view of student learning. Assume that the production function of learning financial management concepts can be represented by a production function of the form: 150

4 (1) Y i = f(a i, E i, D i, X i ), where Y measures the degree to which a student learns, A is information about the student s native ability, E is information about the student s effort, D is a categorical variable indicating demonstration method or mode, and X is a vector of demographic information. As noted above, this can be reduced to an estimable equation. The specific model used in this study is presented as follows: (2) SCORE i = B 0 + B 1 GMAT i + B 2 GPA i + B 3 HW i + B 4 FOREIGN i + B 5 FEMALE i + B 6 ONLINE i + B 7 ITV i +u i. The dependent variable used in measuring effectiveness of student performance is score (SCORE) on a comprehensive final exam. The variable associated with the final exam score is measured in percentage terms and serves as a proxy for measuring student knowledge of financial management concepts. Every effort was made to keep the content and course requirements consistent across the three instruction modes in order to make multiple comparisons viable. The descriptive statistics in Table 1 reveal the average score on the comprehensive final is 85.19, with a maximum of 100 and a minimum of 48. Fifty percent of the student grade is determined by the final exam score for this study. Table 1: Summary Statistics Variable Mean Maximum Minimum Std. Deviation SCORE GMAT GPA HW FOREIGN FEMALE ONLINE ITV Notes: n = 96. The student s innate academic ability is measure by the GMAT admission exam. The average GMAT score for the research cohort is 501, with a maximum of 690 and a minimum of 320. A second control variable for academic ability is grade point average (GPA). Average grade point average for the cohort is 3.52, which is relatively high but reflective of a sample derived from graduate students. The percentage score on the homework assignments (HW) measures student effort. The homework grade is used to measure effort since students are not constrained by time, research material, or ability to ask the course instructor questions when completing course assignments. Twenty-five percent of the student course grade course is determined by homework assignments. The demographic variable FOREIGN is included in the study to separate international students from domestic students. Foreign business students that are in the United States on a student visa are required to enroll in at least six hours of campus courses each long semester and do not have the option of enrolling exclusively in online courses. Thirty-six percent of the research cohort is classified as a foreign student. The variable FEMALE is included in the model based on the finding of previous researchers (Bagamery, Lasik, & Nixon, 2005; Black and Duhon, 2003; Mirchandani, Lynch, & Hamilton, 2001) that male student performance on business examinations is higher than female students. The research cohort is almost evenly split along gender lines, with 49 percent of the research cohort represented by female students. Enrollment in a campus, online, or hybrid course is noted by the categorical variables NET (online course) and ITV. Enrollment of the 96 graduate business students in the three instruction modes is 33 percent in the online mode, 28 percent in the ITV mode, and 39 percent in the campus mode. 151

5 RESULTS Results from the ordinary least squares estimation of equation (2) are presented in Table 2. The total usable sample size is 96, with 9 students eliminated from the global sample of 105 because of incomplete information, usually relating to not taking the final exam (Douglas & Joseph, 1995). None of the independent variables in the model have a correlation higher than 0.61, providing evidence the model specification does not suffer from excessive multicollinearity. The equation (2) model explains over 53 percent of the variance in final exam performance. Four of the seven independent variables in the model are statistically significant. Table 2: Estimation of Equation (2) Variable Coefficient t-statistic Intercept GMAT * GPA * HW * FOREIGN * FEMALE ONLINE ITV Notes: R-square =.5323, F = 14.47, *p<.05, and n = 96. Ability as measured by the variables GMAT and GPA have a positive and statistically significant impact on final exam performance. The statistically significant impact of standardized entrance exam scores and grade point average is consistent with previous research (Mirchandani, Lynch, & Hamilton, 2001; Black & Duhon, 2003; Bagamery, Lasik, & Nixon, 2005; Bycio & Allen, 2007; Terry, 2007). The significance of the GMAT variable could simply be based on the observation that students with innate academic ability for standardized exams perform at a relatively high academic level. The positive and significant impact of GPA on exam score is anticipated since students with high grades are more likely to learn and retain core business information than are students with a relatively low grade point average. Consistent with Mirchandani et al. (2001), overall GPA has a strong internal validity and provides a measure of student performance related to the curriculum of the school. Student effort as measured by percentage score on homework assignments yields a positive and significant coefficient. The HW variable is a proxy for willingness to work until complete and adequate homework answers are obtained, organized, and presented to the course instructor. Financial management is one of the more difficult and demanding courses in the graduate business program so the positive and significant impact of ability (e.g., GMAT and GPA) and effort (e.g., HW) on final exam performance is not surprising. The demographic variable controlling for foreign students is positive, with international students scoring 4.3 percent higher on the exam than domestic students. International students are often recruited to diversify the campus environment and raise the level of academic standards via performance on standardized entrance examinations like the GMAT. International students often face unique language, psychic, and cultural challenges that might negate some of their innate academic ability. On the other hand, many international students have relatively strong quantitative skills, which appear to significantly enhance performance in financial management. 152

6 The gender variable (FEMALE) is negative but highly insignificant. Unlike previous research, the results of this study do not find any evidence of a gender differential with respect to performance on the final exam in a graduate financial management course. The most interesting result from Table 2 relates to the two instruction mode variables (ONLINE and ITV). Holding constant ability, effort, and demographic considerations, the results imply instruction mode does not impact student performance on the final exam. The ONLINE coefficient is positive, with online students scoring 2.5 percent higher than campus students on the final exam. The ITV coefficient is negative at a magnitude of 0.45 percent. Ultimately, both of the instruction mode variables are not statistically significant. The statistically insignificant results provides support to the notion that modern technology has created online and ITV learning environments that are equivalent to the traditional campus environment. The results of this study are derived from one institution, which somewhat limits the ability to broadly interpret and apply the results. Despite this limitation, the positive and significant impact of the ability and effort variables on final exam score provides evidence the empirical results are derived from an appropriately specified model. CONCLUSION This study compares online, campus, and ITV modes of instruction. Controlling for ability, effort, and demographic considerations, the research results indicate no statistical difference in the three instruction modes based on student performance in graduate financial management courses at a midsized comprehensive public university located in the Southwestern part of the United States. Consistent with previous research, the empirical results indicate that academic ability measured by the GMAT graduate business school entrance exam, grade point average, and effort measured by homework grades are the primary determinants of student performance. In addition, international students earn a higher score than domestic students on the final exam in a quantitative course like financial management. A demographic control variable for gender is not statistically significant in this study. Future research should focus on creating a more robust sample with broader applicability by employing data derived from multiple courses or from multiple universities. REFERENCES Bagamery, B., Lasik, J., & Nixon, D. (2005). Determinants of success on the ETS Business Major Field Exam for students in an undergraduate multisite regional university business program. Journal of Education for Business, 81, Becker, W. (1983). Economic education research: New directions on theoretical model building. Journal of Economic Education, 14(2), 4-9. Black, H. & Duhon, D. (2003). Evaluating and improving student achievement in business programs: The effective use of standardized tests. Journal of Education for Business, 79, Bowman, J. P. (2003). It s not easy being green: Evaluating student performance in online business communication courses. Business Communication Quarterly, 66(1), Bycio, P. & Allen, J. (2007). Factors associated with performance on the Educational Testing Service (ETS) Major Field Achievement Test in Business (MFAT-B). Journal of Education for Business, 83, Chizmar, J. & Spencer, D. (1980). Testing the specifications of economics learning equations. Journal of Economic Education, 11(2), Davisson, W. & Bonello, F. (1976). Computer assisted instruction in economics education. University of Notre Dame Press. 153

7 Douglas, S. & Joseph, S. (1995). Estimating educational production functions with correction for drops. Journal of Economic Education, 26(2), Fann, N. & Lewis, S. (2001). Is online education the solution? Business Education Forum, 55(4), Fortune, M. F., Shifflett, B., & Sibley, R. (2006). A comparison of online (high tech) and traditional (high touch) learning in business communication courses in Silicon Valley. Journal of Education for Business, 81(4), Ghoshal, S. (2005). Bad management theories are destroying good management practices. Academy of Management Learning & Education, 4(1), Hirschfeld, M., Moore R., & Brown, E. (1995). Exploring the gender gap on the GRE subject test in economics. Journal of Economic Education, 26(1), James, M. & Voight, M. (2001). Tips from the trenches: Delivering online courses effectively. Business Education Forum, 55(3), Kearsley, G. (1998). Distance education goes mainstream. Technological Horizons in Education, 25(10), Lezberg, A. (1998). Quality control in distance education: The role of regional accreditation. American Journal of Distance Education, 12(2), Martell, K. & Calderon, T. (2005). Assessment in business schools: What it is, where we are, and where we need to go now. In K. Martell & T. Calderon Eds, Assessment of student learning in business schools (pages 1-26). Tallahassee, FL: Association for Institutional Research. Martell, K. (2007). Assessing student learning: Are business schools making the grade? The Journal of Education for Business, 82(4), Mawell, N. & Lopus, J. (1994). The Lake Wobegon effect in student self-reported data. American Economic Review, 84(2), McCormack, C. & Jones, D. (1998). Building a web-based education system. New York: Wiley Publishing. Mirchandani, D., Lynch, R., & Hamilton, D. (2001). Using the ETS Major Field Test in Business: Implications for assessment. Journal of Education for Business, 77, Okula, S. (1999). Going the distance: A new avenue for learning. Business Education Forum, 53(3), Peach, B., Mukherjee, A., & Hornyak, M. (2007). Assessing critical thinking: A college s journey and lessons learned. The Journal of Education for Business, 82(6), Pfeffer, J. & Fong, C.T. (2002). The end of business schools?: Less success than meets the eye. Academy of Management Learning & Education, 1(1), Pringle, C. & Michel, M. (2007). Assessment practices in AACSB-accredited business schools. The Journal of Education for Business, 82(4), Siegfried, J. & Fels, R. (1979). Research on teaching college economics: A survey. Journal of Economic Literature, 17(3), Terry, N. (2007). Assessing the difference in learning outcomes for campus, online, and hybrid instruction modes for MBA courses. Journal of Education for Business, 82(4), Terry, N., Owens, J., & Macy, A. (2000). Student and faculty assessment of the virtual MBA: A case study. Journal of Business Education, 1(2), Trapnell, J. (2005). Foreward. In K. Martell & T. Calderon Eds, Assessment of student learning in business schools. Tallahassee, FL: Association for Institutional Research. Worley, R. B., & Dyrud, M. (2003). Grading and assessment of student writing. Business Communication Quarterly, 66(1),