A Hybrid-Online Course in Introductory Physics Homeyra R. Sadaghiani California State Polytechnic University, Pomona, USA hrsadaghiani@csupomona.edu INTRODUCTION The recent demand and interest in developing online and hybrid courses in both public and private institutions of higher education has intensified in recent years [1]. For the university, the benefits of online and hybrid courses include reduced costs and increased ease of disseminating courses. For students, these types of courses mean more distance learning opportunities, convenience, and flexibilities. As an educator and researcher, my goal is to improve student learning by continuously refining both the content and associated pedagogy of the physics courses. To this end, my objective in this project is to collect and analyze data in order to compare the learning, attitudes, and experiences of students in a hybrid physics course with the traditional face-to-face format. It is increasingly difficult for traditional teaching techniques to capture and maintain student interest because new generations have grown up in such a digitally advanced era. Many educators consider multimedia technology as a useful teaching tool that can attain the attention of today s students. Multimedia combines voice, animations, and words to create a more dynamic representation of course material. In the past decade, the field of multimedia learning has emerged as a coherent discipline with a great deal of research at its base. Research shows that students can benefit from technology-enhanced learning supplements that unify concepts and are delivered on-demand over the Internet [2], [3]. We have implemented multimedia rich online course material, developed by University of Illinois at Urbana Champaign (UIUC) Physics Education Research (PER) group, in teaching introductory calculus-based electricity and magnetism at California State Polytechnic University in Pomona (CPP). In this paper, we will discuss details of a control study that was designed to investigate the effectiveness of the material on student learning and achievement in context of a hybrid-online course. MULTIMEDIA LEARNING MODULES (MLM) Multimedia Learning Modules (MLM) are combinations of various media within a single computer program streamed online. These 12-15 minutes modules are research-based presentations and learning activities designed to introduce the key course concepts through flash animations with synchronous narration. Figure 1 demonstrates a screenshot
of the Ampere's Law MLM [4]. Each module has two or three questions that are embedded into the content itself. Students must answer the questions correctly in order to proceed. Automatic feedback and related prompts guide students in determining the correct answers to these questions. Students view the material as prelectures before attending each class meeting. In designing Multimedia Learning Modules, the content is guided by findings of physics education research, and presentations of the content are based on principles of multimedia learning. Research suggests that the use of dual learning channels (audio and visual) improves student learning [5]. Thus, by utilizing both auditory and visual channels to assimilate information, multimedia maximizes students short-term memory. By focusing on the principles of multimedia [6] and how people learn from words and pictures in computerbased environments, the MLM coordinates the animations with narration to enhance learning outcomes. METHODOLOGY Figure 1- Screenshot of the online multimedia prelecture on Ampere s Law in electromagnetism. We started piloting the online MLM prelectures in Fall 2008 by implementing three modules into a third quarter of introductory calculus-based electricity and magnetism course (PHY 133) for engineering and physics majors at CPP. Later, in Spring 2009 we designed a control study to investigate the effectiveness of the online prelectures in student learning, achievement, and preparation for class discussions. Students in one section of PHY 133 served as the Control group (face to face or a traditional format) while a second section was offered via a hybrid format using MLM as prelectures. Both classes met twice a week. The traditional class met 75 minutes each period while the hybrid class met for 50 minutes and the remaining 25 minutes (one-third of the class time) was offered to students in an online format through the use of Multimedia Learning Modules. Students were required to view the MLM before each face-to-face class meeting. Figure 2 illustrates the schematic layout of the research study. Here we will discuss the impact the multimedia prelectures had on student performance before class, in the classroom, and on nationally known conceptual tests. Thirty-four students were enrolled in the Hybrid course and forty-eight in the Control group. The two classes were very similar in their background as measured by the survey. For example, the average GPA for the Hybrid class was 3.0 compared to 2.9 for the
Control group, and the percentage of students receiving grades A, B, and C in their previous physics course (PHY 131, Mechanics) was also comparable. Figure 1. A schematic representation of the control study design. DATA COLLECTION We have collected a variety of data across both sections to compare student performance in the two courses. For example, we used a nationally known electricity and magnetism conceptual survey. We then collected data on student responses to identical clicker questions during the lecture, and collected survey data on usefulness of different course component from both groups. Below, we will discuss each data set and in more detail. I. Conceptual Survey: Using the Concept Survey of Electricity and Magnetism (CSEM) [7], we measured student conceptual understanding of the topics presented in this course. The CSEM is a 32-item, multiple-choice instrument of understanding of subjects covered in a typical introductory electricity and magnetism course. CSEM was used as both a pretest (first week of the quarter) and as a post-test (last week of the quarter). In order to compare the results across both sections, we calculated the normalized gain < g > for the students in the two classes where <g> is the ratio of the actual to the maximum possible: < g >= ( posttest! pretest)% (100! pretest)% ( 1) No incentives were offered to students for taking the survey and we noted that students responses on the survey would not affect their course grade in any way. We only included the data for students who completed both pre and post-test surveys. We gave students in both groups only thirty minutes to complete the pre and post-tests. However, thirty minutes was not sufficient; therefore, we only included the questions that majority of the students had completed which was up to question number 25. Most of these
Courses Pretest Post-test <g> Hybrid (N=29) 24% + 4% 58% + 4% 45%+ 4% Control (N=40) 27%+ 3% 54%+ 3% 37%+ 3% Table 1. The pre/post and normalized gain on CSEM. questions focus on electricity and not all the questions regarding magnetism were included in the analysis of the scores. The results of the student performance in CSEM are summarized in Table 1. The pretest data was slightly lower for the Hybrid group compared to the Control (24% vs. 27%). However, the Hybrid group outperformed the Control group in the post-test resulting in 8% higher normalized gain (45% vs. 37%). As for comparing the data to published data, [see ref 7] typical pretest scores are 31% (calculus-based) and 25% (algebra-based); post-test data only rise to 47% and 44% respectively. Data was collected for over 5000 students at 30 different schools. However, since our results do not include the entire 32 items on CSEM, there is no good means for comparison. II. Clickers questions and class discussion Clickers are hand-held devices similar to the remote controls which can send a specific electronic signal (e.g., radio frequency) to a central receiving station connected to a computer equipped with software that tabulates the responses and displays the distribution of answers on a bar graph on the screen. Often the instructor poses a multiple choice or true/false question. These questions can be content questions asking for recall of information, conceptual questions seeking evidence of understanding, application questions, critical thinking questions, synthesize questions, or problem solving questions. Students respond by pushing buttons for answers (a), (b), (c), and so on. Then, the instructor displays the bar graph of student responses which in turn provides an immediate feedback to students on where they stand in terms of how well they understand the material, and where they stand in relation to their peers. Along with clickers, we also used the think-pair-share from the Peer Instruction [8] method and asked students to talk to their neighbors in class before voting. We compared student correct responses to thirty-six identical clicker questions on relevant topics of lecture and textbook reading. The average percentage of the correct answers in the Hybrid class was (60.3+4.0)% while this number for the Control group was (53.8+3.3)%. The graphical representation of the data is shown in Figure 3. This difference in means leads to a small Effect Size (ES) of 0.27; where ES is a statistical concept that measures the strength of the relationship between the two variables. The ES can be calculated by dividing the two population mean differences µ 1 and µ 2 by their standard deviation! : ES = µ 1! µ 2 "
The small value of the ES (0.2) corresponds to a small effect, 0.5 is considered a medium effect, and 0.8 and greater reflect a large effect size. Figure 3. Student responses to identical clicker questions in Control and Hybrid classes III. Student feedback: In addition to collecting class discussion data via clickers and the CSEM concept test, at the end of the quarter we administered a survey designed to elicit specific feedback on student attitude and experience with the MLM and with the hybrid-online course delivery format. This survey comprised of multiple-choice, ranking scales, and open-ended questions. Among other questions, we asked students in both classes to rank the usefulness of different course components in their learning in the scale 1 to 5 (1 = not useful at all, 5 = very useful). The result of student responses to this question is represented in a bar graph in Figure 4. The textbook was ranked the lowest in both classes; many students stated that they do not find the textbook useful in their learning. Student average ranking of the prelectures was much higher than the textbook. The average ranking of the lecture and clicker usefulness by students in the Hybrid class was slightly higher than the Control group, indicating that the view of MLM did not diminished their interest in class discussion and interaction. In fact, students in the Hybrid class seem to be more engaged with the questions and discussions than the Control group; perhaps due to some prior preparation via MLMs.
Figure 4. Student ranking of different course component in 5-scale survey DISCUSSION We have conducted an experiment to evaluate the effectiveness of using MLM prelectures in teaching Electricity and Magnetism course at CPP. Our data indicated that overall introduction of MLM prelectures as an online component of the hybrid course had positive impact on the course. First, students showed a larger conceptual gain, measured by SCEM, in the class where MLM were utilized. Secondly, students who viewed MLM outperformed the Control group in class discussion and clicker questions. Finally, students believed that MLM helped them learn better and was ranked more useful than the course textbook. It is important to note that this study does not promote online courses, rather demands higher quality, better learning outcomes, and increased student engagement for online course material. One can argue that traditional, face-to-face teaching is always better. However, the development of technology and amplified use of computers will bring more and more online courses upon us. Online enrollments have grown much faster than overall higher-education enrollments over the past few years. During the period 2002-2007, enrollment in online courses grew 19.7%, compared with 1.5% growth in the overall college-student population. [see ref 1] These numbers suggest tremendous interest in online teaching and learning and the possible increase on the number of online courses that will be offered in near future. The Internet provides a platform for learning about and interacting with the world which higher education could employ as an educational tool. It is economical, flexible, convenient, and more importantly it is an integral part of our students lives. Thus, as educators and researchers we should be concerned and involved with the quality and delivery of the online course material and use the results of research to demand higher quality and excellence.
REFERENCES [1] "Staying the Course: Online Education in the United States," published by the Sloan- C, 2008 report. [2] e-learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, Ruth Colvin Clark & Richard E. Mayer, Wiley, 2008. [3] Richard E. Mayer, Multimedia learning: Are we asking the right questions? Educational Psychologist, Vol. 32, 1997. The Cambridge handbook of multimedia Learning, Edited by Richard E. Mayer, Cambridge University Press, 2005. [4] Full MLM modules can be viewed at: https://online-s.physics.uiuc.edu/courses/phys212/gtm/no_login/page.html [5] A. Baddeley et al. Dementia and Working Memory, The Quarterly Journal of Experimental Psychology, V. 38, Issue 4, Pages 603 618, 1986. John Sweller, Cognitive load theory, learning difficulty, and instructional design, Learning and Instruction, Vol.4, 1994. [6] R. Moreno et al. Cognitive principles of multimedia learning: The role of modality and contiguity, Journal of educational, Vol 91, 1999. Multimedia learning, By Richard E. Mayer, Cambridge University Press, 2009. [7] D. Maloney, T. O Kuma, C. Hieggelke, and A. Van Heuvelen, Surveying students conceptual knowledge of electricity and magnetism, American Journal of Physics, Vol. 69, 2001. [8] Eric Mazure et al., Peer Instruction, American Journal of Physics, Vol. 67, 1999. CH Crouch, E Mazur Peer instruction: Ten years of experience and results, American Journal of Physics, Vol. 69, 2001