Evaluating the impact of personal response systems with lecture capture on student learning outcomes.

Transcription

1 Evaluating the impact of personal response systems with lecture capture on David L. Smiley, M.S. Craig M. Ross, Re.D. Indiana University Bloomington A proposal for the Echo360 Active Learning Grants. This submission is for consideration for the Newcomer s, Adopters, and Active Learning Merit categories.

2 Section 1: Executive Summary Abstract: The use of electronic response systems in classrooms in higher education is becoming more widespread. While much of the literature has evaluated the benefits from the instructors perspective, as well as the effects on student learning outcomes; there is little research to date on the combined effect of electronic response systems and lecture capture on student s learning outcomes. To gain a greater insight into the relationship of these two tools, we propose to integrate Echo360 and LectureTools into an existing Festival & Event Management class at Indiana University Bloomington, during the 2014 and 2014 academic sessions. While many disciplines have been evaluated in previous studies, this will be one of the first studies in the tourism and event management field. Our aim is to collect data in the fall and spring semesters on the student s evaluations and classroom performance against a set of learning outcomes. These findings will advance the understanding of the effect of combining these two technologies and their effect on improving Section 2: Detailed Overview Description: The purpose of this study is to examine the effect of Echo360 with lecture capture and LectureTools technologies and their effect on Lecture capture is an active learning technology that is gaining the attention of instructors and student s on college campuses and involves the recording of classroom activities (both audio and video) using various software and then making the recordings available electronically (Zhu & Bergom, 2010). The rationale for using this technology is to enhance student learning but there is still relatively little research into how viewing videoed lectures can enhance the learners experience or improve learning outcomes (Young & Moes, 2010, p. 5). This new environment allows students to access current material, supplement classroom discussions and highlight new course content (Sherer & Shea, 2011). Previous research conducted on lecture capture technology has shown mixed results indicating an increase in student performance with positive results from webcasting/podcasting (Brotherton & Abowd, 2004; Holbrook & Dupont, 2011) to decrease class attendance (Traphagan, Kucesa, & Kishi, 2010; von Kosky, Ivins, & Gribble, 2009). In today s university environment it is critical that students are viewed as customers and the end consumers. Satisfaction of service quality is a customer issue. It is the customer, or in this case the students, who will ultimately decide if they have received it. This study will evaluate the impact of the technologies from the student s perspective. According to Alves and Raposo (2008), it is the "responsibility of the institutions of higher education to understand the formation process of student satisfaction and also, to find reliable forms of measuring satisfaction. The authors also go on to say that only with reliable measurements can the institutions know their own reality, comparing it with other institutions and analyzing its evolution over time. Page 2

3 The student s satisfaction measurement will be an indicator of the effectiveness of the two technologies in the classroom. Research on satisfaction is extensive in the career development and educational field and factors related to satisfaction have been well studied and documented. Over the years there have been numerous attempts to define service quality and the related concept of customer satisfaction (Oliver, 1980: Tse and Wilton, 1988). Measuring satisfaction within the educational environment was first conducted by Berdie (1944). This study will evaluate the student s satisfaction with both technologies. Beginning in Fall 2013, David Smiley will integrate active learning technologies into his course: Festival and Event Management. This 70-person class instructs upper-level undergraduate students in developing and hosting an actual event. For the spring and fall semesters they will be developing and running one of the most prestigious debate tournaments in the country. The Cross Examination Debate Association (CEDA) tournament will be held at Indiana University in March of This event will cover 7 days and include up to 500 participants. The class structure for organizing this event is a perfect laboratory for active learning and the Echo360 and LectureTools platforms. The Collaborative Learning Studio will be requested as the site for this course. The Studio is one of four innovative classroom spaces on the Bloomington campus that uses advanced technologies to support collaborative learning in large classes. Students can work as teams at technologyenhanced tables and have their work displayed on a 20-foot wide video wall for sharing. The Studio was designed to offer an alternative to the traditional lecture hall, facilitating active learning approaches that include team-based learning, problem-based learning, and other collaborative pedagogies. Without a traditional instructor lectern, or even a clear front to the room, the Studio has a de-centered feel, allowing the instructor to provide short bits of instruction from anywhere in the room, interspersed with student-based learning activities that call on students to apply course concepts to solve challenging problems and engage in other Page 3

4 hands-on collaborative projects. The grant funds will allow us to maximize the possibilities of this studio and develop methods for utilizing the technologies to their fullest capacity. Objectives: The specific objective of this project is to examine the correlation between student use of lecture capture and personal response systems and the effect on student s attainment of the class learning outcomes. The quest for enhancing our ability to teach effectively and increase student learning is gaining importance as an identifiable research stream and is evolving in its academic scholarship (Young, Klems & Murphy, 2003). The results of this study will be measured over two class sections with one in the fall and one in the spring. The fall class will be the control group and will utilize only LectureTools in the presentation of the class. The spring section will utilize both LectureTools and Echo360 in the classroom. Both sections will be administered a pre and post-test questionnaire of the student s knowledge related to the class learning outcomes. The testing and all of the protocols will be submitted for approval to the Office of Research Administration and the IRB for human study compliance prior to the implementation of the study. At the completion of the spring semester results will be evaluated using the statistical software SPSS The results will be used to answer the following research questions: RQ 1 - What is the relationship between the student s use of LectureTools and their attainment of the class learning outcomes? RQ 2 - What is the relationship between the student s uses of Echo360 combined with LectureTools and their attainment of the class learning outcomes? RQ 3 - Is there a statistically measurable difference of the attainment of class learning outcomes between the two classes? We will also be measuring the impact on student s attendance as a covariate indicator. Satisfaction will also be measured using a researcher developed Student Satisfaction Survey. The results will provide insight into the effect of the two different learning technologies and their resultant impacts. Dissemination:. We believe the results of our research will not only benefit the Department of Recreation, Park, and Tourism Studies and the Indiana University School of Public Health but will also be transferable to the I.U. community as well as contribute to the scholarship of teaching and learning literature. This proposal supports the scholarship of teaching and learning and will provide the investigators and faculty colleagues an opportunity to document the effectiveness and critical evaluation of the lecture capture and the use of personal response systems for the assessment of student learning outcomes and to publish and present our findings in several peerreviewed forums. In addition to submitting results in the IU Scholarship of Teaching and Learning event series, which would include campus colleagues, we plan to present the results of our study at the following national discipline specific conferences: Page 4

5 2014 National Recreation and Park Association (NRPA) Congress in Charlotte, NC, October NRPA is the leading organization in our field dedicated to the advancement of public parks, recreation, tourism, and conservation Meeting Professionals International World Education Conference, Minneapolis, MN, August 2-5. MPI, with more than 21,000 members from 86 countries, serves meeting and event planners with professional development in building a global meeting industry community EDUCAUSE Annual Conference, Orlando, FL, September 29-October 2. EDUCAUSE provides a variety of professional development opportunities that serve the learning needs of the higher education community. In addition, the following activities will be included in the plan to disseminate information about the grant award and the findings from the research project: NRPA refereed pedagogy publication, Schole-A Journal of Leisure Studies and Recreation Education, of the Society of Park and Recreation Educators. IU press releases announcing Indiana University s selection for the award and the completion of the project and findings. Presentation of the project and its findings to take place at an Echo360 User Conference. Presentation at the department s doctoral teaching seminar SPH-R645. School of Public Health-Bloomington sponsored teaching workshop. Project Team: Section 3: Appendix Indiana University, IU Bloomington is the largest university campus in Indiana and is the flagship campus of Indiana University s eight campuses statewide with a spring 2013 enrollment of more than 42,000 students. The IU School of Public Health-Bloomington has approximately 3,000 students and offers a traditional campus experience enriched by 21st-century innovation. More than 120 faculty in five academic departments -- Department of Kinesiology; Department of Applied Health Science; Department of Recreation, Park and Tourism Studies; Department of Environmental Health; and Department of Epidemiology and Biostatistics -- conduct major research, teach and engage with communities across a broad spectrum of health, wellness, and disease-prevention topics. The Center for Innovative Teaching and Learning (CITL), The Center for Innovative Teaching and Learning (CITL), a partnership between the Office of the Vice Provost for Undergraduate Education and University Information Technology Services, provides comprehensive services in support of excellent teaching and learning at Indiana University Bloomington. Page 5

6 David L. Smiley, M.S. Mr. Smiley is a clinical faculty member in the Department of Recreation, Park, and Tourism Studies at Indiana University-Bloomington and will be the project coordinator and instructor of the course to be used in the proposed research. Mr. Smiley has over 25 years of experience in the Recreation, Park and Tourism industry and has received numerous undergraduate and graduate awards and scholarships relating to this major. In addition to his teaching, he is currently developing an online-class in tourism for the Eppley Institute for Parks and Public Land. Craig M. Ross, Re.D., Professor. Dr. Ross is a full-professor in the Department of Recreation, Park, and Tourism Studies at Indiana University-Bloomington. He has received 12 national or University teaching awards including the National Recreation and Park Association-Society of Park and Recreation Educators Excellence in Teaching Award, the School of Health, Physical Education, and Recreation Outstanding Teacher Award, the Indiana University President s Award for Excellence in Teaching, the Indiana University system wide Faculty Colloquium on Excellence in Teaching (FACET) award, and five Indiana University Trustee Teaching Excellence Recognition awards. Project Plan: Phase 1: Early Fall, 2013 course development 1. Examine current SPH-T 323 Festival & Event Management course orientation, learning outcomes, lecture materials, assignments, and exams. 2. Content will remain essentially the same as used in Spring, 2013 course with no lecture capture (audio/video media components) and no use of personal response systems. 3. Assignments, exams, and assessments will remain the same as Spring, Develop and administer pre-test survey in August, Phase 2: Late Fall, 2013 course development 1. Create revised SPH-T 323 Festival & Event Management course orientation, learning outcomes, lecture materials, assignments, and exams using lecture capture and personal response systems. 2. Create content areas, folders and other components using IU Oncourse course management system. 3. Create any new tutorials to aid students in course content. 4. Implement personal response system technology. 4. Administer post-survey in December, Phase 3: Spring, 2014 course development 1. Implement lecture capture technology in addition to personal response systems. 2. Monitor student access and usage during spring semester. Page 6

7 3. Administer pre-test in January, Administer post-test in April, Phase 4: Summer, Complete summative assessment for the end of Fall, 2013 course and the end of Spring, 2014 course. 2. Conduct ANOVA, post-hoc, and any correlative analysis between two courses. 3. Complete study results and report. 4. Disseminate results. References: Alves, H., & Raposo, M. (2008). The measurement of the construct satisfaction in higher education. The Services Industries Journal, 29(2), doi:10:1080/ Berdie, R. F. (1944). The prediction of college achievement and satisfaction. Journal of Applied Psychology, 28, Brotherton, J. A., & Abowd, G. D. (2004). Lessons learned from eclass: Assessing automated capture and access in the classroom. Transactions on Computer-Human Interaction, 11(2), Holbrook, J., & Dupont, C. (2011). Making the decision to provide enhanced podcasts to postsecondary science students. Journal of Science Education and Technology, 20(3), Oliver, R.L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), Sherer, P., & Shea, T. (2011). Using online video to support student learning and engagement. College Teaching, 59(2), Traphagan, T., Kucsera, J. V., & Kishi, K. (2010). Impact of class lecture webcasting on attendance and learning. Education Technology Research Development, 58, Tse, D.K., & Wilton, P.C. (1988). Models of consumer satisfaction formation: An extension. Journal of Marketing Research, 25, Page 7

8 Von Kosky, B. R., Ivins, J., & Gribble, S. J. (2009). Lecture attendance and web based lecture technologies: A comparison of student perceptions and usage patterns. Australian Journal of Education Technology, 25(4), Young, C., & Moes, S. (2010). How to move beyond lecture capture: Pedagogy guide. 2lIswwVGSu4IglkLChqSQzYvauZbM9TV1naPVEOd*r6PqstreF/Pedagogy_guide_nov_ draft.pdf Young, M. R., Klemz, B. R., & Murphy, J. W. (2003). Enhancing learning outcomes: The effects of instructional technology, learning styles, instructional methods, and learning behavior. Journal of marketing Education, 23, Zhu, E., & Bergom, I. (2010). Lecture capture: A guide for effective use. Center for Research on Learning and Teaching, University of Michigan, 1-8. Page 8