Using Lecture Capture to Create a Blended Learning Experience: Investigating Student Learning in a Flipped, Large Enrollment Course Dawn Zimmaro Ph.D. Stephanie B. Corliss, Ph.D. Jennifer Ebbeler, Ph.D. The University of Texas at Austin This proposal is in response to a Call for Submissions from Echo360. The Call for Submissions and related criteria can be found at the following URL: http://echo360.com/community/grants-program/ 1
1. Executive Summary A. Title Using Lecture Capture to Create a Blended Learning Experience: Investigating Student Learning in a Flipped, Large Enrollment Course B. Institution The University of Texas at Austin C. Abstract The flipped classroom is an emerging instructional trend where the coverage of content (traditional lecture) occurs outside of the classroom and the application of the content occurs within the classroom. Previous Echo360 supported research determined this instructional model could work within a large enrollment classroom if: 1) out-of-class learning focused on content deliver, 2) in-class learning activities were designed around active learning, 3) all learning activities were well-aligned with course learning outcomes, and 4) the instructor adequately supported the instructional method. A limitation of the previous study was that the flipped method was for a single class experience. This proposal builds upon those results by testing the effectiveness of the method for an entire course where lectures are captured using Echo360 and viewed prior to class, and in-class time is devoted to active reflection and discussion of case studies. With funding, we will be able to identify successful strategies for implementing blendedlearning in a flipped classroom experience and examine the impact on student learning within large enrollment classes. We believe transforming students experiences in class is a key to improving student success in higher education. Results of this research will have implications for instructional technology administrators, designers, and faculty members implementing a total or partial flipped classroom model. 2
2. Overview A. Description In 2010, a research study was conducted at the University of Texas at Austin to explore the use of the flipped classroom method, where core course content is presented outside of class and inclass time involves active learning, in large enrollment higher education courses. Specifically, in the flipped classroom, passive activities like listening to lectures or content presentations take place outside of class, so that during class time students are engaged in deeper, hands-on learning activities while the instructor guides their thinking (Gannod, Burge, & Helmick, 2008). This instructional method also allows instructors to move from traditional teacher-centered classroom lectures to student-centered experiences during class. These strategies require students to be more active in constructing their own knowledge through activities such as class discussions and collaborative learning (Allen & Tanner, 2005; McCray, 2000; Wright, 2011). This instructional method has gained popularity with the emergence of new technologies that make multimedia data and out-of-class course material more accessible and ubiquitous to students. It has been a recent topic of discussion on several educational blogs (Bruff, 2011; Kim, 2011; McShea, 2011; Talbert, 2011), and discussed in the higher education press (Kolowich, 2010; Young, 2009). However, few research studies have investigated the practice of flipping the classroom empirically or produced a set of best practices. With generous support through a 2010 Echo360 grant, three instructors at UT Austin used lecture captures to flip one class session of their courses and we researched the impact on student learning and instructor and student attitudes. Results from this study led to the development of a model that represents the relationship among four key components necessary for successful implementation of the flipped classroom in higher education. In particular, flipping the large classroom was found to be most successful in improving student learning, satisfaction, and engagement when: 1) out-of-class learning focused on content deliver, 2) in-class learning activities were designed around active learning, 3) all learning activities were well-aligned with course learning outcomes, and 4) the instructor adequately supported the instructional method. See Figure 1. 3
Figure 1: The Flipped Classroom Model In the current research study, we propose to build upon our initial results and test the flipped classroom model by applying it to an entire course. We have partnered with the instructor of an undergraduate classics course, who has recently been informed that course enrollment will increase from approximately 200 to 400 students. With the current budgetary strains in higher education, public institutions are increasingly looking for ways to improve student completion, institutional productivity, and stretch limited physical resources. Creating larger classrooms and using blended learning approaches are often part of these discussions. In Fall 2012, Professor Jen Ebbeler will flip her course, Classical Civilization: An Introduction to Ancient Rome. A major goal of flipping this course is to cover additional content related to ethics and leadership, engage the students in more regular thoughtful discussions of primary texts, to significantly deepen student learning about Roman history, and to inculcate a habit of active engagement in the students, particularly incoming freshmen. Through this additional research, we strive to identify successful strategies for implementing the flipped classroom model for an entire semester in a large course that is traditionally taught through lectures. To empirically investigate the full application of the model, we will collaborate closely with the instructor in preparing well-aligned, learning materials for students to absorb outside of class, and in creating interactive, student-centered learning opportunities in the classroom. We will draw from the findings of our previous Echo360 research to apply lessons learned from the partial flipped classroom experiences. To facilitate collaboration and diffusion of this new way of teaching across campus, the grant funds would allow us to provide faculty stipends to those who are currently flipping their classroom and those who are interested in trying this new way of teaching to participate in meetings and discussions about the method. We will also readily share our findings, lessons learned, and strategies for implementing a flipped classroom with the UT community and beyond. We believe transforming students experiences in class is a key to improving student success in higher education. Results of this research will have implications for instructional technology administrators, designers, and faculty members implementing a total or partial flipped classroom model. 4
Course Description Classical Civilization 302: Introduction to Ancient Rome is a cornerstone course of the Classics Department s annual course offerings. The aim of the course is to offer an introduction to the history and culture (literary and material) of Ancient Rome from the Iron Age (c. 1000 BC) to the end of the Western Roman Empire in 476 AD. In Fall 2012, the instructor is adding an ethics and leadership component to the course to satisfy a university core curriculum requirement. To incorporate the new material, which consists of students applying the basic principles of systematic moral analysis to episodes in Roman history, she would either have to cover less material than she usually does, or change her teaching method. The instructor will flip the course to free up class time for exploring the complexities of individual case studies through in-class discussion. She will be collaborating with the lead audio engineer at the Liberal Arts Instructional Technology Services (LAITS) to use the Echo360 system to prerecord approximately 45 short lectures (c. 25-30 min.) in which she digests, clarifies, and expands upon the assigned textbook readings. Included in this series of filmed lectures will be an introduction to the concept of a flipped classroom and how this will change the students classroom experience; an introduction to the concept and practice of systematic moral analysis (what it is and why it matters); an overview of the Echo360 system; and tips for successfully navigating this type of course. The course will be organized as follows: Mondays will be devoted to the review of a short, primary text in which the narrative of a particular case study is presented. Wednesdays will be devoted to an expanded discussion of questions identified in the Monday class. On Fridays, attendance will be optional (though strongly suggested), and no new material will be introduced. A teaching assistant will lead a review of the assigned textbook readings and video viewings. B. Objectives The objectives for this project are to answer the following research questions: 1. What impact does the flipped classroom experience have on students learning in a large enrollment course? 2. What impact does the flipped classroom experience have on instructor, teaching assistant, and student attitudes? 3. Do core academic skills gained during the flipped classroom experience (e.g., ethically informed decision-making, oral communication) transfer to other courses and to everyday life? Data Collection and Analysis We are using a mixed methods approach to answer the research questions. During the Fall 2012 semester, we will look at pre- and post-test scores to examine students learning in the course. Across semesters, we will examine how Fall 2012 student performance on common assessment items differs from student performance in previous semesters the course was taught. Student, teaching assistant, and instructor attitudes will be measured through surveys containing both 5
open-ended and forced-choice answers, allowing us to gather quantitative and qualitative data. To examine the transfer of skills to other courses, we will track students performance in future courses and conduct a follow-up survey in Spring 2013 asking students to comment on how they use the skills learned in this course in future courses. Table 1 summarizes the multiple data collection methods, and is followed by a description of each data source and how that data will be analyzed. Table 1: Research Questions and Data Collection Methods Research Questions Student Characteristics Exam Item/ Rubricbased Assignment Scores Course Grades & Drop Rates Classroom Observations Student Mid- Semester Survey Student End-of- Semester Survey TA/ Instructor Survey Course Instructor Survey Student Followup Survey Subsequent Course Grades & Drop Rates What impact does the flipped classroom experience have on students learning in a large enrollment course? What impact does the flipped classroom experience have on instructor, teaching assistant, and student attitudes? Do core academic skills gained during the flipped classroom experience (e.g., ethically informed decisionmaking, oral communica tion) transfer to other courses and to everyday life? X X X X X X X X X X X X 6
Student Characteristics: Data about student background (e.g., high school GPA, high school courses taken, ACT/SAT scores, gender, race, SES, parents education level, classification, major, college, first generation college student) will be gathered from university s databases. The data will be used to examine differences across different types of students within a semester and to control for any differences among students across semesters. Exam Item/Rubric-based Assignment Scores: Common exam items and/or rubric-based assignments used in the Fall 2012 semester and previous semesters in which this course was taught will be identified and scores will be compared across semesters. Within a semester, student scores on pre- and post-assessments will be examined to determine levels of student learning. Student Course Grades & Drop Rates: Student course grades and drop rates from the Fall 2012 semester will be compared to previous semesters. Classroom Observations: An observation form will be used to evaluate perceived student engagement and attitudes during class. A researcher will attend approximately two to three class sessions. These observations will also be used as formative feedback for the instructor to learn more about what is working and what is challenging for students in this new way of teaching. Student Mid-semester Survey: Students will complete an online survey following their first exam to document their perceptions of the lecture capture assignments, the in-class activities, engagement with the material, the development of oral communication skills, and the use of systematic ethical analysis in the course. The instructor will use the survey results for formative feedback. Student End-of-semester Survey: Students will complete an online survey at the end of the semester. The survey will be similar to the mid-semester survey measuring student perceptions of the lecture capture assignments, the in-class activities, engagement with the material, the development of oral communication skills, and the use of systematic ethical analysis in the course. We will compare student perceptions from mid-semester to end-of-semester to examine any changes. The instructor will also use the survey results to inform changes to her class in the following semester. TA/Instructor Survey: The course instructor and teaching assistant will complete an online survey to document their perceptions about the flipped classroom and the effects on teaching, the classroom environment, and student learning. Course Instructor Surveys (CIS): CISs contain questions related to overall satisfaction of a course and the instructor, and direct questions related to instruction, workload, and classroom assessment. The surveys are administered at the end of each semester. CIS scores will be compared across semesters. Student Follow-up Survey: Students from the Fall 2012 semester will receive a follow-up survey towards the end of the Spring 2013 semester. The survey will measure student perceptions of the usefulness of the skills they learned in the previous course in their current courses and everyday lives. Subsequent Course Grades & Drop Rates: Student course grades and drop rates from subsequent courses will be examined and compared to student cohorts from previous semesters. 7
C. Dissemination Results of this research will be disseminated to both the University of Texas at Austin community and the national learning technology community in the following ways: The University of Texas community: Results of this research will be presented in a presentation and/or workshop with other UT faculty interested in flipping their classroom. Results of the research, along with student and faculty comments and/or case studies will be presented on the Liberal Arts Instructional Technology Services (LAITS) and Center for Teaching and Learning (CTL) websites. The final project report will be downloadable from the CTL website. The national learning technology community: Researchers will submit a proposal to share findings at that the Annual Meeting of the American Educational Research Association. AERA is a national research society whose mission is to advance knowledge about education, to encourage scholarly inquiry related to education, and to promote the use of research to improve education and serve the public good. Researchers will share findings in one or two articles submitted to national higher education journals. If requested, researchers will share findings and lessons learned in an Echo360 webinar, press release, and blog posting. * * * * * References Allen, D., & Tanner, K. (2005). Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biology Education. 4(4), 262 268. Bruff, Derek. (2011, April 28). Mobile Learning and the Inverted Classroom. [Blog Post]. Retrieved from http://derekbruff.com/site/blog/2011/04/28/mobile-learning-and-the-invertedclassroom-edusprint/ Gannod, G. C., Burge, J. E., and Helmick, M. T. 2008. Using the inverted classroom to teach software engineering. In Proceedings of the 30th international Conference on Software Engineering (Leipzig, Germany, May 10 18, 2008). ICSE 08. ACM, New York, NY, 777-786. Kim, J. (2011, August 10). On Lecture Capture and Course Quality. Inside Higher Ed. Retrieved from http://www.insidehighered.com/blogs/technology_and_learning/on_lecture_capture_a nd_course_quality. Kolowich, S. (2010, September 13). Open Source Lecture Capture. Inside Higher Ed. Retrieved from http://www.insidehighered.com/news/2010/09/13/matterhorn. McCray, G.E. (2000). The hybrid course: Merging on-line instruction and the traditional classroom. Information Technology and Management, 1, 307 327. 8
McShea, Jack. (2011, July 15). At a Loss for Words The Future of Lecture Might be Less Talk. [Blog Post]. Retrieved from http://www.hg2s.com/blog/tag/inverted-classroom/ Talbert, Robert (2011, January 11). The inverted classroom and student self-image. [Blog Post]. Retrieved from http://castingoutnines.wordpress.com/2011/01/11/the-invertedclassroom-and-student-self-image/ Wright, G. B. (2011). Student-centered learning in higher education. International Journal of Teaching and Learning in Higher Education, 23(3), 92-97. Young, J. (2009, July 20). When Computers Leave the Classroom, So Does Boredom. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/teach- Naked-Effort-Strips/47398/ 9
3. Required Appendices A. Project Institution & Team The University of Texas at Austin, founded in 1883, is one of the largest and most respected universities in the nation. It is the flagship campus of the University of Texas System, which is made up of nine academic universities and six health institutions. UT Austin employs over 24,000 staff and faculty members and has a student enrollment of about 51,000. The university is comprised of 17 colleges/schools, and awards approximately 12,000 degrees annually. The mission of The University of Texas at Austin is to achieve excellence in the interrelated areas of undergraduate education, graduate education, research and public service. The university provides superior and comprehensive educational opportunities at the baccalaureate through doctoral and special professional educational levels. The Center for Teaching and Learning (CTL) at The University of Texas at Austin works closely with colleges, departments, students, and individual faculty to support strategic initiatives that advance educational excellence. CTL conducts educational research and development through the integration of pedagogy, assessment, and technology. The Liberal Arts Instructional Technology Services (LAITS) at the University of Texas at Austin is dedicated to providing technology support for instruction, research, and administration throughout the College of Liberal Arts. This includes classroom technology, desktop computers, web sites, networks and servers, digital audio and video, and many other technologies both old and new. We also support new construction and renovation projects. Principle Investigators: Principle Investigator: Dawn Zimmaro dawn.zimmaro@austin.utexas.edu 512-232-1506 Co-Investigator and Project Coordinator: Stephanie Corliss Stephanie.corliss@austin.utexas.edu 512-232-1506 Co-Investigator and Lead Faculty Member: Jennifer Ebbeler ebbeler@mail.utexas.edu 512-471-7570 DAWN ZIMMARO is the Senior Director and Director of Assessment at the Center for Teaching and Learning at the University of Texas at Austin. In her role, Dr. Zimmaro collaborates with faculty, administrators, and staff on higher education issues related educational research, testing, classroom assessment, student placement, credit by examination, and student evaluations of instruction. She also oversees organizational operations, strategic planning, and policy development and implementation for the center. Previously, Dr. Zimmaro served as an Educational Testing Associate for University Testing Services and Coordinator of Learning Assessment Programs and Research for the Schreyer Institute for Teaching Excellence at Penn 10
State University. Dr. Zimmaro has been involved in supporting several educational research and evaluation grant projects for university, state, and federal grant programs. Dr. Zimmaro has taught educational psychology courses at the undergraduate and graduate level. She has a Ph.D in educational psychology focused on measurement and evaluation with a minor in statistics and a MS in educational psychology from The Pennsylvania State University. STEPHANIE CORLISS is the Research Program Coordinator at the Center for Teaching and Learning at the University of Texas at Austin. She consults with faculty, staff, and administration and performs advanced research related to teaching and learning to provide feedback to departments, colleges, and the university. Dr. Corliss has been researching the use of instructional practices to enhance the teaching and learning experience for over eleven years. She previously worked at the University of California, Berkeley, where she led professional development, research, and assessment efforts with teachers integrating technology-enhanced instruction into their curriculum. Additionally, Dr. Corliss has taught educational psychology and instructional technology courses at the undergraduate and graduate level. She has a Ph.D. in educational psychology focused on learning, cognition, and instruction and an MA in program evaluation from the University of Texas at Austin. JENNIFER EBBELER is an Associate Professor of Classics at the University of Texas at Austin. She specializes in the history and literary culture of Latin late antiquity; the writings of St. Augustine; the history of Roman Africa; and the practice of ancient letter writing. She has published numerous articles and book chapters in her areas of specialty. Her book, Disciplining Christians: Correction and Community in Augustine's Letters is forthcoming in 2012 from Oxford University Press. She teaches a broad range of courses, from graduate seminars on Roman Africa and Trajan's Rome to undergraduate Latin courses on Nero. She also teaches a number of different Classical Civilization courses, including the large enrollment Introduction to Ancient Rome. Since Fall 2009, she has served as the supervisor of the graduate student Latin instructors in the Classics Department and recently redesigned and taught the Classics Department's Latin pedagogy course for graduate students. She has a Ph.D. in classical studies and an M.A. in classical studies and comparative literature from The University of Pennsylvania. Contributing Research Team Members: MICHAEL WALLACE is an Instructional Consultant at the Center for Teaching & Learning at the University of Texas at Austin. His primary focus is consulting with faculty on the Course Transformation Program as to create learning-centered instruction for freshmen level, largelecture courses. Dr. Wallace has developed flash interventions on learning strategies that will assist science students in developing effective strategies for learning within these transformed/flipped/blended-learning classrooms where students become actively engaged in applying concepts during class. He previously worked at Morehead State University, where he served as Director of Retention and Academic Recovery and as an Assistant Professor of Science Education. As a professor, he transformed a traditional lecture/lab physical science course for pre-service elementary teachers into an inquiry-based course aimed at training teachers in both content and how they should teach. He has a Ph.D. in Science Education Curriculum & Instruction, and an M.S. in Physics from the University of Missouri at Columbia. 11
MICHAEL HEIDENREICH is the lead audio engineer for the College of Liberal Arts at the University of Texas at Austin. He has been working as an audio engineer for 17 years. He specializes in the delivery of new and remastered archival audio via online means. He has worked on many online projects including a reading of the original script EL Cantar de Mio Cio (http://www.laits.utexas.edu/cid/), The Onda Latina Collection (http://www.laits.utexas.edu/onda_latina) and Liberal Arts ITS Audio Catalog (http://www.laits.utexas.edu/itsaud/), and he has provided archival services to the Dolph Briscoe Center for American History, the Harry Ransom Center and the Blanton Museum of Art. He also digitized and remasterd hundreds of field recordings for Dr. Megan Biesele s National Science Foundation grant Documenting the Khomani and Ju/hoan Languages. He serves as the College of Liberal Arts Instructional Technology Services project manager for the pilot program using Echo360 lecture capture appliances. He holds a B.S. degree in Radio-Television-Film from the University of Texas at Austin. ERIN REILLY is a Graduate Research Assistant at the Center for Teaching and Learning. She is currently an acting assessment specialist focusing on the development, implementation and analysis of large-scale assessment projects. She is pursuing a Ph.D. in educational psychology and an M.A. in quantitative methods from the University of Texas at Austin 12
B. Project Plan The timeline of project activities, below, assumes project start date of September 3, 2012 and a project end date of October 31, 2013. Project milestones are noted. Date Task Begin End 6/4/2012 8/24/2012 Project Planning & Materials Development* 6/4/2012 8/24/2012 Record lecture captures and develop in-class activities 6/18/2012 7/2/2012 Create assessment plan and identify common assessments across semesters 7/2/2012 7/20/2012 Develop new assessments & scoring rubrics 7/20/2012 8/27/2012 Arrange faculty meeting with mentor 9/3/2012 5/24/2013 Data Collection 9/3/2012 12/7/2012 Conduct classroom observations 9/3/2012 9/21/2012 Gather student pre-test data 9/3/2012 9/30/2012 Create mid-semester survey 10/1/2012 10/5/2012 Arrange faculty meeting with mentor 10/8/2012 10/12/2012 Administer student mid-semester survey 10/15/2012 10/26/2012 Review survey feedback for formative assessment purposes 11/5/2012 11/16/2012 Create end-of-semester survey and TA/instructor survey 11/19/2012 11/23/2012 Arrange faculty meeting with mentor 11/19/2012 11/30/2012 Draft mid-project report 11/30/2012 11/30/2012 Deliver mid-project progress report to Echo360 11/26/2012 11/30/2012 Administer end-of semester survey 11/26/2012 12/6/2012 Gather student post-test data 12/6/2012 12/17/2012 Gather course grade and drop records 12/10/2012 12/14/2012 Administer TA/instructor survey 12/17/2012 12/21/2012 Review survey feedback for formative assessment purposes 1/14/2013 1/25/2013 Arrange faculty meeting with mentor 1/14/2013 1/25/2013 Gather student characteristic data 1/28/2013 2/1/2012 Gather CIS data 1/28/2013 2/8/2013 Create student follow-up survey 4/15/2013 4/26/2013 Administer student follow-up survey 5/15/2013 5/24/2013 Gather subsequent course grade and drop records Data Analysis 1/7/2113 1/18/2013 Analyze classroom observations 1/21/2013 2/1/2013 Analyze TA/instructor survey 2/4/2013 2/15/2013 Merge student characteristic, test, assignment, grade, and survey data 2/18/2013 3/15/2013 Analyze student learning data 3/18/2013 4/26/2013 Analyze student and instructor attitude data 4/29/2013 5/17/2013 Analyze student follow-up survey data 5/27/2013 6/7/2013 Analyze subsequent course success data 6/10/2013 6/28/2013 Draw conclusions and develop recommendations Report Results 7/1/2013 7/12/2013 Draft AERA conference proposal 7/15/2013 8/2/2013 Draft final report 8/5/2013 10/31/2013 Draft article for publication 10/31/2013 10/31/2013 Deliver final project report to Echo360 1/31/2014 1/31/2014 Article submission and Echo360 presentations to have taken place * Project planning and materials development will take place prior to the grant funding period 13
14