Designing a Blended Course: Using ADDIE to Guide Instructional Design

Transcription

1 RESEARCH AND TEACHING Designing a Blended Course: Using ADDIE to Guide Instructional Design By Ike Shibley, Katie E. Amaral, John D. Shank, and Lisa R. Shibley The ADDIE (analysis, design, development, implementation, and evaluation) model was applied to help redesign a General Chemistry course to improve student success in the course. A team of six professionals spent 18 months and over 1,000 man-hours in the redesign. The resultant course is a blend of online and face-to-face instruction that utilizes stable base groups throughout the face-to-face portion of the course. Students complete online assignments prior to class and must take a weekly online quiz each week. During face-to-face time, students answer clicker questions in groups throughout each class so class lecture has been essentially eliminated. The systematic attention to each step in ADDIE provided a well-organized process to help guide the team throughout the redesign process. The resulting design has significantly increased the average GPA for the course and significantly decreased the failure rate. A team of six individuals worked to redesign the firstsemester General Chemistry course taught at a small branch campus of a research university. The team chose to use a blended approach and followed the ADDIE (analysis, design, development, implementation, and evaluation) model for the redesign (Dick, Carey, and Carey 2005). Using this structured approach helped guide the team through a number of critical decision points to arrive at a design that has been used for the past three years to teach almost 1,000 students. During the past decade, higher education has seen the growth and development of blended learning, sometimes called hybrid learning. These blended courses use various online components, including lessons, activities, resources, and assessment (Garrison and Vaughan 2008). Higher education institutions have recognized the need to support faculty in the development of these blended courses because of the unique challenges associated with blended learning (i.e., technology, instructional strategies, new modes of communication and assessment). Because the terms blended and hybrid learning have only recently emerged in higher education, there is currently no authoritative, generally accepted definition (Osguthorpe and Graham 2003). For the purposes of this paper, the following definition was adopted: Blended learning is the thoughtful fusion of face-to-face and online learning (Garrison and Vaughan 2008). The blended course design focused on transforming the nature of the traditional lecture course so that students would be more actively engaged with course content both inside and outside of the classroom. In the past, the course had been taught using a variety of methods that depended primarily on the instructor. Most course delivery was via lecture with little to no online component. The redesigned course aimed to create a more engaging learning environment with a corollary goal of establishing more consistency among sections. An expected outcome of the blended course design was improved understanding of chemistry as demonstrated by an improved course GPA. To achieve the goal of shifting to a more active learning pedagogy, the design team focused its efforts on migrating content to a course management system so that students could interact with the content outside of the classroom. The nature of much of the online content included extensive multimedia and interactive activities. With much of the content moved outside the classroom, the team focused on creating a peer-led team learning environment that allowed students to work collaboratively with course content inside the classroom. ADDIE method In January 2005, college administrators began considering several possible ways to enhance the learning environment on campus. An examination of all courses taught at the college identified the first semester of General Chemistry as a multisection course with one of the lowest average grades. The suggestion was made to create a blended course to increase both student satisfaction and retention. The administration expected that a blended course would 80 Journal of College Science Teaching

2 enhance the students learning in a measurable way. At the beginning of the fall 2005 semester, the administration assembled a group of professionals to design the course. The design team consisted of the two chemistry faculty, the director of the Center for Learning and Teaching, an instructional designer, a multimedia specialist, and the director of Planning, Research, and Assessment. The course design occurred during a two-year period from August 2005 to August 2007, with most of the significant design and development occurring during the first year. The redesigned chemistry course was first offered in August Analysis The analysis stage of the design focused on assessing student learner characteristics, as well as identifying learning objectives for the course. Because one of the underlying motivations behind the design was to create a more consistent course among all sections, learning objectives were developed and shared among the other chemists at the college to arrive at an agreed-on set of learning goals that consisted of six or seven specific goals for each of the 12 units taught in the course. Difficult concepts were identified via an analysis of exams from a previous semester. More detailed learning goals were created for the more difficult concepts. Once the learning objectives were created, the design team focused on identifying those objectives that students struggled with the most. Both quantitative (previous test scores) and qualitative (instructors and students perceptions from previous courses) data were analyzed. As the design moved forward, the team analyzed how well the objectives were being met. Additionally, a review of the learning environment in the course was conducted via the Chemistry Self-Concept Inventory for students in classes prior to the start of the blended course (Bauer 2005). Analyzing student attitudes was deemed important as the design phase began because attitudes could help inform the online and face-to-face activities that were created. Design Blended courses usually have decreased face-to-face time. The firstsemester General Chemistry course previously included a weekly onehour review session, but the blended design obviated the need for that session because students were provided online activities in which they interacted with the content. The faceto-face course time was therefore trimmed by 25%. Although many blended courses have a more drastic reduction of face-to-face time, the design team determined that the difficulty of chemistry for the typical student necessitated that the course continue to meet three hours per week. Although reduction of class time was one goal, increasing student success in the course was a higher priority. The design stage focused on instructional strategies that would enhance student interaction with the course content (Ross and Schulz 1999; Trindade, Fiolhais, and Almeida 2002). During this stage, the design team discussed broader learning outcomes that could be taught in addition to the traditional course content. The most important learning outcomes identified were critical thinking, team work, and logical analysis. The design team decided that creating a collaborative learning environment that allowed students to work in teams during face-to-face interactions would accomplish these goals. The most difficult concepts identified in the analysis stage became the intentional focus of these interactions (Olapiriyakul and Scher 2006). All concepts would be included online, but special attention would be paid to the development of multimedia resources that would provide online assistance for the most difficult concepts. Development After careful analysis and design, the team started to work toward the creation of the online and face-toface portions of the course. The faceto-face interactions are explained in the next section. The main time spent in development was for the online resources. The design team unanimously decided that the best strategy to help students learn online was to create a class guide (see Figure 1) that would lead the students through the most important course content. The guides were organized so that topics would be first introduced online and then explored more fully during face-to-face work. Each guide included the following components: a table of contents, topic/ chapter learning goals, action items, and a learning resources page. The goal of the class guide was to create a functional student resource that would clearly identify, scaffold, and organize critical content. A guide includes action items that help students stay organized. The guides also include learning materials designed to enhance the students out-of-class learning experience on difficult topics and concepts (see Figure 2). The online resources included instructor-selected clips (video, animations, and graphics provided by the textbook publisher), supplementary internet-based multimedia resources, locally developed interactive online tutorials, and Vol. 40, No. 6,

3 research and teaching instructor-created minipodcasts (3 5 minutes in length). Choices were made by searching for free-access learning objects that addressed the most important learning goals identified in the analysis phase. The multimedia FIGURE 1 Example of class guide table of contents. FIGURE 2 Example of additional multimedia learning resources. specialist then helped modify many of these learning objects to specifically address the learning goals. A key component of the class guides is the content associated with the action items, such as identifying the critical concepts that students should focus on in reading their course textbook. By creating a guide that identifies specific parts of the book, the designers tried to avoid excessive verbiage. The class guides include a preclass assignment that requires students to answer questions about activities within the class guide as a means of ensuring that students utilize the class guide prior to class. The preclass assignments are worth a small number of points; the intent was to encourage students to spend time-on-task prior to face-toface time. The assignments are posted by the students in ANGEL, and the students receive full credit for completing them. Weekly online quizzes are designed to test students on the material following the week s material (over the weekend). The quizzes are multiple choice and are automatically graded, reducing the instructor investment in grading. Implementation The implementation stage of the design began in August 2006 with the beginning of the fall semester. The blended design has been used in every section taught at the campus since 2006: 15 sections with approximately 65 students per section. The use of collaborative group work begins prior to the first class, when students are assigned to heterogeneous groups by their instructor (Johnson, Johnson, and Smith 1991; Millis and Cottell 1998). Criteria for sorting entails each student s intended major and gender, with no student being the sole male or female in his or her group. The groups are assigned in the very first class period, and the groups are immediately engaged in activities designed to break the ice. Group members are instructed to exchange phone numbers, screen names, and 82 Journal of College Science Teaching

4 other contact information. In subsequent classes, the groups are expected to sit together by rearranging their desks into a circle. The assigned work in class consists almost entirely of questions disseminated via a classroom/student response system. Each student is required to purchase a transponder, familiarly called clickers, at the bookstore. The clickers provide instant feedback to the student. During each class session, the clicker questions provide scaffolding for students as they answer lesscomplicated questions first and work toward the most complex questions by the end of the class period. The clickers also provide immediate feedback to the instructor, who can decide how best to proceed. If students do well on several similar questions, then the instructor can skip a few questions to move to the next concept. The group members sometimes work alone at first but then ask each other for help as they compare work that they have done. Each group seems to develop its own rhythm as students articulate their answers to each other, listen to solutions posed by other group members, and depend on their colleagues. Peer mentors are present in the class to assist the students with their group interactions and to help them as they struggle with the chemistry concepts and problems. The peer mentors have previously taken the course and have evidenced good helping behaviors. The mentors move from group to group within their designated area offering tips, advice, analogies, and study strategies while helping to keep the groups focused and working on task. Two other decisions were made in this phase. First, because of the group format of the course, the decision was made to offer the course twice a week for 75-minute sessions. The usual approach was to offer the course three days per week for 50-minute sessions. Second, the team decided to make the grading system for the redesigned course similar to earlier versions with quizzes and exams constituting the bulk of the grade in order to ensure individual student accountability. Evaluation The evaluation of several sections of the newly designed course has been completed. One conclusion is that the redesigned course has improved the average student grade in the course. GPA was calculated for all sections of the course taught five years prior to the design and compared to all sections taught in the past two years. Because the spring sections historically have lower SAT scores than fall sections, the students were separated into four groups: (1) fall nonhybrid (919 students), (2) fall hybrid (351 students), (3) spring nonhybrid (640 students), and (4) spring hybrid (246 students). The letter grades that the students received in the course were converted to numerical values on the 4.0 scale in use at the university (see Table 1). Using an independent samples z-test, students who had taken the course in the blended format during both the fall and spring semesters had a significantly higher GPA than did those students who had taken the course in the lecture format, as shown in Figure 3 (1.77 GPA for nonhybrid vs GPA for hybrid fall students, p =.000; 1.75 GPA for nonhybrid vs GPA for hybrid spring students, p =.010). The grades are broken down further in Figure 4. The breakdown of grades shows that, in the hybrid sections, the percentage of F and W grades decreased while B and A grades increased. TABLE 1 GPA equivalents of letter grades. GPA equivalent Letter grade 4.00 A 3.67 A 3.33 B B 2.67 B 2.33 C C 1.00 D 0.00 F 0.00 W Note: W = withdrawal from the course. Another measure of success is the number of students earning a grade of C or better (GPA 2.00) in the course. The ratio of students who received a passing grade (A through C, inclusive of + and ) compared with those who received a D, F, or W grade was calculated. Because the course is a C-required course for most majors, a grade of D means that the student will need to retake the course. A higher ratio indicates a higher success rate. The students who were in the blended course during the fall semester had a higher success rate (2.41 students passed for every one that failed) than the students who took the course in a lecture format (1.61 students passed for every one that failed, p =.000). The spring semester students experienced a smaller, but still significant, increase in success in the blended format (2.08 students passed vs students passed for every one that failed, p =.022). Students received a questionnaire at the end of each semester asking them to rate several aspects of the course. One of the open-ended questions asked the students to list tools that were helpful to them when they Vol. 40, No. 6,

5 research and teaching were trying to learn the concepts in the course. Because the question was open ended, students needed to select those parts of the course that helped them without any prompting. Over 40% of the respondents listed online quizzes as being helpful, and 35% listed the class guides. A 5-point Likert-type scale was used to ask whether the blended design was effective in helping to understand the concepts in the course. Over 90% of respondents either strongly agreed or somewhat agreed that the blended design was helpful. FIGURE 3 Average GPAs in General Chemistry. FIGURE 4 Grade distribution in General Chemistry. Discussion The course described in this paper has now been taught in six consecutive semesters. As a result of the success of the course redesign, every indication is that the course will be taught using a blended format from this point forward. The major components of the course are listed in Table 2. Most of the course was altered by applying the ADDIE model, including the creation of online class guides, associated learning objects (multimedia course content resources), and collaborative base groups. The design elements seemed to work synergistically; the use of class guides allowed the instructor to move lecture content online and freed up class time to allow students to work collaboratively in the classroom. Although there is literature on peerled team learning in chemistry (Gafney and Varma-Nelson 2007; Wamser 2006; Lyle and Robinson 2003; Gosser and Roth 1998), little work has been published on the use of blended learning in chemistry. Blended learning course redesigns are still relatively new to higher education and, as such, will need further research and investigation in order for us to more fully understand and develop best practices in the field. Chemistry is a discipline with complex, analytical content that is often difficult for students. As pedagogies are developed to enhance learning, difficult subjects like chemistry may benefit from research on how to apply learning strategies such as the ones described here to enhance student success in learning challenging content. The initial results from this course design suggest that when technologies and instructional strategies are appropriately aligned and thoughtfully integrated into the instructional process, a blended learning environment can facilitate and enhance face-to-face as well as out-of-class student learning. One way to view the student success is through the lens of time-on-task: Students have more structured learning opportunities outside of class than they have had previously. The increased time-on-task seems to improve learning. The success of the General Chemistry course described in this paper mirrors other successful blended learning course redesigns (Garnham 84 Journal of College Science Teaching

6 and Kaleta 2002; Riffell and Sibley 2005; Olapiriyakul and Scher 2006). Future work will therefore seek to build on the success of this design. Assessment will be directed toward specifically measuring the impact of individual components of the course. Additional research questions will involve exploration of the various aspects of the course redesign. One such question involves the efficacy of the various multimedia components of the class guides. Another involves student attitudes toward the collaborative learning as well as the use of the student-response system. Still another involves the design of blended courses for complex course content such as general chemistry. Although it is interesting to try to tease apart how the different aspects of the course design helped to facilitate learning, the main outcome of the project was that dramatic changes in student success are possible. Work has therefore begun on designing the second semester of chemistry as a blended course. In the near future, Organic Chemistry will be redesigned. The goal is to have a seamless chemistry curriculum that uses a similar pedagogical approach in all courses. The pedagogical benefits achieved in this blended course redesign can hopefully be replicated as appropriate in other courses across the curriculum. n References Bauer, C.S Beyond student attitudes : Chemistry Self-Concept Inventory for assessment of the affective component of student learning. Journal of Chemical Education 82 (12): Dick, W., L. Carey, and J.O. Carey The systematic design of instruction. 6th ed. Boston, MA: Allyn & Bacon. Gafney, L., and P. Varma-Nelson TABLE 2 Significant elements of redesigned course. Face-to-face (time in classroom) Use of clickers Stable, heterogeneous groups (four per group) Peer mentors to assist students Minilectures on concepts deemed most difficult Evaluating peer-led team learning: A study of long-term effects on former workshop peer leaders. Journal of Chemical Education 84 (3): Garnham, C., and R. Kaleta Introduction to hybrid courses. Teaching with Technology Today 8 (6). www. uwsa.edu/ttt/articles/garnham.htm Garrison, R.D., and N.D. Vaughan Blended learning in higher education: Framework, principles, and guidelines. San Francisco: Jossey-Bass. Gosser, D.K., Jr., and V. Roth The workshop chemistry project: Peer-led team learning. Journal of Chemical Education 75 (2): Johnson, D.W., R.T. Johnson, and K.A. Smith Active learning: Cooperation in the college classroom. Edina, MN: Interaction. Lyle, K.S., and W.R. Robinson A statistical evaluation: Peer-led team learning in an organic chemistry course. Journal of Chemical Education 80 (2): Millis, B.J., and P.G. Cottell Cooperative learning for higher education faculty. Westport, CT: Oryx. Olapiriyakul, K., and J.M. Scher A guide to establishing hybrid learning courses: Employing information technology to create a new learning experience, and a case study. Internet and Higher Education 9 (4): Osguthorpe, R.T., and C.R. Graham. Online (out-of-class time) Class guides completed prior to class Drop boxes for preclass assignment Weekly quizzes completed after class Multimedia exercises on most difficult concepts Blended learning environments, definitions and directions. The Quarterly Review of Distance Education 4 (3): Riffell, S., and D. Sibley Using web-based instruction to improve large undergraduate biology courses: An evaluation of a hybrid course format. Computers & Education 44 (3): Ross, J., and R. Schulz Can computer-aided instruction accommodate all learners equally? British Journal of Educational Technology 30 (1): Trindade, J., C. Fiolhais, and L. Almeida Science learning in virtual environments. British Journal of Educational Technology 33 (4): Wamser, C.C Peer-led team learning in organic chemistry: Effects on student performance, success, and persistence in the course. Journal of Chemical Education 83 (10): Ike Shibley (ias1@psu.edu) is an associate professor in the Chemistry Department, Katie E. Amaral is an assistant professor in the Chemistry Department, and John D. Shank is an associate librarian, all at Penn State Berks in Reading, Pennsylvania. Lisa R. Shibley is assistant vice president in the Department of Institutional Assessment and Planning at Millersville University in Millersville, Pennsylvania. Vol. 40, No. 6,

7 Copyright of Journal of College Science Teaching is the property of National Science Teachers Association and its content may not be copied or ed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or articles for individual use.