WHY QUALITY MATTERS: STRATEGIES FOR DESIGNING QUALITY E-LEARNING ENVIRONMENTS

Transcription

1 WHY QUALITY MATTERS: STRATEGIES FOR DESIGNING QUALITY E-LEARNING ENVIRONMENTS Dr. Teresa Franklin Professor, Instructional Technology Patton College of Education Ohio University Abstract This paper examines the need to improve the quality of course development and delivery in online environments. The Quality Matters rubric will be discussed as an assessment framework for the connection between delivery and design using peer assessment to guide the development of quality online courses that are studentcentered and support active learning and student engagement in the online environment. The Quality Matters rubric can be used to improve course design as well as engage faculty in a discussion concerning the quality of e-learning courses offered on a university campus. INTRODUCTION In this discussion, e-learning and online learning are viewed as the same delivery method in which the instructor and students are separated by time and distance and content may be delivered synchronously or asynchronously depending upon the design of the course, faculty and student abilities as well as institutional decisions concerning the delivery of content across the internet (Moore, 1993). The terms may at times be used interchangeably when discussing course design and delivery as within the United States the term online learning is more readily used while internationally the term e-learning is used (Moore, Dickson-Deane, Galyen, 2011). Whether called online learning, e-learning, distributive learning, distance learning, virtual learning, or MOOC, universities are examining the need to expand the delivery of their courses within an electronic delivery system. Many universities do this to provide more access to people who are not in locations conducive to attending a face-to-face campus course, while others have seen a drop in campus enrollment and want to use this electronic delivery as a means of balancing the budget within the university by acquiring more students in online environments. For the discussion within this paper, the term online learning will be used to describe those courses that are delivered electronically across the Internet and have a faculty member in one location and students in other locations accessing information in a common electronic location and format (Moore, et al., 2011). 35

2 Students are also part of the issue driving universities to seek online environments as a way to lessen the burden of costs and loans associated with earning a degree. Students having grown up in a digital environment want a more flexible means of obtaining an education, allowing them to stay at home to cut costs in earning a degree. Parents with small children, who do not have child care, are also seeking to earn a degree without leaving home. Finally, the online environment allows for 24/7 engagement, thus, providing educational access to many different styles and times for learning by individuals. THOUGHTS ON QUALITY COURSE REDESIGN Myths of e-learning Course Development There are many myths that surround the development and delivery of online courses. In particular, students see online courses as easier and self-paced. In the majority of e-learning and online delivery, courses a course schedule with required due dates and deadlines. The anywhere/anytime notion typically refers to the access to the course in which the student has 24/7 access although the instructor is not present and the work is completed in a more flexible manner than the typical traditional classroom. An exploration of some of these myths is in order to help faculty begin to understand the differences when moving to an online/elearning environment. Myth 1: My course content is the same online or face-to-face. Many beginning online educators believe that the content of their course can be taken directly from the classroom to electronic without any change in design or delivery. Unfortunately, this is not the case. Those notes from the classroom have to be developed into an electronic format, edited, possibly presented as video and then paced to work within the delivery of the learning management system (LMS) being used at the institution. No matter the LMS product being used, there will be a particular way in which the content is uploaded and viewed. The faculty member must be aware of these differences and plan the delivery of the content accordingly. While the actual content may be the same, the delivery of that content will be different (DeMari, R., & Bongiovanni, 2010). Myth 2: Online learning is not personal and my passion does not translate. The thought that an online classroom is impersonal can also be played out in the traditional classroom. Students can be allowed to lurk in either setting if the faculty member is willing to just ignore the lack of participation by the student. Students often note their lack of participation in the real-world classroom and may attempt to carry this to the online classroom space. The perceived anonymity of the computer screen and the fact that the student is working from the personal environment of his/her home allows many students to take risks and share personal and professional learning when in the LMS as this online environment is a familiar place. Technology can bring students together when there are problems within the LMS in a way that promotes a connectedness of surviving the technology (DeMari, R., & Bongiovanni, 2010). In the same way thoughtful preparation is required to engage students fully in the 36

3 passion of learning in a face-to-face environment, the online learning environment requires the same. The passion of the educator comes with preparation of a learning environment that facilitates active and engaged learning to draw the student into the passion (DeMari, R., & Bongiovanni, 2010). Myth 3: Everyone knows how to use the technology but me and is doing other things when I am teaching. The e-learning/online classroom requires a teaching delivery that matches with the characteristics of the virtual environment. This can be daunting for an instructor who may lack needed technology skills. The virtual classroom has all the same communication features of the traditional classroom -- slides, flip boards, whiteboards, voice, video -- they are just in a different version. Using these well-established classroom tools to deliver learning online limits instructors' options and ability to engage students. They must to use classic tools differently and exploit the virtual qualities of the learning experience. The virtual classroom can becomes a safer place for creative risk taking. Instructors can also become so enamored with the technology they lose site of the delivery of content. Students, who are active users of technology, find the online environment a great distraction and the ability to discuss electronically with friends on Facebook or SMS, makes the issue of control difficult for some instructors. The best strategy is to use the ability of the virtual environment allow for texting, chatting, and breakout into group areas to accelerate learning and deeper understanding of the content. Simply using these well-established classroom tools to deliver learning online limits instructors' options and ability to engage students. In fact, educators risk becoming disembodied online voices presenting "stuff" they need to use these tools differently and exploit the virtual qualities of the learning experience. The virtual classroom becomes a safer place for creative risk taking. The technological expertise of the instructor in operating the virtual classroom builds confidence in working in this new venue but the technology itself is not the end result. While myths will continue to abound in the area of online, e-learning and distance education as a whole, most of the world is moving forward in using these virtual environments to reach the many students that seek flexibility and access to learning. First Look at Quality in Course Redesign In April of 1999, the Pew Charitable Trust funded an 8.8 million dollar experiment to determine the nature of online course development called the Program in Course Redesign ( The work hosted by Rensselaer Polytechnic Institute, sought to help colleges and universities in their efforts to redesign classroom instruction through the infusion of technology in the hopes of not only achieving quality course design but cost savings as well (Twigg, 2003). This work is now seen as seminal to the beginning discussions on course redesign in the movement to virtual learning. The project managed by the Center for Academic Transformation, each institution was required to focus on learning outcomes measured by student performance and achievement 37

4 along with a rigorous evaluation of these measurements. Experts provided oversight to examine the assessment to ensure reliability and validity. Cost analysis of the redesign was also a consideration, as the Center noted that change has a cost and this cost has to be reasonable or institutions will not engage in redesign. Out of the work of the Pew Charitable Trust funding and the Center for Academic Transformation six characteristics of course redesign were identified (Twigg, 2003). Whole Course Redesign Active Learning Computer-based Learning resources Mastery Learning On-demand Help Alternative Staffing (Twigg, 2003, p. 30) While each college or university used these six characteristics, each characteristic was impacted by the discipline involved, the student body make-up in the institution and the faculty teaching the course. After an examination of the different grant projects within institutions, five models emerged with the range being denoted along a continuum of fully online to fully face-t0-face. The following examples, (1) supplemental, (2) replacement, (3) emporium, (4) fully online and (5) buffet, were identified and suggest ways in which the characteristics of redesign can be embedded within the learning model of the institution as well as support improved course redesign (Twigg, 2003). The Supplemental Model. The supplemental model kept the lecture intact as a delivery method in the course and continued the number of class meetings typical to the course. A group of supplemented lectures, and textbooks with additional computer-based activities were added. Three weekly online mastery quizzes were developed which allowed the student to retake the quiz until a perfect score was reached (Twigg, 2003, p. 31). The Replacement Model. This model reduced the number of class-meeting time replacing face-to-face time with activities, which included online meetings to engage the student in interactive learning within small groups. The redesign identified that some of the activities of the class could be better developed for online learning and that group work could be completed at any location and outside of course contact time thus expanding the time on task for the student in completing coursework. This model does not assume that face-to-face meetings are the best setting for all activities and that classes need to meet according to the desired learning outcomes not just so that the students are face-to-face (Twigg, 2003, p. 32). The Emporium Model. The emporium model was first developed at Virginia Technical University was developed on the idea that students learn best when ready to learn. Students of mathematics were allowed to choose when to access the course materials, which materials to use and how quickly to work through the content with the support of instructional software and one-on-one face-to-face help. This model removes all class meetings and replaces them 38

5 with a learning resource center. The online materials in the learning resource center include on-demand personal assistance. It is easy to see that this model requires extensive commitment of time, space and equipment to be successful (Twigg, 2003, p ). The Fully Online Model. The use of fully online in the study was limited due to the understanding that in most cases faculty work alone in the design and development making this a labor-intensive endeavor. The one fully online example completed in the study was based on the Academic Systems mathematics software and the addition of a nonacademic course assistant. The Academic Systems software presents the content at such a quality level that instructors do not spend course time delivering the content. The course non-academic assistant was responsible for non-math questions and to monitor progress. This allowed the instructor to concentrate on academic interactions with students. Class size was increased to 100 students concurrently, which was the typical amount for 4 classes. The instructor took advantage of the Academic Systems software s large bank of problems and answers for each topic to increase comprehension and past. A built in tracking system allows the instructor to know every student s performance including the time on task in each module. Successful completion of math was increased by 6% and the number of sections taught decreased while class size increased (Twigg, 2003, p. 36). The Buffet Model. The final model of the quality redesign to successfully improve the quality of student learning while reducing the cost of instruction is the Buffet model, which takes into account the individuality of students. This model seeks to personalize the instruction through the use of information technology. This model customizes the learning environment for each student by offering an assortment of interchangeable paths to match their individual learning styles and abilities. These offerings included lectures, individual discovery labs both in-class and web-based, group reviews, small group study sessions, remedial procedures, training modules, contractual learning modules, written presentations, problem-solving, large group activities, homework assignments and graded or self-graded projects. Each student was provided with a buffet of learning resources, contexts and designs in which the student determined the best way to learn the material for the course. To increase the likelihood of success, each student enters into a contract that indicates the student choices for learning, an orientation of the buffet style of learning study skills assessments and suggestions for completing the course (Twigg, 2003, p ). All of the models have a unique manner for engaging students in the learning process. What was unique across all models was the treatment of courses as a set of products and services continually being improved. The dedication of faculty teams to work to redesign these courses spoke to their dedication to student learning. It is important to note, however, that the information technology enabled best practices to be captured in the form of interactive Web-based materials and sophisticated course-management systems to help faculty view the performance of students and to adjust accordingly. 39

6 TPACK as a Theoretical Model for Online Development At the heart of the work of present day educators are the three knowledge bases of content, pedagogy and technology. As each knowledge base (content, pedagogy, and technology) enter the classroom setting, there is a relationship of each with the student and also with the teacher as well as the interaction of these components and the unique understanding that we have of each. These three knowledge bases (content, pedagogy, and technology) form the core of the technology, pedagogy, and content knowledge (TPACK) framework (e.g., Koehler & Mishra, 2008; Mishra & Koehler, 2006), which can be used to extend the quality of instruction and engage students in higher order thinking skills. This new framework presented by Mishra and Koehler in 2006 is being examined as a framework for the development of new an more integrative ways to think about teacher practice and the integration of technology. The TPACK perspective is consistent with Shulman s (1968) idea of pedagogical content knowledge (PCK) and now seeks to include educational technology. The framework of TPACK can play a major role in the development of online learning environments as instructors seek to understand the use of technology to support their content and identify ways in which the traditional class pedagogy can be implemented in the online environment. As an instructor critically examines his/her own content, pedagogy and technology use, an understanding emerges from the interactions among content, pedagogy and technology knowledge. TPACK is the basis of effective teaching with technology, requiring an understanding of the representation of concepts using technologies; pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help redress some of the problems that students face; knowledge of students prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge to develop new epistemologies or strengthen old ones. By simultaneously integrating knowledge of technology, pedagogy and content, expert teachers bring TPACK into play any time they teach. Each situation presented to teachers is a unique combination of these three factors, and accordingly, there is no single technological solution that applies for every teacher, every course, or every view of teaching. Rather, solutions lie in the ability of a teacher to flexibly navigate the spaces defined by the three elements of content, pedagogy, and technology and the complex interactions among these elements in specific contexts. (Koehler, M. & Mishra, 2009, p. 66) 40

7 With the increased use of online environments which are inherently based in technology and its use, the TPACK framework offers several possibilities for research in the examination of faculty professional development and faculty s use of technology as well as options for examining the complex phenomenon of technology integration not only by instructors of online courses but students within those courses as creators of content (Peruski & Mishra, 2004). The use of technology tools for analysis within LMS environments may provide researchers with opportunities to focus on the ecological integration of technology by faculty and student in virtual learning environments. How do these three core knowledge bases, pedagogy, content and technology play out within the context of anywhere, anytime learning? Does the solutions lie in the ability of an instructor to flexibly navigate the spaces defined by the three elements of content, pedagogy, and technology and the complex interactions among these elements in specific contexts? (Koehler, M. & Mishra, 2009, p. 68). The Quality Matters Process Faculty development is a critical component for any robust online program. The Quality Matters Program (QM) is an international organization of broad institutional sharing and collaboration in an effort to understand online course quality. QM is a quality assurance process that has been developed to improve and certify the design of online and blended courses (Quality Matters Program, 2013, para. 1). The membership with QM is by institutional subscription although there is opportunity for individual subscription at a much higher cost. These membership institutions work together to provide trained peer reviewer and support other institutions as they implement the QM process (Quality Matters Program, 2013). The QM Program is a not-for-profit subscription service that provides the tools and training to support a quality assurance process in online course design and comes from the work of the University of Maryland s participation in a FIPSE grant. All areas of the QM process are focused on student learning. The QM process recognizes that the faculty member is an integral part of both course design and course delivery. To this end, QM provides intense professional development to member institutional faculty. The QM process provides on-site, online, and Web-based professional development opportunities to instructional designers, faculty, administrators, and adjunct instructors (Quality Matters Program, 2013). Quality Matters promotes a peer review process in which QM Peer Reviewers are selected from a database of trained professionals to review a course per an institution or faculty request. Any subscribing institution may conduct internal or informal reviews or contract with Quality Matters to conduct an official review. Courses that successfully meet the QM rubric standards in an official course review are eligible for QM recognition. QM is 41

8 dedicated to the continuous improvement of online course design. The QM process is composed of four distinct characteristics and the process is: 1. Continuous a. The process is designed to ensure that all reviewed courses will eventually meet expectations. b. The process is a rubric-based review integral to a continuous quality improvement process. 2. Centered a. The development of the rubric is based on national standards of best practice, the research literature, and instructional design principles. 3. Collegial a. The review is part of a faculty-driven, peer review process. b. The review process is intended to be diagnostic and collegial, not evaluative and judgmental. 4. Collaborative a. The review is based on collaboratively identified evidence found in the course rather than the personal preference of an individual reviewer. b. The review is flexible and not prescriptive (many ways to meet each standard). c. The review team consists of three experienced online instructors as reviewers along with the course faculty developer. The QM process is about course design, which is seen as the forethought, and planning a faculty member puts into the development of a course. Course delivery is about the actual teaching and implementation of the course design. QM is about design not delivery or faculty performance and should be seen by administrators seeking to improve online quality at an institution as a first step in securing faculty quality in online teaching and learning in the institution. The QM Rubric. There are many factors that influence course quality. These factors include: (1) course design, (2) course delivery, (3) course content, (4) institutional infrastructure, (5) learning management system, (6) faculty readiness and (7) student readiness. While there are many factors, the QM Rubric only examines course design. The Quality Matters Rubric contains 8 general standards and 41 specific standards to evaluate the design of online and blended courses. The Rubric is complete with annotations that explain the application of the standards and the relationship among them. A scoring system and set of online tools facilitate the evaluation by a team of reviewers (Higher Ed Program>Rubric, 2013, para. 1). The rubric is divided into the eight critical course standards below: 1. Course Overview and Introduction 2. Learning Objectives (Competencies) 3. Assessment and Measurement 4. Instructional Materials 42

9 5. Learner Interaction and Engagement 6. Course Technology 7. Learner Support 8. Accessibility Each of these standards is further divided to help the reviewer and faculty member identify the elements of the standard within each course component. Each element is given a point value of 1 to 3 that is used in the scoring of the course standard. An instructional designer or faculty member developing a course can use the rubric not only as a peer review of their course but to guide them in the development process. Institutions and faculty benefit from the peer review process through improved consistency and rigor of course design, professionalism and commitment to online learning and useful and constructive feedback. It is important to note that less than 50% of courses in a QM-managed review meet the rubric standards upon initial review; however, all meet the requirements after amendment of the site to meet the deficiencies identified in the review. The Quality Matters Rubric can be found at The rubric is copyrighted so the use of the rubric as an individual for one time use is allowed; however, use within an institution would require a subscription to the Quality Matters Program which at the Basic level is $ per year at the time of the writing of this paper. In the end, the goal of the Quality Matters Process is to improve online instruction to facilitate student learning and ensure institutional quality in the delivery of online content. DISCUSSION AND CONCLUSION The use of the Quality Matters Rubric provides a foundation in which the content, pedagogy and technology of the TPACK framework can be used to examine course design. While the characteristics provided in the discussion on redesign are clearly identified as needed, the Quality Matters Rubric takes the redesign to a new level. Just a quick examination of the list of critical course standards suggests that the use of the TPACK framework could easily support the theoretical research of a faculty member in course design. TPACK can be seen in the design of course objects (content); instructional materials (content); learner interaction and engagement (pedagogy); accessibility (pedagogy) and course technology and learner support (technology) of the QM rubric. The QM Rubric places a means of self-reflection on a faculty member s course design that can allow for a continuous improvement model that not only informs the faculty member but also informs the university of course design quality. Universities can use this information to begin to develop professional development that embraces the QM Rubric and builds a set of standards for the institutions faculty to meet as they design their online courses. This is of particular importance in that most faculty work alone in the design of their course without an instructional designer, media specialist or pedagogical coach. Through self-reflection on the 43

10 interchange of technology, content and pedagogy and how it aligns within the QM Rubric for design of a course, the faculty member may find new and engaging ways to reach students and to engage in active learning in an online environment to improve student learning in their courses. REFERENCES Curtis, D. D. & Lawson, M. J. (2001). Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5(1), Moore, J. L., Dickson-Deane, C., Galyen, K. (2011). e-learning, online learning, and distance learning environments: Are they the same? Internet and Higher Education, 14, Pp Moore, M. G. (1993). 2 Theory of transactional distance. Theoretical Principles of Distance Education, 22. Twigg, C.A. (September/October 2003). New Models for Online Learning. EDUCAUSE Review, DeMari, R., & Bongiovanni, T. (2010). The 10 biggest myths about synchronous online teaching. EDUCAUSE Review. Retrieved November 10, 2013, from Higher Ed Program>Rubric. (2013). Retrieved from December 8, 2013, Koehler, M. & Mishra, P. (2009). What is Technological Pedagogical Content Knowledge (TPACK)?. Contemporary Issues in Technology and Teacher Education, 9(1), AACE. Retrieved December 7, 2013 from Peruski, L., & Mishra, P. (2004). Webs of activity in online course design and teaching. ALT-J: Research in Learning Technology, 12(1), Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), Quality Matters Program (2013). Quality Matters: A national benchmark for online course design. Retrieved on December 1, 2013,