Transcription

1 I. Introduction Quality Matters at Prince George s Community College A Proposal to Implement the Quality Matters Process June 6, 2005 Prince George s Community College is committed to ensuring student access to high quality distance learning course offerings. This report reviews the Quality Matters Project and the rubric it developed, summarizes the use to date (the Pilot Project) of the Quality Matters (QM) rubric to assess the quality of a selection of PGCC online courses, proposes PGCC adopt the QM rubric and course peer review process as its measure of quality for online courses, and outlines an implementation plan including some issues to be resolved for successful implementation.. II. The FIPSE Grant Quality Matters Project Quality assurance of online courses is critical to various stakeholders in higher education. However, perspectives vary when it comes to actually defining and assessing quality in an online course. In 2003, the U.S. Department of Education Fund for the Improvement of Postsecondary Education (FIPSE) awarded MarylandOnline a three-year grant titled Quality Matters: Inter-Institutional Quality Assurance in Online Learning to develop the tools, process, and infrastructure to do just that. The Quality Matters (QM; project is creating an interinstitutional continuous improvement process for assuring the quality of online courses. This process is faculty-centered, peer review-based, and grounded in research literature, best practices, and instructional design principles. The QM project involves numerous faculty, instructional designers and staff at Maryland Online (MOL) institutions, a consortium of 14 community colleges and 5 senior institutions dedicated to inter-institutional collaboration. The QM team identified 40 key elements of an online course, distributed across eight broad standards, shown to positively impact student learning. These elements were then used to create the web-based rubric with each element receiving a rating from 1 (important) to 3 (essential). For each element, annotations and multiple examples from real online courses were compiled. An example of the rubric, its annotations and scoring is provided in Appendix A. A team of three experienced online faculty members is trained to use the rubric to review an online course. The reviewers include one content expert and at least one member from

2 Proposal Page 2 outside the course s home institution. Review team members work both individually and collaboratively, and in communication with the course s faculty developer, to provide a compiled report that highlights exceptional elements and provides specific recommendations for improvement. The QM project then provides instructional design support to implement the recommendations. A course receives QM Recognition when it demonstrates all 3-point essential review elements and receives a combined total score of 68 out of 80 possible points. The grant has reviewed 37 courses and plans to review at least 65 courses during the project. III. Summary of Quality Matters Activity at PGCC Mary Wells serves as the QM project s co-director (Dr. Christina Sax, UMUC, is also a co-director). Dr. Andrew Habermacher, Professor of Anthropology, and Lauren Fairley- Wright, Coordinator, Distance Learning Center, serve as the Administrative Representatives to the grant project. By May 2005, twenty-three Prince George s faculty members have participated in the QM process either as course peer reviewers or as course developers; in Fall 2005, an additional seven faculty will participate in Peer Course Reviewer training; Eight online courses have been reviewed as of Spring 2005; EDU 205, Assessment of Students (taught by Dr. Tamela Hawley) will be reviewed through the grant in Fall Reviewed online courses and faculty participating in the QM process are listed in Appendix B. Pilot Project During FY 05, with the approval and support of Dr. Zdravkovich, Vice President for Instruction, and under the leadership of Dr. Andrew Habermacher, Professor of Anthropology, the college piloted three peer course reviews (CJT 251, EGL 101, PSY 101). Each review team consisted of two college faculty members (of whom, one was the committee chair), and one external reviewer. The chairs were: CJT 251, Professor Mara Doss; EGL 101, Dr. Carolyn Hoffman; and PSY 101, Dr. Mary Helen Spear. The course reviews were conducted in accordance with the QM guidelines for independent peer reviews; once the course meets expectations, it will be entitled to use the QM logo and will be recognized on the QM web site. IV. Recommendations Pursuant to the Pilot Project Peer Reviews Dr. Habermacher and Mary Wells, Director of the Distance Learning Center, met with the review team chairs following the completion of the course review to discuss the process and their perspective on implementing it at the college. A. The Review Team Chairs Comments and Recommendations

3 Proposal Page 3 1. The chairs approve of the QM process and believe that it works well and results in course improvement. 2. The chairs would like to see the QM process implemented at the college in a formal, comprehensive way, beginning in Fall The chairs do not believe that the QM process for online course review should be tied to the current college Assessment Process. The two processes are very different in their methodologies and outcomes. The QM process is very specific and concrete, relating to a single course taught be a single instructor. The college Assessment process relates more generally to a single course taught by many different instructors. B. Benefits to the Adoption of the QM Process for Continuous Improvement of online courses at PGCC 1. Demonstrate college commitment to excellence in online learning. 2. Assure continuous quality assurance of online courses through a systematic process. 3. Implement a process that is student-centered and faculty-driven. 4. Encourage and support on-going faculty development through the training of peer course reviewers. 5. Raise the visibility of quality standards in course design for online, hybrid and f2f classes. V. Proposed Implementation Plan At the conclusion of May 2005, the college has 142 existing online courses; this number does not include new courses developed in FY 05. Of these only 10 have been reviewed or are scheduled for review in Fall 05. The plan below is based on the concept that the college will review each existing online course over the next four fiscal years; after that each course, will be reviewed only once every five years. A. Selection of Courses Courses will be selected and scheduled for review using the following guidelines: 1. Courses with high enrollments will be selected first. 2. Courses selected for review must have been taught online at least for two semesters. 3. The division chair or dean requests a review. 4. The faculty member requests a review. B. Costs Each peer review has the following fixed costs associated with it: 1. $150 paid to an external reviewer to serve on each peer review team. The external reviewer must be a QM trained faculty member from another institution who has at least 2 semesters of online teaching experience

4 Proposal Page 4 Fiscal Year 2. $150 paid to the PGCC chair of the peer review team 3. $100 paid to each internal faculty reviewer. 4. $150 paid per course to MarylandOnline a. $75 paid to MarylandOnline to use the QM web-based rubric b. $75 paid to MarylandOnline for tracking, web recognition of courses that meet expectations. The cost per course to conduct a review is $ 550. $ 75 Use of MOL rubric 150 PGCC Team Chair 100 PGCC Internal Reviewer 150 External Reviewer 75 Tracking, web recognition $550 The annual costs for the next 5 fiscal years are summarized in the table below: Courses To Reviewed By QM PGCC At PG Faculty Reviewers Needed Ext $150/ course for chair Cost Internal Reviewers $100/ course for rev. Cost- Ext. Rev. MOL Costs Total Annual Cost for Reviews F Sp Su FY $ 450 $ 450 $ 900 FY $5,700 $3,800 $5,700 $5,700 $20,900 FY $6,000 $4,000 $6,000 $6,000 $22,000 FY $6,000 $4,000 $6,000 $6,000 $22,000 FY $6,000 $4,000 $6,000 $6,000 $22,000 FY $4,500 $3,000 $4,500 $4,500 $16,500 FY $4,500 $3,000 $4,500 $4,500 $16,500 C. College Faculty Participation Although it is important to review the college s online courses in a systematic fashion, it is also important that this review process be driven by faculty and provide an opportunity for on-going professional development. Faculty peer reviewers typically report that the QM training peer review process result in positive changes to their own online courses. To promote peer review as a faculty development activity, the college should establish an expectation that each online faculty member, whether adjunct or full time, will participate in at least one QM course review every other year. D. External Faculty Participation In keeping with the process established by QM, the college should continue to ensure that there is at least one external reviewer for every course team. This

5 Proposal Page 5 D. Training reviewer must be the QM criteria for a peer reviewer: taught online within the last two semesters. Faculty who serve as peer reviewers must participate in QM training prior to the review. All faculty training will be done in-house by Mary Wells and/or Rhonda Spells. Training can be accomplished with a one-day face-to-face session or a 3- week online workshop. E. Management Of This Process This process will be under the general supervision of the Vice President for Instruction. The QM process will be implemented by the Director of the Distance Learning Center; operational activities such as tracking, establishing a schedule, forming review teams and other such tasks will be undertaken by the Distance Learning Center staff. An advisory committee of faculty will be appointed by the Vice President for Instruction to provide guidance and support for this effort. E. Issues to Resolve. Issues which must be resolved to ensure the successful implementation of the QM process include: 1. Lack of instructional design support The College has a critical need for an Instructional Designer who will provide leadership and support to faculty in planning effective course design, incorporating instructional standards into courses, and incorporating technology into instruction. The position of Instructional Designer is key for an institution that is serious about teaching and learning; it provides a dedicated professional who can assure that high standards of instruction are developed, incorporated and assessed in an effective, systematic way. This is particularly important for the QM process which requires that all courses submitted for peer review must have access to instructional design support to make recommended changes and improvements. 2. Lack of administrative support within the Distance Learning Center The pilot conducted during Spring 2005 clearly indicated that this is a lengthy, complex process. Its immediate implementation in FY 06 can be covered by existing staff. As the number of online courses and enrollments increases, it will be important to monitor whether this existing staff can accommodate both the peer review operation as well as increasing demand and expansion. An additional support staff member should be included in the FY 07 budget. 3. Cost While $20,000 per year is significant, the benefits gained by implementing the

6 Proposal Page 6 QM process are also significant. Finding the funds necessary to implement the QM process immediately establishes that the college is serious about its commitment to quality courses, sets a standard for instruction and course design, and guarantees that faculty who teach online will participate in ongoing and effective professional development activities. While it was initially thought reasonable to ask college faculty to participate without compensation, this approach would seem to de-value the process (if it s so important, the college should pay for it). The $100 sum paid to each peer reviewer and the $150 paid to the chair is a modest amount that will result in great impact on courses, faculty and students. It indicates that the college recognizes that this will be an additional burden on its faculty and is serious about its commitment to the quality assurance process. 4. Time The quality assurance of courses could be accomplished more quickly If an industrial model of course reviews were adopted. For example, a small team of faculty, already trained in the QM process could be hired to review a large number of courses over the summer. While this would quickly accomplish the initial review, it would not have the significant impact on faculty professional development and pervasive adoption of quality standards that are desirable by-products of the QM process when carried out by mixed faculty teams. Although it will take longer (up to 5 years), it seems that the benefits obtained from involving all faculty in the process will be worth the extra time, effort, and resources required.

7 Proposal Page 7 Summary of Activity Appendix B Twenty-three Prince George s faculty members have participated in the QM process either as course peer reviewers or as course developers; eight online courses have been reviewed. Faculty who have been trained as Quality Matters Peer Reviewers as of Spring 2005: 1. Linda DiDesidero 2. Mara Doss 3. Laura Ellsworth 4. Diane Finley 5. Ray Gross * 6. Andrew Habermacher 7. Carolyn Hoffman 8. Nelson Kofie 9. William Lauffer 10. Tereza Marks 11. Barbara Mechler 12. Allison Miner 13. Mary Helen Spear 14. Robert Spear 15. Rhonda Spells 16. Barbara Thornton-Lewis Courses Reviewed Through the Quality Matters Grant as of Spring 2005: 1. ANT 103 Cultural Anthropology Andrew Habermacher (in Spring 2004) 2. ENT 283 Telecommunications William Lauffer (in Spring 2004) 3. HST 141 History of the US I Carolyn Hoffman (in Fall 2004) 4. CHM 202 Organic Chemistry II Ray Gross (in Spring 2005) Courses Reviewed In-house during FY 05: MOL sponsored paid peer review; MOL will identify the peer reviewers and perform all administrative services necessary to complete the review. 1. BIO 101, General Biology Tereza Marks Internal peer reviews; college pays for one outside peer reviewer per team and a fee to MOL for the set up and services necessary to complete each review. 2. CJT 251, Criminal Law Michael Nisson 3. EGL 101, Composition I Gledy Wariebi 4. PSY 101, Intro to Psychology Diane Finley Faculty who will be trained as Quality Matters Peer Reviewers in Fall 2005: In Fall 2005, an additional seven faculty will participate in Peer Course Reviewer training: 1. Bernadette Dantley 2. Fran Etner 3. Roxann King 4. Cassandra Moore-Crawford 5. Angela Rabatin 6. Jeffrey Snodgrass 7. Marilynn Thomas Course To Be Reviewed Through the Quality Matters Grant in Fall 2005: 1. EDU 205, Assessment of Students Tamela Hawley