This paper outlines the quality assurance processes that will be utilised by the International Programmes for Massive Open Online Courses (MOOCs).

Transcription

1 QASL 24/3 Quality Assurance for Massive Open Online Courses Purpose This paper outlines the quality assurance processes that will be utilised by the International Programmes for Massive Open Online Courses (MOOCs). Background and Context The International Programmes launched five MOOCs on the Coursera platform in June 2013 and these were considered by a pilot approval process to complement the quality assurance procedures of Coursera. This pilot process has been used to develop the processes for the approval and review of MOOCs outlined in this paper. QASL 22 (minutes ) asked that a formalised quality assurance process be developed for future MOOC developments. In common with new programme developments the reports for the approval element of the quality assurance process will be considered by the Learning, Teaching and Assessment Sub- Committee. QASL will receive review and monitoring reports. Risk Issues The purpose of the paper is to identify how risks associated with MOOCs will be mitigated against and managed. Recommended Action QASL are asked to consider the Quality Assurance Process for Massive Open Online Courses Emma Price Head of Quality, Corporate Performance and Quality August 2013

2 Quality Assurance for Massive Open Online Courses Context Massive Open Online Courses (MOOCs) allow participants to undertake bitesize units of learning. The University of London International Academy became a partner with Coursera in September 2012 and is the chosen platform for International Programme MOOCs, the first of which launched in June The MOOCs are initially offered to participants as a free to complete learning experience on topics drawn from the subject areas typically covered by existing University of London International Programmes modules or courses. This does not exclude MOOCs being developed on topics drawn from the breadth of University of London lead college expertise and providing indirect links to the University of London International Programmes. The MOOCs are intended to provide participants with an engaging and high quality learning experience, with assessment designed to provide feedback to participants on their understanding of the learning outcomes for each MOOC. Successful completion of all assessments will lead to a Statement of Accomplishment. The MOOCs are not credit rated. Therefore the Statement of Accomplishment would not be eligible for Accreditation of Prior Learning for a University of London award. The MOOCs will be given an indicative FHEQ level for internal benchmarking purposes. Quality Assurance This quality assurance process is to ensure coherency of the curriculum, teaching, learning and assessment activities of each MOOC and to consider the participant experience of completing the MOOCs. The process is also designed to complement the internal quality assurance procedures of Coursera. In addition the process covers the review of MOOCs following their completion by participants and will include statistical analysis and participant feedback. This process does not replace the processes for programme approval, monitoring and review that are set out in the Quality Framework for credit rated awards. MOOC Approval Procedures In many cases the content for each MOOC will be drawn from existing subject content related to the International Programme, designed to attract participants to enrol on the full programme of study. Where this is the case it can therefore be assumed that the contents have been through quality assurance processes to confirm the contents as being delivered for and appropriate to the identified learning outcomes. Thus, the quality assurance approval process will focus on the participant experience. However, where the contents are new or drawn from non-current programmes peer review will be undertaken to confirm the appropriateness of the subject matter for the stated learning outcomes and for the quality of the learning experience. Our MOOCs do not result in the award of credit or contribute directly to a University of London award. Nonetheless, it is important that procedures are in place to ensure that University of London International Programme MOOCs are informed by good practice and provide an engaging and high quality learning experience. A standing panel of the Learning, Teaching and Assessment Subcommittee (LTAS) will review each MOOC to ensure it is fit for purpose before it is available to participants. 2

3 This panel will meet on an ad hoc basis, as required. A timeline for the process can be found in Appendix one. Composition of the MOOC Panel 1 Chair to be drawn from the membership of LTAS Student to be drawn from the body of International Academy student committee members Panel members (x 2) with experience of distance/technology enhanced learning, drawn from the Federal University of London or the Centre for Distance Education (CDE) International Academy member with experience of the technology of VLE platforms Peer Reviewer drawn from outside the Federal University of London if the MOOC includes content that is not part of the University of London International Programmes provision or other Lead college on campus provision. Executive Officer to be drawn from within the Corporate Performance and Quality Directorate, International Academy Project Manager International Academy staff member with responsibility for MOOC delivery will be in attendance The MOOC panel may co-opt MOOC Practioners or former MOOC Instructors where appropriate. The current membership of the MOOC panel can be found in appendix two. Terms of Reference (MOOC Panel) 1. To ensure appropriate scrutiny of the content of the MOOC against its proposed title; 2. To consider the MOOC outline (curriculum, teaching, learning and assessment specification) and pedagogy to establish: (a) the coherency and relevance of the intended curriculum; (b) the accuracy of the information to be publicly provided to prospective participants about the nature of the MOOC (including the status of the Statement of Accomplishment) so as to inform participant expectations; 3. To determine that the MOOC aligns with the University of London International Programmes brand and associated reputation; 4. To consider whether the assessment activities test the learning outcomes in an appropriate manner and timescale; 5. To consider the viability of the technical delivery of the MOOC and to further test and, where appropriate, review the utilisation of the content into the International Academy s Virtual Learning Environment (VLE) for the corresponding International Programme; 6. To consider the overall participant experience, particularly in the context of mass open learning; 7. To provide suggestions for the enhancement of future MOOC developments and to identify good practice for wider dissemination; 8. To agree one of the following outcomes for each MOOC: approval identification of improvements and enhancements rejection. 1 External representation on the MOOC Panel is provided by the Intercollegiate members 3

4 Reporting Mechanism Following each meeting of the MOOC panel, a report containing the outcomes (including conclusions and recommendations) will be produced for noting by LTAS. Through the minutes of LTAS, there will be onward reporting of the panel s conclusions to the University of London International Academy Academic Committee. The report will also be received by the International Academy s Senior Management Team (SMT). The International Academy Programme Development Steering Group will monitor operational progress as the MOOCs are delivered and report regularly to the Senior Management Team. MOOC Review Following the delivery of each MOOC they will be reviewed by the Academic Development directorate. This review will utilise statistical data from the Coursera platform, participant feedback and instructor feedback to evaluate the delivery of the MOOC and identify any best practice for use in future MOOC developments. A report from this review will be considered by the Quality Assurance and Student Lifecycle Sub-committee (QASL). In addition there will be an annual report to LTAS, QASL, Academic Committee and the Board provided by the Academic Development Directorate, with input from MOOC Instructors and capturing the MOOC standing panel s recommendations. The report will include commentary on participant statistics and feedback on the experience and a summary of the MOOCs delivered in the previous annual cycle. The report will also identify best practice for dissemination to future MOOC developers and the International Programmes. Procedure Review The procedures and Terms of Reference outlined above have emerged from a pilot using the first phase of MOOC and this procedure will apply to the second phase of the International Programmes MOOCs (Spring 2014) and will remain under review. This procedure will require review and revision if the University of London shifts to awarding credit for the successful completion of MOOCs or allows them to be used for admission onto University of London awards. 4

5 Appendix one MOOC timeline 6-12 months before launch Initial course proposal submitted. MOOC Panel considers overall academic suitability of proposal. < 6-12 months before launch Agreement of all parties to proceed, and sign off of development schedule. Course design and development work commences. Filming schedule agreed. < 6-12 months before launch Lead Instructor confirmed. Coursera course description page reviewed by Coursera, academic and UoL staff, signed off and published. > 2 months before launch MOOC Panel consider course syllabus, content, structure and assessment details of course (as detailed in Terms of Reference above), including first week's video lectures. Copyright clearance begins. 2 months before launch Following approval from MOOC Panel, Coursera Course Development Agreement signed by University of London International Programmes institutional head and instructor. > 1 month before launch Instructor and university staff upload subsequent weeks course content, including lecture videos and assessment. Welcome s and demographic survey prepared for distribution to registered students Launch Course starts. Evaluation Review of course statistics and participant feedback to inform future course development 5

6 Appendix two Initial MOOC Panel Dr Roger Mills (Chair), CDE Fellow Dr Harvey Mellar, Emeritus Reader in Technology Enhanced Learning, IoE Dr Simon Rofe, Programme Director and Senior Lecturer, SOAS Jonathon Thomas, Policy and Development Manager, International Academy Student Representative, TBC Executive Officer: Emma Price, Head of Quality Project Manager: Barney Grainger/Julia Leong Son/Judith Collier 6