Massive Open Online Courses (MOOCs): Goldsmiths 1. Responsive Website Basics: Code with HTML, CSS and JavaScript 2. Responsive Web Design As part of a MOOCs specialisation: Responsive website development and design Purpose Members of the Learning, Teaching and Assessment Sub-committee (LTAS) are invited to receive the report from a panel event held in the time period 10 16 August 2015 to consider the two MOOCs: Responsive Website Basics: Code with HTML, CSS and JavaScript Responsive Web Design Background A MOOC specialisation is a multi-mooc sequence completed by a capstone project. To date, Coursera has launched 34 specialisations. In April 2015 Coursera announced a request for proposals in a number of high-demand topic area specialisations for launch in September 2015. University of London Lead College MOOC course teams were invited to express interest in these developments. The Creative Programming MOOC team from Goldsmiths was selected by Coursera to develop a specialisation on the topic of Full Stack Web Development following a competitive interview round. This will be the first specialisation launched on Coursera by a UK university. The first course of this specialisation will launch on the 15 September 2015 with the full specialisation available in December 2015. Recommended Action Members of the Learning, Teaching and Assessment Sub-committee are invited to note that, on the basis of the recommendations made by the panel considering the two MOOCs Responsive Website Basics: Code with HTML, CSS JavaScript and Responsive Web Design and following consultation with the Chair of the MOOC Panel; Chair s Action was taken on behalf of LTAS to support the launch of the two courses (September and October 2015). Quality, Standards and Governance University of London International Academy September 2015
LEARNING, TEACHING AND ASSESSMENT SUB-COMMITTEE Report of a panel convened on behalf of the Learning, Teaching and Assessment Subcommittee to consider the approval of two Massive Open Online Courses (MOOCs): Responsive Website Basics: Code with HTML, CSS and JavaScript Responsive Web Design As part of a MOOCs specialisation: Responsive website development and design Date: The Panel considered the MOOCs in the time period 10 16 August 2015. PART A: THE APPROVAL PROCESS AND FORMAT OF THE EVENT Introduction and Background 1. MOOCs allow participants to undertake free, bitesize units of learning. The University of London International Academy became a partner with Coursera 1 in September 2012 and is the chosen platform for the International Programmes MOOCs. To date, the University of London has launched 11 MOOCs, with a further 6 MOOCs planned for development in 2015. This is in addition to the development of the first UK MOOC Specialisation: Responsive website development and design. The two MOOCs Responsive Website Basics and Responsive Web Design form part of this specialisation. 2. A MOOC specialisation is a multi-mooc sequence completed by a capstone project. To date, Coursera has launched 29 specialisations. In April 2015 Coursera announced a request for proposals in a number of high-demand topic area specialisations for launch in September 2015. University of London Lead College MOOC course teams were invited to express interest in these developments. 3. The Creative Programming MOOC team from Goldsmiths was selected by Coursera to develop a specialisation in Full Stack Web Development 2 following a competitive interview round. This will be the first specialisation launched on Coursera by a UK university. The first course of this specialisation will launch on the 15 September 2015 with the full specialisation available in December 2015. Structure 4. The specialisation will contain five MOOCs and a capstone project and are listed below along with the launch date: 1 https://www.coursera.org/ 2 The original title was later changed to Responsive website development and design following extensive Search Engine Optimisation analysis. 2
MOOC MOOC title Launch date 1. Responsive Website Basics: Code 15 September 2015 with HTML, CSS, and JavaScript 2. Responsive Web Design 15 October 2015 3. Introduction to Meteor.js 16 November 2015 Development 4. Web Application Development with 15 December 2015 JavaScript and MongoDB 5. Responsive website tutorial and 15 December 2015 examples Capstone project Responsive website development and design capstone project 15 December 2015 5. All MOOCs within the specialisation will be created on Coursera s new platform called Ondemand. Previously, Coursera used a session-based platform with set start and end dates for each course (along with set assessment deadlines), on the Ondemand platform, courses will always be open. This provides learners with more flexibility by allowing them to fit the courses around their own lifestyle and work commitments and has proved very popular as an alternative to the session based courses. Certificates 6. In order to obtain a Course Certificate (this is available for each MOOC within the specialisation), a learner must verify each of their assessment attempts through recording their unique typing patterns and submitting photo identification, via a webcam. The learner will receive a certificate if they have passed the course and have paid the requisite fee ($49). Upon successful completion of the capstone project, learners will be eligible for the Specialisation Certificate. 7. The Course Certificate is designed to provide participants with feedback to indicate that they have met the intended learning outcomes. MOOCs are not credit bearing. Therefore the Course Certificate would not be eligible for Accreditation of Prior Learning for a University of London award. Content 8. A full stack web developer is the industry term for someone who builds and designs responsive websites. Essentially, this is a programmer who has capabilities in all the technical layers of a typical website. They should understand how underneath the familiar interface of their favourite web sites there are several layers of technology, leading from the text layout in the user interface all the way back to the database structure. 9. This specialisation takes a novice programmer through all of these layers; it starts at the front end with user experience design using HTML and CSS then moves into client side scripting with JavaScript and JQuery. Following that, the student will learn the meteor.js framework, which makes it possible to implement a reactive, multi-user web application using JavaScript and MongoDB. Along the way, the learner will be exposed to supporting knowledge such as the use of git repositories and unit tests, JavaScript libraries and HTML/ CSS templating with Bootstrap. Throughout the first four courses, 3
the student will see an image sharing application built from scratch, eventually using all of the techniques taught. In the fifth course, four new meteor.js applications will be built from scratch. Finally, the learner will carry out a structured project from a set of options or of their own design, which will challenge and expand their knowledge of all the layers. 10. As students progress through the MOOCs, they will begin to understand how the web pages they look at every day are generated from dealing with basic content layout through to how that content is stored and accessed from a database, and everything in between. This learning will be reinforced by threading a single project throughout the specialisation, where the students gradually build a complete website as they proceed. Each MOOC will add parts to the website, until by the end of the last MOOC they have seen the top to bottom construction of a fully functional, non-trivial, full stack website application. MOOC structure 11. Each MOOC within the specialisation will follow the same structure to provide consistency and familiarity across the courses. Each MOOC will contain four modules of work which is roughly equated to about two to three hours of learning and therefore we would expect students to complete a module within a week. Each module is broken down into lessons which will either be a single video lecture, a reading or a form of assessment. Assessment 12. Formative: Each lesson will have a short pop quiz (no more than 3 or 4 questions in a multiple choice questions quiz style) to test the learner s knowledge. 13. Summative: Each MOOC will have a summary quiz to complete at the end of each module and two peer assessments where students will be asked to practically apply their learning from that module; this piece of work will be assessed by 3 peers. Each quiz will contribute to 10% of the overall grade and the peer assessments will contribute 30% to overall grade. Students must achieve an average mark of 65% to achieve a Course Certificate. Capstone project 14. The capstone project is started once a learner has completed and passed all five MOOCs. In the capstone project, learners will be asked to create a complete, responsive, multi- user, data-driven website which aims to solve a particular problem. Learners will be encouraged to create a website addressing a problem that they are interested in, however students will also be given project ideas suggested by industry partners. 15. The capstone project will be a slightly longer course, currently structured at 8 weeks long and, during these weeks, learners will consolidate learning from across the five previous MOOCs. 16. Only learners who have passed the five MOOCs within the specialisation (and have paid for the certificate) will be eligible to graduate to the capstone project. 4
International Programmes Approval 17. The University of London International Academy has procedures in place to ensure that University of London International Programmes MOOCs are informed by good practice and provide a coherent learning experience. A Panel is convened on behalf of the Learning, Teaching and Assessment Sub-committee (LTAS) to review each proposed MOOC. 18. Members of the Panel were invited to consider the coherency of the curriculum, the appropriateness of the teaching, learning and assessment activities, and the overall experience of the MOOC participants. The Terms of Reference for the MOOCs Review Panel are attached in Appendix 1. 19. The MOOC review process was conducted online, with discussion carried out via email. Panel members were provided with a pro-forma to help focus the Panel s considerations, with any conflicting views reconciled by the Chair. Panel Constitution 20. Membership of the MOOC Review Panel was as follows: Dr Roger Mills (Chair) Dr Karsten Seipp Mr Dele Ogunjumelo Ms Sarah Sherman Ms Inger-Lise Moen (Secretary) Member of the Learning, Teaching and Assessment Sub- Committee Distance Education Consultant Peer Reviewer Senior Digital Designer/Developer Imperial College London Student member of the University of London Systems and Technologies Sub-Committee Member of the Systems and Technologies Sub-Committee Service Manager, Bloomsbury Learning Environment, RVC Head of Student Affairs and Quality University of London International Programmes PART B: CONSIDERATION OF THE MOOC Introduction 21. Panel members were provided with access to Coursera s Ondemand platform (learner view) where the materials to be considered were available. Some content was unavailable at the time of the review and so the Panel focused its attention on a sample of material, in particular: 5
MOOC 1: Responsive Website Basics: Code with HTML, CSS and JavaScript Modules 1 and 2, and MOOC 2: Responsive Web Design, Modules 3 and 4 22. Materials for consideration by the Panel included: Videos/video lectures Peer assessments/quizzes Overview information provided to participants Summary of Observations Made by the Panel 23. A synopsis of Panel members observations concerning both MOOCs is provided in the following paragraphs. The Appropriateness of the Title of the MOOC in Relation to its Content 24. The Panel acknowledged that the choice of title for the MOOCs may be related to a Search Engine Optimisation analysis, but panel members were of the view that the use of the term responsive in the title could benefit from clarification. Based on the materials the Panel had reviewed, it was noted that there was no section devoted to describe what responsive means. Coherency and Relevance of the Curriculum 25. The Panel found that overall, modules were well structured and cover relevant topics. The content was directly linked to the learning outcomes. The Panel noted that the structure allowed for a gradual expansion of the participant s knowledge of the topic in MOOC 2, however, the Panel commented that participants who were new to web design may benefit from certain topics being covered in more detail in MOOC 1 to enable all learners to master responsive website creation, rather than just understanding the fundamentals of the process. 26. The Panel noted that it would be useful for participants to receive advice on navigating the course, as there is no navigation menu apart from on the starting page of the Module. For example, information could be included on how to use the breadcrumb rather than the back button in the browser, and how the left/right buttons work to navigate through the pages of content throughout the course. 27. In terms of structure, the external subject specialist of the Panel observed that MOOC 1 may benefit from a different approach and focus. A broader introduction to fundamental concepts would enable the participant to get a thorough understanding of the topic. In particular, a detailed description/discussion of CSS Media Queries and relative sizes should be included as these are considered the bones of webdesign. Although the Panel was of the view that the MOOC essentially delivers what it promises in the course overview, it was felt that the lack of depth may deter participants to gain expertise in the topic. 6
28. The Panel observed that the Module learning content is divided in to weeks. In light of the specialisation being available on Coursera s Ondemand platform, thus allowing the participant to progress through the modules at their own pace, Panel members commented that in terms of learning progress, the structure of the Module should reflect more flexibility both on the learner platform and in the videos. The accuracy of information provided to prospective participants 29. The Briefing note provided for Panel members explained that this specialisation would take the novice programmer through all required technical layers. Panel members who were unfamiliar with the subject area noted that it would be beneficial with a statement clarifying the level of skills/experience that was required on order to successfully complete the MOOCs. It was felt that on the one hand, this course may be too speedy if it is aimed at beginners. On the other hand, more experienced web designers are likely to find the course too simplistic. 30. The Panel found the general information on the MOOCs to be clear, but was of the view that it would be useful for participants to have a list of what they need in advance order to complete the course, for example which internet browser to use. 31. The Panel also noted that there are frequent references to other MOOCs within the specialisation, and that in light of this it should be made clear to participants that these MOOCs are designed as a package. Although the Ondemand platform allows participants to sign up for individual MOOCs within the specialisation, the Panel noted that the MOOCs were not primarily designed for stand-alone study and participants should be informed about this. Teaching, Learning and Assessment Methods 32. The Panel found the video lectures/lessons interesting and engaging. Videos progressed the topics in a logical manner. However, it was also noted that for a novice the pace was quite fast. The videos were supplemented with transcripts of text, and although this was seen as useful for the participant, it would be helpful for the learner to have access to better edited text based materials so that they can refer back to key points and stages, in order to confirm their learning. 33. The Panel noted that participants could benefit from clear information on the progression requirements within the MOOC. The Panel was also of the view that learners would welcome some background information on the lecturers in the videos; for example who they are, where they are working and main area of research. 34. The Panel found that, overall, the assessment activities were clear, fair and appropriate. The questions relate directly to the video content. The instructions for the assessment are clear. 7
The suitability of the assessment activities to test the learning outcomes described in the MOOC outline 35. The Panel was of the view that the assessment activities were generally suitable to test the learning outcomes for the Modules. Each module was assessed by summary quizzes and a peer-assigned assessment. 36. The Panel confirmed a clear link between the videos and the quizzes, but noted a risk in that the learner would be able to give the right answer by listening carefully to the videos. This will, however, not necessarily reflect the level of understanding of the topic by the learner. The Panel felt that while the quizzes tested the participant s understanding of the content of the videos, the participant had no feedback on their ability to write code before embarking on the peer assessment task. One Panel suggestion was to introduce an opportunity for the participant to do a practical exercise (for example a simple website or coding for a website) to which the learner could receive automated feedback. This could be introduced prior to the peer review activity. The alignment of the MOOC with the University of London s brand and associated reputation 37. The Panel noted that the University of London logo is clearly represented in all videos, although it is not visible throughout the videos. The Panel noted that as all lecturers were from Goldsmiths, it might be beneficial with an explanation of the connection between Goldsmiths and the University of London. The viability of the technical delivery of the MOOC 38. Panel members provided detailed feedback on videos and other materials, all of which have been forwarded to the MOOCs development team for urgent consideration. A summary of this feedback is provided in Annex 1. The overall participant experience of the MOOCs 39. Overall the Panel felt that the MOOCs were stimulating and interesting and gave an insight to how web sites are developed. The videos were engaging and short and focused on one topic at the time. 40. The Panel also noted that there seemed to be an assumption in the videos that the participants have an understanding of other programming languages, but overall the lecturers appeared friendly, encouraging and on top of their subject. 41. Panel members noted that the MOOCs felt a bit rushed. The Panel identified scope for including more depth at the beginning. In addition, the Panel felt that it would be helpful 8
for the participant to have written materials refer to, rather than expecting them to backtrack the videos for repeat of information. PART C: CONCLUSIONS Outcome 42. The Panel gives its support to the introduction of the MOOCs Responsive Website Basics: Code with HTML, CSS, and JavaScript Responsive Web Design 43. In light of having only viewed two modules for each MOOC, the Panel did, however, express concern about a. the volume of the minor corrections required b. issues raised with the quizzes and videos Details are set out in Appendix 2. Panel recommendations 44. The panel made the following recommendations: a. That the introductory information about the MOOCs be revised to clarify who the target audience for these MOOCs are, taking in to account the Panel s view that the pace may be too speedy for a novice; and that the content may be too simplistic for an experienced programmer; b. That, for MOOC1, a detailed description/discussion of CSS Media Queries and relative sizes are included. A fundamental understanding of these concepts is necessary to enable the participant to get a thorough understanding of the topic; c. That the term responsive is defined and explained in the context of web design; d. That the video lectures include some key information about the lecturer; e. That the Modules be structured in appropriate study sections rather than weeks to reflect the flexibility in terms the participant s learning progress; f. That course information be clarified to explain to potential participants that the MOOCs within this specialisation have been designed as a package and although it is possible to complete an individual MOOC in the specialisation, the learning materials make reference to knowledge gained in previous MOOCs; g. That, in order to test the learner s comprehension, the introduction of additional formative assessment activities designed to give automated feedback prior to the peer review activity (which is summative) is considered; h. That the navigation between screens is made easier, and that further information is provided on how to use breadcrumbs and left/right arrow; 9
i. That consistency is applied in terms of covering up logos/brands on technical equipment displayed in the video lectures; j. That written learning materials be provided to support the videos, particularly summarising key points and URLs 45. The Panel requested that the recommendations in this report be: 1) considered and, where practicable, incorporated before the launch or; 2) considered and, where practicable, incorporated in other MOOCs within this specialisation Response from the MOOC development team 46. The Panel requested a response from the MOOCs development team on the recommendations made (paragraph 44, a-i). 47. The Panel has also requested an immediate response from the MOOCs Development Team to the issues listed in Appendix 2. [The following appendices are available on request from the Secretary of LTAS: Appendix A: Programme team overall response to the Panel report Appendix B: Programme team response to specific queries raised by the Panel report] 10
11 LTAS21.8