Introduction and definitions. Module evaluation

Transcription

1 Guide to policy and procedures for teaching and learning Section 4: Student representation, evaluation and complaints Requirements for Student Evaluation of Teaching and Learning [Approved by the University Board for Teaching and Learning on 9 July 2013 for implementation from October 2013] [For the purposes of the processes described in this document, in Henley Business School the Programme Area Director will be fulfilling the functions of the School Director of Teaching and Learning in respect of programme evaluation and the Director of Studies will be fulfilling the functions of the School Director of Teaching and Learning in respect of module evaluation.] Introduction and definitions 1 These Requirements for Student Evaluation of Teaching and Learning were informed by the recommendations contained within the Final Report of the Working Group on Programme and Module Evaluation, which was approved by the University Board for Teaching and Learning on 5 November The University seeks and obtains feedback from its students in a range of ways and at a number of levels. This includes module questionnaires, Student/Staff Liaison Committees, programme evaluations, discussion and feedback sessions and informal dialogue. 3 In seeking student opinion, the University aims to provide useful information to relevant staff members, including module convenors, School Directors of Teaching and Learning and Associate Deans (Teaching and Learning). This information is essential in terms of both quality assurance and the enhancement of the student experience of teaching and learning. 4 In establishing the requirements for student evaluation set out below, the University aims to harmonise practices across Schools and Faculties in order to promote transparency, support quality assurance principles and enable comparison across the Faculties. 5 This document is primarily concerned with student evaluation at module and programme level. For further information on the operation of Student/Staff Liaison Committees (SSLCs), please refer to the policy on Student representation at the University of Reading. 6 This policy applies to undergraduate and taught postgraduate programmes. It also applies to taught modules offered within professional doctorate programmes, but does not apply to sessions offered in the Researcher Development Programme nor to the evaluation of doctoral programmes, which is fulfilled through processes developed by the Graduate School. Module evaluation 7 Whilst supporting the appropriate use of a range of formal and informal feedback mechanisms by Schools, the University s policy is that formal end-of-module evaluation in the

2 form of module questionnaires is an essential component of the student feedback process. Each module should normally be evaluated on a bi-ennial cycle. Method of evaluation 8 The University has established a standard Module Evaluation Template for use in formal endof-module evaluations (please see Appendix 1). The use of this form is not mandatory for the Session 2013/4, but will be for the Session 2014/5 by which time the University hopes to have in place a University-wide electronic system with additional centralised support for Schools. Section 1 of the Template comprises a series of questions grouped into four sub-sections (Content and Structure, Assessment and Feedback, Quality of Teaching and Learning and Learning and Teaching Resources), which must be included in all module evaluation questionnaires. Students are asked to respond using a five-point numerical scale and are also given the option of including qualitative comments under each sub-section. 9 Section 2 of the Template gives Schools the option of including up to 10 locally-determined, module-specific questions which might comprise a combination of numerical scale questions and more open-ended qualitative questions, as desired. Appendix 2 provides a list of possible open-ended questions for Schools to refer to as a guide, although this should not be seen as a definitive list. 10 The Template requires Schools to clearly state where and when students responses will be reviewed. 11 Adoption of this method is intended to ensure a transparent approach and a degree of harmonisation across Schools and Faculties, and to prepare a path toward a more automated system of module evaluation in due course. It enables cross-institution comparison of data obtained under Section 1, whilst recognising the need for flexibility to satisfy local requirements. Qualitative feedback is seen as complementary to the quantitative information gained from the numerical scale questions. 12 The Module Evaluation Template is available as a Word document ( and as a proforma on Blackboard. Frequency of evaluation 13 Schools are not expected to evaluate each of their modules every year. Instead, they are required to operate a bi-ennial cyclical evaluation programme for all modules, with the following exceptions where annual evaluation will be required for a period of three years: (i) new modules; (ii) modules which have undergone significant changes; (iii) modules where significant concerns have been raised, for example by SSLCs, a Periodic Review panel or by previous module evaluations. School Directors of Teaching and Learning will be ultimately responsible for determining which modules fall within categories (ii) and (iii) 14 Boards of Studies will have oversight of module evaluations for programmes within their remit and will be responsible for maintaining a comprehensive record of the cycle of module evaluations and any underlying rationale. This information can then feed into the external

3 examining process as appropriate, the SPELT process and the six-yearly Periodic Review process. Questionnaire Administration 15 Schools may choose to use either paper-based or electronic questionnaires depending on local circumstances and resources available, including cohort-size, the availability of software for collation/analysis of results (for example, Optical Mark Recognition software) and staff time available for data-inputting. 16 Whilst the use of Blackboard or other online survey tools such as Bristol Online Surveys (BOS) can ease the administrative burden on Schools in respect of data collation and analysis, it should be noted that electronic methods of module evaluation can lead to decreased response rates. Paragraph 17d below suggests a number of approaches which might be adopted in order to maximise completion rates when using electronic methods. 17 The University has established a set of good practice principles to guide Schools in respect of questionnaire administration as follows: a. Where paper questionnaires are being used, questionnaires should be distributed towards the beginning of the lecture/seminar/workshop to prevent the tendency to complete questionnaires in haste at the end of a session; b. Where paper questionnaires are being used, a Course Rep or a member of staff other than the module convenor/lecturer should be responsible for collecting the completed questionnaires and taking them to the relevant School Office in order to preserve anonymity and encourage honest and constructive feedback; c. Similarly, where Blackboard or other online questionnaire methods are employed within a class setting, the module convenor/lecturer should ensure that he/she is clearly positioned out of the line of vision of students PC/laptop screens; d. In order to maximise completion rates for electronic surveys, Schools may wish to adopt one or more of the following approaches: taking groups of students to IT labs towards the end of a lecture, encouraging students to access Blackboard via a smart phone or tablet, or using specialist polling software; e. Schools should aim for a minimum completion rate of 50%. Where small cohorts are involved, it may be necessary to aim for a higher rate of completion. Data collation and analysis of module evaluation 18 Schools are required to collate and analyse the results obtained from module evaluation questionnaires at the end of each Academic Session. 19 The results of module evaluation questionnaires should be presented as a series of data ideally in graphical or tabular format. Qualitative comments obtained should be included alongside quantitative data. Reporting of module evaluation 20 School Directors of Teaching and Learning will maintain oversight of the analysis of student evaluation findings and will be responsible for submitting a results package, complete with

4 lecturer(s) responses, to SSLCs and Boards of Studies for review at the first meeting of the next Academic Session. It is recognised that evaluation results should be treated with caution since it may not be beneficial or reasonable for some comments to be in the public domain - personal comments about lecturers, for example. Names of staff and students should be redacted from student evaluation findings in advance of their submission to SSLCs and Boards of Studies. Personal issues should be dealt with privately by SDTLs in conjunction with module convenors. If issues cannot be resolved, then the Head of School will need to be informed and an appropriate course of action decided upon. 21 Boards of Studies are formally responsible for responding to student evaluation including results of module evaluations. They are required to have a standard item for report on their agenda on student evaluation, which should, depending on the timing in the academic year, include results of module evaluations. The Student representation at the University of Reading policy also specifies that SSLCs should receive and discuss the results of module evaluations and programme evaluations. 22 Schools are required to make SSLC and Boards of Studies minutes, incorporating student evaluation findings and actions, available to all students within the School via Blackboard. Names of staff and students should be redacted from published information, as appropriate. Chairs of SSLCs/Boards of Studies are responsible for ensuring that minutes are made available in this way. 23 As part of the annual programme reporting process, the Associate Deans (Teaching and Learning) (ADTLs) will ensure that any issues raised by module evaluation within their Faculties are dealt with appropriately and in a timely manner. A section within the Annual Programme Report template requires Boards of Studies to comment on the key outcomes of evaluation and review mechanisms, including programme and module evaluation, identifying key issues and any actions that will be taken in response, and examples of good practice. Faculty Annual Quality Reports provide assurance that Boards of Studies have considered, analysed and evaluated the relevant data and identify any particular issues which require further action at Faculty or University level. 24 Schools/Departments may wish to consider involving Course Reps in devising, distributing and reporting the outcomes of end-of-module evaluations, although ultimate responsibility will remain with the SDTL/module convenor. Where Course Reps are involved, they should be reminded of the potential sensitivities around the data collected and the need for confidentiality. Providing feedback on module evaluation to students 25 Schools must ensure that students receive informative and timely feedback on the actions taken as a result of module evaluations (and not just feedback on the data obtained from evaluations). Such feedback may be transmitted in a variety of appropriate ways but should be communicated by the relevant module convenor directly to those on the module and to a wider School audience. 26 To ensure that actions taken as a result of the module evaluations are communicated to students, a range of the following mechanisms could be considered: a. publication on Blackboard of a summary of points raised and actions undertaken in response; b. publication of SSLC and Boards of Studies minutes on Blackboard (see paragraph 22 also);

5 c. communication via Course Reps; d. use of School/Departmental noticeboards; e. oral updates during lectures by the module convenor, Programme Director or School Director of Teaching and Learning; f. use of social media; g. dissemination via paper or electronic newsletter, for example making use of a You Said We Did format; h. publication of feedback actions in programme or module handbooks; i. sent to relevant students. Mid-module evaluation 27 In addition to the requirements set out above in relation to end-of-module evaluation, all Schools are strongly encouraged to undertake some form of informal, light-touch mid-module evaluation for all modules on an annual basis. This can be particularly useful in the resolution of practical and operational issues and provides an opportunity for module convenors to react quickly and efficiently to feedback from students and to make (minor) changes which will benefit the current cohort. Student feedback obtained via the Students Union and from research carried out by the National Union of Students and the QAA on the student experience in 2012 suggested that this would be welcomed by students and would encourage them to engage in future feedback opportunities. 28 Module convenors may wish to select from a variety of methods of mid-module evaluation, including (but not restricted to): (i) (ii) (iii) verbal feedback gathered informally by an academic member of staff with no responsibility for the module under review or by a Course Rep, during a lecture; use of the Poll Everywhere tool administered using mobile phone responses during a lecture either via PowerPoint with no live feed or via a web connection with a monitored live feed; Post-it and flip-chart method; for example, students are asked to note what is working well on the module and what, if anything, could be improved. 29 A trial of mid-module evaluation across a sample of different disciplines and module sizes was undertaken during the Spring Term A table summarising the effectiveness of, and resource required by, the various survey methods trialled is included as Appendix 3. Programme evaluation 30 From onwards, all Schools will be required to carry out some form of programme evaluation for all programmes on an annual basis, in addition to module evaluations. This can be done either at end-of-part and/or on completion of the whole programme. 31 The University acknowledges the resource implications of carrying out formal, paper-based methods of programme evaluation and considers that formal methods of evaluation might not always be appropriate given the level of survey activity for final year students in particular. Schools are therefore encouraged to make use of alternative, less formal methods of programme evaluation, including (but not restricted to): a. student end-of-year focus groups; b. informal feedback sessions held with ADTLs/Course Reps (in the absence of academic staff members from the relevant School).

6 32 Schools may wish to refer to the good practice examples summarised in Appendix 4 in respect of end-of-part/programme evaluation. 33 Schools must ensure that students receive informative and timely feedback on the actions taken as a result of programme evaluations, which can be transmitted in a variety of appropriate ways (please see paragraph 26 above).