Quality Assurance Manual for Teaching Staff

Transcription

1 Quality Assurance Manual for Teaching Staff April 2014 FMNS Quality Assurance Group

2 Table of contents: General introduction.3 Course unit evaluations 4 Study progress supervision 5 Assessments.6 Degree programme evaluations 9 Appendix 1: standard course unit evaluation questionnaire.11 Appendix 2: FMNS Step-by-step assessment plan.13 Appendix 3: FMNS Faculty model Course unit assessment overview.15 2

3 General introduction Education is important to the Faculty of Mathematics and Natural Sciences. The Faculty has ambitions with respect to its degree programmes, their international orientation and their level. To secure and, if necessary, improve teaching quality a continuous process of quality assurance is required. The Quality Assurance Manual for the Faculty is formulated to implement a standardized quality assurance cycle for all degree programmes in order to enable comparisons between different course units and programmes. A second important motive for writing the Manual of Quality Assurance is the introduction of a new accreditation procedure in Matters common to all degree programmes (e.g. staff policy) can now be assessed at University level by means of the institutional audit, which means this is no longer done during degree programme visitations. However, even when the University of Groningen has been awarded institutional accreditation, there is still a requirement for all degree programmes to implement and document quality assurance processes in a structured way. Design of the quality assurance system Appropriate quality assurance is characterized by a cyclic process of determination, survey, knowledge and judgements. This concerns all levels within the Faculty, ranging from students, teaching staff and course unit to cohort, teaching team and degree programme. Typical quality assurance elements include, for example, course unit evaluations, student satisfaction surveys, assessment plans of the curricula, quantitative data for the programmes and the quality of the facilities. The quality assurance cycle provides continuous input for education improvement. For this purpose, it is important that improvements are not only formulated but also implemented and checked. Making agreements and executing and monitoring these form the final step in the quality assurance cycle: the improvement project. Who is involved? Quality assurance concerns everyone. Deputy programme directors play a central role in the improvement projects and are to a great extent responsible for the entire degree programme. Programme committees, Boards of Examiners, degree programme coordinators, quality assurance officers, managers, the ESC and the Boards of the USS, GSS and Faculty are also all involved in quality assurance. However, you as a lecturer are the most important person involved in the process. You teach our students and feel committed to the quality of your course units and the success and satisfaction of your students. Thus, you have tasks and responsibilities in the quality assurance cycle. A full overview of all FMNS agreements about quality assurance can be found at along with the complete FMNS Quality Assurance Manual and other information. The current document provides a concise summary of this, presented from your point of view. 3

4 1. Course unit evaluations A course unit evaluation is an assessment of the quality of a course unit. Both the lecturer and degree programme acquire information about the quality of the course unit as well as suggestions for improvement by organizing a standardized survey. General Who Programme Committee / Evaluation Committee When Propaedeutic course units: annually Other course units: once every three years Lecturer Tasks What Remarks Committee reports to the deputy director Exceptions (assessed again after one year): - Redesigned course unit - Course unit with a new lecturer - Course unit with a previously unsatisfactory mark Remarks Commence survey during course unit Course unit evaluations will always be discussed in the lecturer s Results and Development Interviews What is expected from you? Commencing the survey: You will be sent the survey (see Appendix 1) and must ensure that it is conducted before the completion of your course unit and that the completed forms are returned. A good moment to conduct the survey is often immediately after the examination. At that time, the response percentages are usually high and all parts of the course unit have been concluded. Students are thus able to evaluate all aspects. If there is no examination relating to your course unit, a good time to conduct the survey is during the final lecture or seminar, or other final meeting for the course unit. Assessment of the course unit: You will be notified of the outcome of the survey and given the opportunity to respond. The Evaluation/Programme Committee advises the deputy director with regard to the assessment of the course unit based on the outcome of the survey, the course unit report (containing quantitative data such as pass rate, number of enrolled students, etc.) and your feedback. The deputy director determines the final result of the course unit evaluation and sends you an assessment grade: very positive (A + ), positive (A), satisfactory (B) or unsatisfactory (C). If the Committee has doubts regarding the accuracy of the assessment grade, or in the event of a potentially unsatisfactory grade for your course unit: - You will be invited for an evaluation interview (unless the cause of the problem is absolutely clear) - The participants in this interview include 3-6 students who have followed the course unit and at least two members of the Programme/Evaluation Committee - If a positive assessment is expected as a result of limited adaptations, concrete and practicable recommendations for improvement will be formulated during the interview and, if you agree with these you will commit yourself to implementing these recommendations. The Committee will send the assessment recommendation to the deputy director, along with the report of the interview and the agreed points for improvement. - If more fundamental changes are required or more complex problems are apparent: the committee will conclude that an improvement plan is required for the redesign of the course unit. If the deputy director is in agreement with this, he will ask you to draw up an improvement plan. - Formulating the improvement plan is thus your responsibility as a lecturer! Obviously, this is also the case for implementing the plan! The deputy director will oversee this. 4

5 2. Study progress supervision The aim of study progress supervision is to prevent study-related problems. Therefore, students are assisted in their study progress and choices and when they encounter problems, so that they find their niche as soon as possible. In addition, study progress supervision provides information to the USS and GSS Boards on intake, enrolled students, selection, study progress and graduates. This information, together with the study advisors advice, contributes to improving the success rate of the degree programme. General Who Study advisor Mentor-tutor system When - Supervision at student s request - General information activities - The extent of supervision decreases over time Lecturer Tasks Remarks What In the formal role of tutor: be active in helping with study-related problems / questions Remarks The main person responsible for requesting supervision is the student The study advisor is at the centre of all supervision activities/tasks At all times the teaching staff are the contact point for students What is expected from you? The following applies to all teaching staff: - You must form a contact point for questions regarding study approach within your area of knowledge and for questions concerning choices for the remaining part of the students programme, such as choosing a Major or Minor, a subject for the final-year project, Bachelor s/master s thesis, etc. If necessary, you pass information on to the study advisor. - You must be committed to the students and show your commitment. - You must do your best to strengthen the relationship between teaching and research. - You must demonstrate the connections between your subject, professional perspectives and societal relevance. Practical information for tutors: - The deputy programme director or the study advisor will evaluate the tutor system through interviews with you and other tutors. - The tutor system starts with meetings in the first period of the first semester. The study advisor will stay in touch with you about the progress. 5

6 3. Assessments A good assessment programme guarantees that graduates attain the learning outcomes of the degree programme at the required level. In addition, it ensures regular feedback to students, teaching staff and the degree programme on the progress, functioning and potential gaps in the programme. All FMNS degree programmes should have good assessment plans for the curricula and course unit assessment overviews as well as a varied and transparent way of assessing students. - The assessment plan for the curriculum sets out all tests and assessments for the degree programme: it is the sum of all tests, their relationship to each other and to the learning outcomes of the degree programme. - The assessment overview for each course unit sets out the details: it explains the relationship between the learning outcomes, modes of instruction and assessment for the course unit. - Together, these documents ensure that students attain the learning outcomes of the degree programme. Teaching staff are responsible for drawing up the course unit assessment overview and providing up-todate information for Ocasys on aspects including learning outcomes, assessment and the calculation of the final mark. General Remarks Who Lecturer Lecturers as examiners are primarily responsible for compiling assessments, testing students and assessing the results of the test When Faculty Board Board USS/GSS Deputy programme director Board of Examiners Annual assessment plan curriculum Annual assessment overview for each course unit Ocasys Lecturer Tasks What Compiling assessments, testing students and assessing tests Drawing up an assessment overview for each course unit Writing and updating texts for Ocasys The Faculty Board and the directors of USS and GSS are responsible for compliance with the rules and regulations (e.g., assessment protocol/plan), supporting teaching staff and providing good infrastructure, organization and logistics The deputy director is responsible for formulating the learning outcomes for the degree programme and drawing up the assessment plan for the curriculum The Board of Examiners is responsible for - assessing students at degree programme level. The Board must be able to ascertain that the study programme will enable students to attain the learning outcomes - monitoring and assuring the quality of assessment in the degree programme The assessment plan for the curriculum is formulated by the deputy director and updated every year (April-May) The assessment overview for the course unit is drawn up every year by the lecturer who coordinates the course unit, and is updated annually before the start of the new academic year New texts to update the information in Ocasys should be ready in spring. The new Ocasys database is published on 1 April but can still be added to after this date Remarks The Step-by-Step Assessment Instructions can be used as a guide when compiling and assessing tests (see Appendix 2) A Faculty model, with instructions, has been formulated for this purpose (see Appendix 3) The lecturer is responsible for correct and up-to-date information in Ocasys 6

7 Explanation of the responsibilities of teaching staff As an examiner, you are the first and main person responsible for designing a good assessment task which adequately assesses the learning outcomes of your course unit. You are also responsible for the good preparation of all the students involved and for passing those students who have satisfactorily achieved the learning outcomes, all within the time reserved for the course unit. These tasks demand your commitment. They require time to prepare students for the test and to design an adequate mode of assessment (content, form and assessment). For larger course units, such as the final-year thesis, good and timely supervision is essential. A further new aspect is that you now have an important task in documenting assessment of your course unit so that the Board of Examiners is able to check the quality and content of assessment. For this purpose you must draw up an assessment overview for each of your course units. What is expected from you? Important: You must be qualified to compile and judge assessments. This is part of the University Teaching Qualification (UTQ). General: 1. You must use an adequate, varied and transparent form of assessment that is appropriate for the modes of instruction and the learning outcomes of the course unit. 2. An important element of the assessment overview for the course unit is the relationship between the learning outcomes and the modes of instruction and assessment. The overview should also explain the modes of testing and assessment. 3. You will provide information for Ocasys. More information: Instructions for determining the learning outcomes, informing students, compiling assessments, checking colleagues assessments and having your own assessments checked by them, and the actual implementation of these assessments can be found in the Step-by-Step Assessment Instructions (Appendix 2). You should systematically follow these instructions. Instructions for drawing up the assessment overview can be found in the form on The most important aspects are listed below: o You are responsible for compiling a good assessment. You must apply peer review to this. The assessment must be representative, selected to achieve the learning outcomes, have clearly formulated questions and an assessment form appropriate to the learning outcomes and the mode(s) of instruction. o If you coordinate one or more course units, you must draw up an assessment overview for each course unit every year. In doing so, you should use the Faculty model and instructions (see Appendix 3 and o The way the final mark is arrived at must be clear before the start of the course unit and must be stated in Ocasys. You are responsible for providing correct and up-to-date information in Ocasys. o You must mark written examinations within 10 working days after the day on which the examination took place, and communicate the results to the student administration office. o In line with the BSA system, some written propaedeutic examinations have a shorter marking term: 5 days rather than 10. If this applies to your course unit, you will be informed by the programme director. o You must mark practicals within 10 working days of the last practical. An exception applies to certain practicals for which the Board of Examiners has determined a maximum term of one month. o As a lecturer you play an important role in the evaluation of written tests. Depending on the pass 7

8 rate, different procedures are applied (see Appendix 2). o The final-year project for the Bachelor s and Master s programmes must be assessed by a minimum of two supervisors, at least one of whom is a member of the permanent academic staff of the University. o When assessing individual projects (including Master s projects), use the standard assessment forms for the degree programme. A requirement of the assessment procedure is that you assess not only the final product, but also the process by which it was arrived at, and that you provide a written explanation for the marks awarded. 8

9 4. Degree programme evaluation A degree programme comprises various course units that are structured according to logical learning pathways and relate to each other in terms of content. Assuring the quality of the programme therefore involves more than simply combining individual course unit evaluations. The quality assurance manual sets out three types of evaluation that are designed to assess the curriculum as a whole. Beginning with the evaluation on the smallest scale, these are: - Programme evaluation: evaluation of the phases of the curriculum - Mid-term evaluation: internal assessment of the degree programme - Visitation: external assessment of the degree programme. Planning for this is based on the accreditation term for the degree programme. The programme evaluation is based on the Faculty student satisfaction survey. The analysis of the survey also takes account of the views of other persons involved. The purpose of mid-term evaluations and visitations is to obtain a picture of the degree programme as a whole. The means that not only the curriculum is analysed but also aspects such as the implementation of assessment policy, available facilities, quality and quantity of staff, and study supervision methods. In the quality assurance manual, a chapter is devoted to each of the evaluations. The main points are summarized on the following pages of this manual for teaching staff. As a lecturer, you do not need to initiate these processes. However, you play an important role in them you are the person who provides the teaching and you have direct influence. When evaluations are carried out, you will be asked for your opinion on the content and quality of the curriculum. The required changes are formulated in cooperation with teaching staff, and implemented by you in your teaching. General Programme evaluation Remarks Who Education and quality assurance staff of Conduct survey, analyse results at Faculty level, the Faculty. compare degree programmes. Programme/Curriculum Committee Thorough analysis at degree programme level. When Every year for propaedeutic, Bachelor s Questionnaire is handed out to students at the and Master s students. end of the relevant study phase. General Mid-term evaluations and visitations Remarks Who Deputy director, on the instructions of the Faculty Board. Planning, designing and conducting the mid-term evaluation and preparing for external evaluation. When Once every three years. Internal and external evaluations alternate in a six-year cycle. Lecturer Tasks Remarks What - Thinking along about the programme - Implementing changes to own teaching - Participating in the internal midterm committee - Holding discussions with the external visitation committee. 9

10 What is expected from you? Every member of the teaching staff may be involved in the programme evaluation to a lesser or greater extent. - When analysing the results of the student satisfaction survey, the Programme Committee or Curriculum Committee also considers the views of others involved (e.g. you in your teaching capacity). This is being done by showing you some or all of the survey results and asking you to respond, and to explain the results. You may also be asked for further information about your course unit(s) or about your experience of matters relating to education and students in general. The Committee may also decide to discuss the curriculum in small groups, jointly seeking solutions to problems that have emerged. As a member of the teaching staff, you may be asked to participate in this type of discussion. - It may be the case that you are asked to be a member of the Mid-Term Committee, or the Committee may wish to discuss matters with you. The Committee holds discussions with teaching staff and takes account of their views in the evaluation and assessment process. You might also be asked to attend a meeting with the external visitation committee. 10

11 Appendix 1: Standard course unit evaluation questionnaire Students are an important source of information on the quality of a programme and the performance of the staff involved. Therefore, please answer the following questions candidly. If you did not follow part of the course unit, leave those questions unanswered (N/A). Please complete the questionnaire in black or blue ink. Mark your answer by completely filling in the box. Otherwise the answers cannot be processed. Answer categories: Totally agree Agree Disagree Totally disagree N/A General 1. Which degree programme are you following? 2. The course unit was well organized. 3. The learning outcomes were clear. 4. The course unit content linked well with my prior knowledge. 5. The study material in the course unit did not overlap or only partly overlapped with the content of other course units in my degree programme. 6. The literature (books, syllabuses, readers) was clear. 7. The literature was relevant to the course unit. 8. The positioning of the course unit in the curriculum was good. 9. The teaching method(s) used was/were in line with the objective of the course unit. 10. The design of the course unit motivated me to actively work with the study material. 11. It was clear how the final grade was arrived at. 12. Compared to the number of ECTS, the student workload for the course unit was: too little good too much. 13. In general I would give this course unit the following mark:.(mark between 1 and 10). Lectures and lecturers The lectures helped me to better understand the material. 15. The lectures complemented each other well. 16. The number of lectures was just right. 17. The lecturer (name) 2 was able to explain the subject matter clearly. 18. The lecturer (name) 2 was available for questions. 19. The lecturer s (name) 2 language proficiency was good. Tutorials/assignments (case studies, homework assignments, etc.) The tutorials/assignments dovetailed well with the lectures. 21. The tutorials/assignments helped me to better understand the material. 22. The requirements for completing the tutorials/assignments were clear. 23. The supervision of the tutorials/assignments was good. 24. The lecturers gave adequate feedback. 11

12 Practicals The practicals dovetailed well with the lectures. 26. The practical assignments were clearly formulated. 27. The requirements for completing the practicals were clear. 28. The feedback on completed assignments was clear. 29. The practical rooms/laboratories had sufficient workspace, equipment and material. 30. The assistance during the practicals was good. Examination The examination was a good reflection of the course unit content. 32. The level of the examination was in accordance with the assignments/exercises during the course unit. 33. The examination questions were clearly formulated. Other assessment type, namely 1, This assessment type was suitable for assessing the learning outcomes of the course unit. 35. The assessment criteria were clear 36. The contribution of each student to group assignments was to a satisfactory extent individually assessed. Own role in the course unit (not included in the course unit evaluation) 37. I kept up with the course unit material from the start. 38. I attended the following percentage of contact meetings (lectures, tutorials, etc. ): 0 < 50%; %; 0 >90%. 39. I spent enough time on all components of the course unit to be able to pass it. Comments/recommendations: 1 Delete the subset(s) of questions if the course unit does not contain a certain element, e.g. practicals. 2 Questions will be repeated for course units with two or more lecturers. 3 Alternative type of assessment, e.g. essay, oral presentation: to be filled in before holding the survey. If there are no group assignments in the course unit, question 36 can be deleted. 12

13 Appendix 2: FMNS Step-by-step assessment plan This is the FMNS step-by-step plan for compiling, setting, assessing and analysing tests. a. Defining learning outcomes Defining learning outcomes is part of the preparation of the curriculum component. What must students know and be able to do after successfully completing the curriculum component, and at what level (knowledge, understanding, attitude and skills). Examination questions and assignment instructions, etc. must be based on these learning outcomes, which are set out in Ocasys and in the assessment overview for the course unit. b. Informing students Students must be given the following information no later than at the first lecture: - the learning outcomes (as in Ocasys/assessment overview) - the compulsory literature (as in Ocasys) - the type of test (as set out in the Teaching and Examination Regulations, Ocasys and the assessment overview) - the weighting of the various parts of the assessment programme (e.g. when a mark is calculated from an examination and a paper), as set out in Ocasys and the assessment overview - the planned examination and resit dates, the deadlines for handing in assignments, and the penalty (in terms of marks) for missing a deadline - the right of inspection - the lecturer responsible and the deputy examiner who is appointed to mark examinations within the specified period if unforeseen circumstances arise (as set out in the assessment plan for the curriculum and in the assessment overview for the course unit). c. Constructing written tests The following points are important for determining the content of a test: - questions/assignments must be based on the learning outcomes (see under a) - the test must be reliable, i.e. it must generate consistent outcomes among different assessors and students; the questions/assignments must have only one possible interpretation - the test must be representative of all the subjects the lecturer considers important, in order to limit the effects of chance - the test must not contain any ambiguities and all information necessary (from the student s perspective) for answering questions or reporting on research must be provided - the weighting of the test must be in proportion to the time available; this applies to examinations but also to more extensive assignments such as writing reports. The following points are important when setting a written test. The cover sheet (or first page) of an examination must state the following information: - the time available - the number of questions and pages - instructions on how questions are to be answered - the weighting of the questions (if this is not the same for all questions). 13

14 d. After compiling a written test - the test is submitted to a colleague who can, for example, improve any weak points in the test and assess its difficulty - where relevant, co-assessors must be informed about the assessment mode. e. Evaluation of written tests with a pass rate <50% If the pass rate for an examination is less than 50%, the following procedure applies: - the lecturer reports to the deputy director, if possible within a day of its being established, that the pass rate for an examination was less than 50% - students are not informed of the results - the deputy director, following consultation with the Board of Examiner and/or relevant persons where appropriate, carries out an analysis of the examination and specifies action to be taken - the deputy director discusses the analysis and actions with the lecturer - students are informed of the results - the analysis and actions are submitted to the Board of Examiners. f. Evaluation of written tests with a pass rate of 50-60% If the pass rate for an examination is 50-60% (the lecturer calculates the pass rate), the OC and/or Evaluation Committee gives extra attention to the course unit in the course unit evaluation. g. Constructing oral tests The following points are important for determining the content of an oral test: - questions/assignments must be based on the learning outcomes - the oral test must be reliable, i.e. it must generate consistent outcomes among different assessors and students; the questions/assignments must have only one possible interpretation - the lecturer must formulate the questions/assignments and a marking scheme in writing, before the test takes place - an assessment form must be used as a basis for providing students with feedback. h. Evaluating written and oral tests After written and oral tests have been held, the course unit is evaluated in accordance with the relevant protocol. 14

15 Appendix 3: Faculty model Course Unit Assessment Overview Course unit assessment overview for <name of course unit, year of study, part of Bachelor s/master s programme> Formal context of the course unit assessment overview The basis for the assessment plan for the curriculum of the degree programme and for the assessment overview for each course unit is the assessment policy of the Faculty of Mathematics and Natural Sciences, as set out in section 6 of the Faculty s quality assurance manual ( version) and the protocol setting out the duties and powers of the Faculty s Board of Examiners (Protocol Board of Examiners, December 2013). Contents 1. a course unit overview (table): ECTS, mode(s) of instruction, lecturer(s), examiner(s), mode(s) of assessment and second examiner/back-up lecturer. You should also state which degree programme is responsible for the course unit. 2. a brief description of the course unit s content 3. the learning outcomes, separated into knowledge and understanding, skills and attitude 4. the position of the course unit in the curriculum and the background of the students 5. a specification of the mode(s) of instruction 6. an overview of the mode(s) of assessment used and the relationship to the learning outcomes 7. the assessment criteria and method for determining the final mark for the course unit 8. how test quality is to be assured. 15

16 1. Course unit overview Course unit name ECTS Mode(s) of instruction: Lecturer(s) and (second) examiner(s) Mode(s) of assessment The <Bachelor s/master s programme in XXX> is responsible for the course unit. 2. Brief description of the course unit s content 3. Specified learning outcomes for the course unit 4. Position in the curriculum, background of students The course unit assumes prior knowledge acquired from <name of course unit(s), degree programme(s)>. Approx. <# (no.) ( )-year students from the degree programme(s)> are participating in the course unit. The course unit is compulsory for the <XXX degree programme(s)> / is an elective in the <track> The course unit is often followed by, or prepares students for, <name of course unit(s), degree programme(s)> in which the learning outcomes attained are required as prior knowledge. 5. Specification of the teaching method(s) applied and the relationship to the specified learning outcomes Mode of instruction hours Learning outcome 1 Learning outcome 2 Learning outcome 3 Learning outcome 4 etc. 6. Overview of mode(s) of assessment applied and the relationship to the specified learning outcomes Mode of assessment Learning outcome 1 Learning outcome 2 Learning outcome 3 Learning outcome 4 etc. 7. Assessment criteria and method for determining the final mark for the course unit 8. Assuring test quality 16