Policy for evaluation of first and second cycle studies

Transcription

1 1 1 July 2003 Reg. no LTH G /03 SLTH Professor Per Warfvinge Vice-dean for first- and second-cycle studies Policy for evaluation of first and second cycle studies Aim This policy for the evaluation of first and second cycle studies at the Faculty of Engineering (LTH) is to contribute to the deliberate and systematic improvement and development of teaching at the faculty. The policy is structured as a framework document with general principles. This is complemented by a number of instructions which constitute a concretisation of the framework policy for various cases. At LTH, we distinguish between operational and reporting evaluation. Operational (formative) evaluation [operativ utvärdering] aims to improve teaching and learning during an ongoing course. Reporting (summative) evaluation [rapporterande utvärdering] is an ex-post assessment of courses. LTH s overarching policy applies to both types, i.e. that the focus is to be on students learning and not on the lecturer s teaching. The university s responsibility According to the Higher Education Ordinance [högskoleförordningen], the university is to provide students with the opportunity to put forward their views on a completed course through course evaluations. The university is to compile the course evaluations and provide information on the result and any decisions taken, as well as making the results available. This obligation is a natural part of the government s running of higher education institutions, which since 1993 has been characterised by comprehensive decentralisation within the framework of a system based on targets and outcomes. The Swedish Agency for Higher Education s evaluation of bachelor engineering degree programmes [högskoleingenjör] (Högskoleverket 2003:20 R) further clarifies how universities are to organise their work. In the report, the group of assessors establishes that There must be centrally designed guidelines and systems for course evaluations in the degree programmes within each higher education institution. Based on this policy, LTH intends to introduce a system for course evaluations that more than meets the general requirements in the Higher Education Ordinance and the specification of these requirements that the Swedish Agency for Higher Education presents. Avd för kemisk teknologi, Box 124, Lund Besöksadress Kemicentrum, Getingevägen 60 Växel Direkt Telefax E-post per.warfvinge@chcmeng.lth.se

2 2 Allocation of roles within LTH LTH has a fully implemented matrix organisation for first and second cycle studies, with three agents: the faculty board (SLTH), the study programmes boards and the departments. SLTH decides on the range of courses and programmes, as well as on the allocation of resources. The study programmes boards plan the activities by proposing a range of courses and a financial framework to SLTH as well as deciding on course and programme syllabi. The departments propose a range of courses and carry out a mandate with resources allocated as a lump sum. In this system the study programmes boards, by delegation from the faculty board, are the commissioners while the departments have a clear executive role. The department indicates the course director for each course in the course syllabus. The course director thereby represents the department for a particular course. With this policy, LTH intends to clarify responsibilities in such a way as to ensure that the various evaluation processes are attributed to the correct level within the organisation. The students role Both the formative evaluation and the summative evaluation are to be based on information about the students experiences and views of the courses. For the summative evaluation, the students provide information via survey responses, while the exchange of information for the formative evaluation can take place in countless different ways. At LTH, the focus is to be on students learning and not on lecturers teaching. For both the formative and the summative evaluation, this means that it is the students encounter with the subject which is to be evaluated, their understanding, their activities, their learning styles, etc. The process is to be oriented towards increasing students responsibility and participation within the framework provided by the tuition. If the students learning does not have the planned outcome, the teaching is to be systematically changed in such a way as to allow the achievement of the intended learning outcomes. For formative evaluation, this can be measured directly with methods that are adapted to the exact situation in which the students and lecturers find themselves. For the summative evaluation, we rely on the development of broad indicators of the factors that reflect the students learning situation. The students, mainly through the student councils [studieråd], also have a crucial role in interpreting and drawing conclusions from the information that emerges. However, the students are not to be burdened with the concrete work involved in the evaluation activities, such as collecting surveys, compiling reports, etc. SLTH s policy for course evaluations aims to reinforce the students engagement in their own learning and influence in various bodies, through the systematic use of all students experiences in the evaluation work.

3 3 Course evaluations so far Course evaluations have been important at LTH for many years. Within several programmes, the course evaluations have been produced in a systematic manner and provided data for planning at the board level and for the planning of courses. On certain study programmes, the management has taken care of these activities, whereas the Engineering Students Union (TLTH), through the student councils, has been responsible for data collection and compilation on other programmes. There have been considerable variations in the structure of documentation, of reporting back to students and of the course evaluations. With this policy, LTH intends to create an appropriate system of course evaluation that is of the highest standard with regard to both implementation and the concrete benefits of the evaluation work. Evaluations as a driving force within educational development and renewal There is every reason to be proud of the development work that is conducted within LTH s first and second cycle study programmes. The teacher training within LTH compares favourably with every other higher education institution in the country, and the teaching academy is a unique model for how teaching expertise can be valued and rewarded. During 2003 two major educational conferences will be held in Lund and Gävle, where lecturers at LTH will show the concrete results of educational development work in an exemplary manner. Educational development builds on the lecturers taking deliberate and well-founded decisions on their teaching, on the basis of solid educational knowledge and reliable information. The course evaluation system that is being introduced aims to provide lecturers with an optimal range of information. It also aims to provide students and study programmes boards with a good basis for their actions, both during an ongoing course and for planning and decisions concerning courses and programmes. With this policy, LTH wants to ensure that lecturers, students and various bodies within LTH have a substantial basis for their discussions and their decisions. Operational (formative) and reporting (summative) evaluation The responsibility for ensuring that formative evaluations are carried out rests with the department, and the responsibility for implementation rests with the course director. It is the lecturers task to organise teaching so as to support the students learning in the best possible way. In order to be able to do this, it is of the utmost importance that the lecturers have a clear picture of what happens during a course. With the help of formative evaluation, the lecturer continuously gathers information on what the engineering students experience as being difficult, what they misunderstand or grasp, etc. All this is then immediately converted into the teaching, and the lecturers decisions are thus well supported. Good lecturers have always, consciously or

4 unconsciously, conducted formative evaluation and they have done so in a number of different ways. 4 The formative evaluation also aims to support the dialogue in the relationship that characterises good teaching. This improves the students understanding of the content of the teaching, their planning and implementation of independent study and their assessments of how the teaching could be modified or maintained. In the other direction, the students gain insight into the lecturers considerations regarding the choice of content, methodology and goals in their teaching. Summative evaluation is an ex post assessment of courses. This form of evaluation has two main objectives: 1. To monitor, describe and document factors that constitute an indicator of quality in teaching 2. To provide documentation for a quality-enhancing dialogue between the study programmes boards, the departments and the students The responsibility for ensuring that the summative evaluation is carried out rests with the study programmes boards. The summative evaluation is carried out in compliance with the instructions issued by SLTH. Utbildningsservice has the necessary support functions for implementing summative evaluations. All stakeholders, students, boards, management, other lecturers, business and industry, the government etc, have a legitimate interest in forming an immediate idea of the quality of the work. The aim is not to enable continuous adjustments in order to improve learning, as in the formative evaluation, but rather to describe the quality indicators and thereby enable better overall decisions and deliberations. The organisation depends on this form of data to be able to state over time to what degree we are doing a good job. This means that the information made available must be general and not overloaded. It should also be presented in a consistent manner over large areas and long periods of time. It is important to bear in mind, however, that summative evaluations only describe parts of the activities, and that more detailed judgements require more direct experiences of what actually happens in teaching. The summative evaluation is to measure factors that influence students learning. Previous investigations (Studentspegeln, the Student Barometer surveys and surveys within Genombrottet) provide ample evidence of what distinguishes the courses at the Faculties of Engineering (in Sweden) both positively and negatively. LTH is now introducing a comprehensive system for formative and summative evaluation of teaching. The departments are given responsibility for the formative evaluation while the study programmes boards are responsible for the summative evaluation. LTH s evaluation system is to direct the faculty towards teaching that is characterised by cooperation and dialogue in such a way that the individual student, within the

5 framework of a reasonable workload, can work and be examined against clear learning outcomes and develop a diversity of skills. 5 Follow-up From the above, it is clear that the formative evaluation is the responsibility of the department and is to be implemented by the course director. As the boards, in a dialogue with the departments, commission and set prices for the courses, the dialogue is also to address the department s ability to offer teaching of a high educational quality. It is thus the study programmes boards that have the responsibility of following up and evaluating the departments strategies for formative evaluation. The responsibility for the summative evaluation rests with the study programmes boards, and is implemented by the programme manager. As the boards are SLTH s preparatory body, SLTH is to follow up that the summative evaluation is carried out in compliance with the stated terms. Programme evaluations In addition to the formative and the summative evaluation of courses, there is a great need to evaluate the study programmes as a whole. This covers aspects such as learning environments, the content of the programmes in relation to the labour market, student welfare aspects such as gender equality etc. These aspects are very difficult to cover at the course level, but are of course crucial to the degree programmes leading to a professional qualification. Since the early 1990s there have been a large number of programme evaluations. These were often initiated and implemented in cooperation between different higher education institutions, and have been of enormous value. As an example, the survey conducted in spring 2001 within Genombrottet can be mentioned, and during spring 2003 the students at the Engineering Physics programme contributed with a study. The programme evaluations of the Swedish Agency for Higher Education can be added to these. At present, SLTH does not intend to put forward a proposal for how programme evaluations are to be conducted systematically, but it will do so before the planned evaluation of the Master s degree programmes in Engineering is carried out in 2005.