Audit of LAppeenrAntA university of technology 2015

Transcription

1 Audit of LAppeenrAntA university of technology 2015 Rowena Pelik Ismo Kantola Eeva Myller Monika Simaškaitė John Taylor Touko Apajalahti Publications 2015:15

2 Audit of Lappeenranta University of Technology 2015 Rowena Pelik Ismo Kantola Eeva Myller Monika Simaškaitė John Taylor Touko Apajalahti Finnish Education Evaluation Centre Publications 2015:15

3 PUBLISHER Finnish Education Evaluation Centre BOOK DESIGN Juha Juvonen LAYOUT Juvenes Print Suomen Yliopistopaino Oy, Tampere ISBN (pb) ISBN (pdf) ISSN-L ISSN (paperback) ISSN (pdf) PRINTED BY Juvenes Print Suomen Yliopistopaino Oy, Tampere 2015 Finnish Education Evaluation Centre

4 ABSTRACT Published by The Finnish Education Evaluation Centre FINEEC Name of publication Audit of Lappeenranta University of Technology 2015 Authors Rowena Pelik, Ismo Kantola, Eeva Myller, Monika Simaškaitė, John Taylor and Touko Apajalahti Abstract The Finnish Education Evaluation Centre has conducted an audit of Lappeenranta University of Technology and awarded the University with a quality label that will be valid for six years from 9 April The quality system of the University fulfils the national criteria set for quality management of higher education institutions and the system corresponds to the European quality assurance principles and recommendations for higher education institutions. The object of the audit was the quality system that Lappeenranta University of Technology has developed based on its own needs and goals. The freely selected audit target chosen by the University was early undergraduate study guidance. The following elements were regarded as key strengths of the quality system: The comprehensive and accessible documentation, which has clearly identified responsibilities and covers the major functions pertaining to the quality management of education, research and societal impact; The commitment of staff at all levels to the continuous improvement of their teaching and to high quality research output encouraged by a link to the pay and reward system; Commitment to active involvement with and synergy between industry, the region and the University, which is reflected back in the commitment of the business and industry leaders to the University 1

5 Among others, the following recommendations were given to Lappeenranta University of Technology: The University should reflect further on how to achieve whole institutional change and widespread ownership of planned changes; and what would be required of its quality system in the future in order for it to achieve the ambitions in its new strategy. The University should consider how the quality system could better enable the University systematically to identify examples of good and innovative practice and to disseminate and embed them across the institution. The audit team advises that a stricter code of practice for supervision of doctoral candidates and a set of minimum standards or minimum expectations for doctoral study be prepared. Keywords Evaluation, audit, quality system, quality management, quality, higher education institutions, university 2

6 TIIVISTELMÄ Julkaisija Kansallinen koulutuksen arviointikeskus Julkaisun nimi Lappeenrannan teknillisen yliopiston auditointi 2015 Tekijät Rowena Pelik, Ismo Kantola, Eeva Myller, Monika Simaškaitė, John Taylor ja Touko Apajalahti Tiivistelmä Kansallinen koulutuksen arviointikeskus on toteuttanut Lappeenrannan teknillisen yliopiston auditoinnin ja antanut yliopistolle laatuleiman, joka on voimassa kuusi vuotta alkaen. Lappeenrannan teknillisen yliopiston laatujärjestelmä täyttää korkeakoulujen laadunhallinnalle asetetut kansalliset kriteerit ja vastaa eurooppalaisia korkeakoulujen laadunhallinnan periaatteita ja suosituksia. Auditoinnin kohteena oli Lappeenrannan teknillisen korkeakoulun laatujärjestelmä, jonka yliopisto on kehittänyt omista lähtökohdistaan ja tavoitteidensa mukaisesti. Yliopiston valitsema vapaavalintainen auditointikohde oli opintojen alkuvaiheen ohjaus. Laatujärjestelmän keskeiset vahvuudet ovat: Kattava ja helposti käytettävissä oleva dokumentaatio, joka määrittelee vastuut selkeästi ja käsittää laadunhallinnan päätehtävät koulutuksessa, tutkimuksessa ja yhteiskunnallisessa vuorovaikutuksessa. Henkilökunnan sitoutuminen opetuksensa kehittämiseen ja korkealaatuiseen tutkimukseen kaikilla organisaation tasoilla, jota palkkausjärjestelmä tukee. Sitoutuneisuus aktiiviseen kanssakäymiseen ja yhteisten etujen löytämiseen teollisuuden, alueen ja yliopiston kesken, joka heijastuu takaisin elinkeinoelämän ja teollisuuden johtajien sitoutuneisuudessa yliopistoon. 3

7 Lappeenrannan teknilliselle yliopistolle esitetään muun muassa seuraavat suositukset laatujärjestelmän kehittämiseksi: Yliopiston tulisi miettiä syvällisemmin, miten yliopistotason muutoksia saavutetaan, miten muutoksille saadaan laajalle levinnyt omistajuus ja mitä laatujärjestelmältä edellytetään tulevaisuudessa, jotta yliopisto kykenee saavuttamaan uuden strategiansa tavoitteet. Yliopiston tulisi pohtia, miten laatujärjestelmä voisi paremmin edesauttaa järjestelmällisesti tunnistamaan hyviä ja innovatiivisia käytänteitä sekä levittämään ja juurruttamaan niitä halki organisaation. Arviointiryhmä kehottaa yliopistoa laatimaan täsmällisemmät menettelyohjeet tohtoriopiskelijoiden ohjaukseen ja kuvaamaan tohtoriopintojen minimivaatimustason. Avainsanat Arviointi, auditointi, laatujärjestelmä, laadunhallinta, laatu, korkeakoulut, yliopisto 4

8 SAMMANDRAG Utgivare Nationella centret för utbildningsutvärdering Publikation Auditering av Villmanstrands tekniska universitet 2015 Författare Rowena Pelik, Ismo Kantola, Eeva Myller, Monika Simaškaitė, John Taylor och Touko Apajalahti Sammandrag Nationella centret för utbildningsutvärdering har genomfört en auditering av Villmanstrands tekniska universitets kvalitetssystem och har beviljat universitetet en kvalitetsstämpel som gäller i sex år från och med den 9 april Villmanstrands tekniska universitets kvalitetssystem uppfyller de nationella kriterier för kvalitetshantering som fastställts för högskolor och motsvarar de europeiska principerna för och rekommendationerna om högskolornas kvalitetshantering. Föremål för auditeringen var Villmanstrands tekniska universitets kvalitetssystem som universitetet tagit fram utifrån sina egna utgångspunkter och mål. Auditeringsobjektet som universitetet kunde fritt välja var handledning i det inledande skedet av studierna. Kvalitetssystemets viktigaste styrkor är: Den omfattande och lätt tillgängliga dokumentationen definierar tydligt ansvarsfördelningen och omfattar kvalitetshanteringens grundläggande uppgifter i utbildning, forskning och genomslagskraft i samhället. Personalen på alla nivåer i organisationen är engagerade i att kontinuerligt utveckla sin undervisning samt i att bedriva högklassig forskning. Lönesystemet utgör ett stöd för detta. Universitetet samverkar aktivt och söker synergier med industrin och regionen. Detta återspeglas som näringslivs- och industriledningens engagemang i universitetet. 5

9 Villmanstrands tekniska universitet ges bland annat följande rekommendationer för vidareutveckling: För att uppnå sina nya strategiska mål borde universitetet djupare reflektera över hur det åstadkommer förändringar på institutionsnivå och omfattande delaktighet i förändringsprocesser samt vad förväntas av kvalitetssystemet i framtiden. Universitetet borde reflektera över hur kvalitetssystemet kunde bättre främja en systematisk identifiering, spridning och förankring av god och innovativ praxis vid institutionen. Auditeringsgruppen uppmanar universitetet att utarbeta noggrannare anvisningar för handledning av doktorandstuderande och beskrivning av doktorandstudiernas minimikravnivå. Nyckelord Utvärdering, auditering, kvalitetssystem, kvalitetshantering, kvalitet, högskolor, universitet 6

10 CONTENTS Abstract...1 Tiivistelmä... 3 Sammandrag Description of the audit process Audit targets Implementation of the audit The organisation and the quality system of the HEI The quality policy of the higher education institution Objectives of the quality system Division of responsibility related to the quality system Documentation and communicativeness of the quality system Strategic and operations management Linkage of the quality system with strategic and operations management Functioning of the quality system at different organisational levels Development of the quality system Procedures for developing the quality system Development stages of the quality system Quality management of the HEI s basic duties Degree education The objectives for degree education Functioning of the quality management procedures The information produced by the quality system The involvement of different parties in the quality work Support services key to degree education Samples of degree education Bachelor s and Master s Programmes in Business Administration Master s programme in Industrial Engineering and Management for Bachelor of Engineering (UAS) graduates Doctoral Programme in Electrical Energy Engineering Research, development and innovation activities The objectives for research, development and innovation activities Functioning of the quality management procedures

11 6.3.3 The information produced by the quality system Involvement of different parties in the quality work Support services key to research, development and innovation activities Societal impact and regional development work The objectives for societal impact and regional development work The functioning of the quality management procedures The information produced by the quality system The involvement of different parties in the quality work Early undergraduate study guidance The objectives for early undergraduate study guidance The functioning of the quality management procedures The information produced by the quality system The involvement of different parties in the quality work Support services key to early undergraduate study guidance The quality system as a whole Present strategic context Overview of the quality system Quality system: degree education Quality system: research and doctoral education Quality system: societal interaction Conclusions on the quality system Conclusions Strengths and good practices of the quality system Recommendations The audit team s overall assessment Higher Education Evaluation Committee s decision Appendices Appendix 1: Table of the audit targets and criteria...67 Appendix 2: The stages and timetable of the audit process...73 Appendix 3: Programme of the audit visit

12 1 Description of the audit process 1.1 Audit targets The target of the audit is the quality system that Lappeenranta University of Technology (LUT) has developed on the basis of its own needs and goals. The focus of the audit is on the procedures and processes that the institution uses to maintain, develop and enhance the quality of its operations. In accordance with the principle of enhancementled evaluation, the higher education institution s (HEI) objectives and the content of its activities or results are not evaluated in the audit. The aim is to help the HEI to identify strengths, good practices and areas in need of development in its own operations. The Finnish Education Evaluation Centre (FINEEC) audits evaluate whether the institution s quality system meets the national criteria (Appendix 1) and whether it corresponds to the Standards and Guidelines for Quality Assurance in the European Higher Education Area (also known as ESG). In addition, the audit evaluates how well the quality system meets strategic and operations management needs, as well as the quality management of the HEI s basic duties and the extent to which it is comprehensive and effective. In this way, the audit focuses on evaluating the institution s quality policy and the development of the quality system, as well as how effective and dynamic an entity the system forms. Lappeenranta University of Technology chose early undergraduate study guidance as its optional audit target. As samples of its degree level education, it chose the Bachelor s and Master s programmes in Business Administration and the Doctoral Programme in Electrical Energy Engineering. The audit team chose the Master s programme in Industrial Engineering and Management for Bachelor of Engineering (UAS) graduates as the third sample of degree education. 9

13 The audit targets for Lappeenranta University of Technology were: 1. The quality policy of the higher education institution 2. Strategic and operations management 3. Development of the quality system 4. Quality management of the higher education institution s basic duties: a. Degree education b. Research, development and innovation activities (RDI), as well as artistic activities c. Societal impact and regional development work 1 d. Optional audit target: Early undergraduate study guidance 5. Samples of degree education: degree programmes: a. Bachelor s and Master s programmes in Business Administration b. Master s programme in Industrial Engineering and Management for Bachelor of Engineering (UAS) graduates c. Doctoral Programme in Electrical Energy Engineering 6. The quality system as a whole 1.2 Implementation of the audit The audit is based on the basic material and self-assessment report submitted by Lappeenranta University of Technology, together with an audit visit to the University on November The audit team also had access to electronic materials that were important for quality management. The main phases and time frame of the audit process are shown in Appendix 2. An international audit team carried out the audit in English. LUT was given the opportunity to comment on the team s composition, especially from the perspective of potential disqualification prior to the appointment of the audit team. The audit team: Director QAA Scotland, Master of Arts Rowena Pelik, United Kingdom (Chair) Professor of Higher Education Management, PhD John Taylor, University of Liverpool, United Kingdom (Vice-Chair) Head of Student Affairs, Master of Education Ismo Kantola, Turku University of Applied Sciences, Finland Master of Science in Electrical Engineering Eeva Myller, formerly the project planner of the Study Ability project of the National Union of University Students in Finland (SYL), Finland. Master s student in Industrial Engineering Monika Simaškaitė, Kaunas University of Technology, Lithuania. 1 Including social responsibility, continuing education and open university education, as well as paid-services education. 10

14 FINEEC staff members: Senior Advisor Touko Apajalahti acted as the project manager of the audit. As indicated, the audit team conducted a three-day audit visit to the University. The purpose of the visit was to verify and supplement the observations made on the quality system based on the audit material. The programme of the visit is shown in Appendix 3. The audit team drew up this report based on the material accumulated during the evaluation and on the analysis of that material. The audit team members produced the report jointly by drawing on the expertise of each team member. LUT was given the opportunity to check the report for factual information prior to FINEEC s decision-making. 11

15 2 The organisation and the quality system of the HEI The Finnish higher education system consists of two complementary sectors: universities and universities of applied sciences. Universities conduct scientific research and offer education based on this research, while the universities of applied sciences offer more work-related education as well as conducting research and development that support education and regional development. Institutions in both sectors receive most of their funding from the Ministry of Education and Culture, based on their performance. The activities of higher education institutions are governed by fouryear performance agreements with the Ministry. Universities offer Bachelor s degrees (180 ECTS) as first-cycle degrees and Master s degrees (120 ECTS) in the second cycle. After having obtained a relevant Master s degree, students can apply for a Doctoral degree. A pre-doctoral Licentiate s degree may be taken before a Doctoral degree. Lappeenranta University of Technology is a university falling under the Finnish Universities Act. The University employs slightly under 1,000 staff members and has around 4,700 students. Approximately 15 % of the employees and 10 % of the students are foreign nationals. At the time of the audit, LUT was organised into three academic faculties with departments under them and into other units as described in Figure 1. The faculties at the time of the audit were: The School of Technology The School of Industrial Engineering and Management The School of Business 12

16 Figure 1: Organisation of LUT in November LUT was preparing an organisational change over the autumn A draft version of the new organisation structure was presented to the audit team during the audit visit and is described in Figure 2. The renewed structure removes the faculty level and merges the academic departments into three schools, and, at the same time, introduces horizontal cross-school solution-oriented research platforms. The new organisation took effect from January

17 DRAFT FOR NEW ORGANISATIONAL STRUCTURE BOARD PRESIDENT VICE PRESIDENT COLLEGIATE BODY PROVOST VICE PRESIDENT DIRECTOR Jarmo Partanen TECHNOLOGY DIRECTOR Heikki Haario NATURAL SCIENCES DIRECTOR Anne Jalkala BUSINESS PROGRAMME DIRECTOR PROGRAMME DIRECTOR PROGRAMME DIRECTOR SOLUTION ORIENTED RESEARCH PLATFORM A SOLUTION ORIENTED RESEARCH PLATFORM B SOLUTION ORIENTED RESEARCH PLATFORM N SUPPORT SERVICES Figure 2: Draft for the new organisational structure as presented during the audit. The LUT quality management system is based on the Plan-Do-Check-Act cycle as depicted in Figure 3. Figure 3: Plan-Do-Check-Act cycle of the LUT quality management system. According to LUT, the purpose of the quality management system is to: ensure the continuous development of the University towards a collaboratively defined vision ensure the high quality of operations and results support the achievement of strategic targets. 14

18 3 The quality policy of the higher education institution The quality system s objectives and responsibilities are clearly defined and an inclusive approach to the goal-setting process is in place. The key people responsible for operations are highly committed and have the necessary skills. At the same, time the audit team recognised that responsibilities would change within the new management structure. The quality system produces a large quantity of information that is well documented, clear and timely. The information is widely available and meets the needs of different user groups, but could be communicated in a more systematic and targeted manner. The University has developed a Quality Manual as a unitary source of information and is implementing a new intranet, which will further strengthen the process of quality management. The quality policy of the higher education institution is at a developing stage. 3.1 Objectives of the quality system The objectives of the University s quality management system are presented in three levels: a. The quality policy outlines the principles of quality management and the quality management system b. The quality targets of core operations outline the directions in which the core operations of the University are developed c. The quality targets of processes define how processes are monitored systematically with specific indicators in order to determine how they are completed The quality policy includes the following principles that guide the University s performance: The University s activities and administration are based on continuous assessment and development 15

19 The University fulfils its mission in accordance with its strategy in ways that promote the well-being of the staff and students and sustainable development The University maintains a quality assurance system which ensures that the University is able to operate in a reliable, efficient and quality-oriented way, taking the needs of customers and other stakeholders and ethical principles into consideration. These objectives and operating principles provide a clear framework for the quality system. Precise targets are reviewed annually in connection with the evaluation of the quality management system. The quality manager is responsible for amendments and for collecting and documenting amendment proposals, in collaboration with experts and persons in charge. University community members may comment on the open amendment proposals. Given that specific targets refer to different levels and perspectives, their understanding and clarity are addressed at different levels of performance guidance. The audit team recognised that the overall objectives and principles were appropriate and offered a strong underpinning for the quality system. However, it was less clear to what extent these ideas had permeated the entire institution. Staff throughout the University were aware of the importance of effective quality management, but it was less clear to what extent such awareness represented a comprehensive and integrated approach. The University is aware of these issues and is taking action to enhance present working arrangements (see 3.3 below). 3.2 Division of responsibility related to the quality system The Rector has overall responsibility for the development and operation of the quality management system. In practice, a Quality Management Committee oversees the quality management system as an entity and considers quality management procedures, evaluation results and new internal quality audits. The audit team was content with the distribution of responsibilities at this level. Currently, the University relies heavily on middle administration/management (Deans and Heads of Department) for the strategic management, performance, quality management and quality system descriptions in their particular unit. The division of responsibility functions well and key people are committed to their duties. However, the audit team found examples where overall University policy was not being disseminated or implemented at middle/lower organisational levels. This did not mean that quality matters were over-looked. On the contrary, it appeared that many staff were collecting information and developing new practices intended to enhance quality, but were doing so in a well-intentioned, but un-coordinated way. Whilst such local initiative is often to be commended, it can create difficulties for the University in the exercise of its overall responsibilities, in the transmission of information and in the monitoring of performance. Moreover, without clear oversight by senior and 16

20 middle management, the risks of issues or individuals falling between the cracks are increased. The University has had some difficulties in the exercise of responsibilities within the quality system at middle management level, but it is important to stress that these issues have been recognised by the University. A new management structure is currently being implemented based around three schools. One of the intentions of the new arrangements is to clarify lines of responsibility and communications, whilst at the same time providing for improved transparency and reduced administrative overheads. The audit team supports these objectives and welcomes the changes as far as they are understood at the time of writing. At the same time, however, it is certain that organisational structures and an allocation of responsibilities for quality management will still be required within the new schools, and need to be agreed and implemented as soon as possible. The University emphasises that every member of the University community has a responsibility to fulfil within the quality management process because everyone s work involves dealing with the University s strategy and targets, describing activities, reporting, evaluation, giving feedback and handling results and feedback. The audit team was impressed by the commitment demonstrated by staff at all levels and the desire to enhance quality. This is not in question, but, in some areas, it is suggested that such efforts need to be channelled more effectively; hence, the importance of strengthening the role of middle management in the quality process. 3.3 Documentation and communicativeness of the quality system The University s entire quality system is described in the University s Quality Manual. This sets out the University s quality policies and goals, key resources, management and performance guidance practices, core aspirations and related responsibilities, quality targets and processes, and practices involving the assessment and development of activities. The Manual aims to provide internal and external stakeholders with a comprehensive picture of the University and its operations, administration and quality management practices. The audit team wishes to commend the University for the development of a comprehensive, unitary source of information on the operation of the quality system. This document forms a very effective basis for the whole quality system. The information needs of staff, students and external stakeholders are taken into account in the documentation. Moreover, there was broad awareness of the existence of the Manual across the University. This will be strengthened further as the University s new intranet, which integrates the Quality Manual, is established. The audit team commends the University for the use of the intranet in this way to ensure that the quality documentation is easily accessible for the whole University community. The intranet is an effective way to ensure that information is communicated in a systematic and targeted manner. 17

21 The University also relies on the development of action plans intended for the achievement of strategic targets. These action plans are also available on the intranet. There is no common practice for implementing or communicating matters relating to action plans. As a result, as recognised by the University, progress is not always clear to all members of the University. The audit team shares this view and found examples where the existence of plans was unknown or where staff were unaware of targets. It will be important for these difficulties to be addressed as the University s new organisational structure is developed. Again, the importance of middle management as a means of communication both upwards and downwards in the organisation is very clear. 18

22 4 Strategic and operations management LUT s vision is to be an agile, international university combining technology and business, representing the top European level in its key areas of expertise. LUT s quality system, applying the PDCA-principle, is reasonably well linked to strategic planning, management and operations management. The quality system relies heavily on the use of strategic performance indicators and quantitative targets to effect change. Strategic performance management is connected to performance evaluation of the academic staff. The quality system generates a substantial amount of information, which serves strategic and operations management. Quality information is utilised at different organisation levels. The Board reviews the results systematically and steers the development work. The top management, middle management and staff are committed to joint quality work. However, LUT should consider how to transform the current action plan procedures in order to make plans more effective and communicative. In addition, responsibilities, timetables and follow-up activities for strategic measures should be defined more carefully. LUT should continue its good work on business intelligence and combine all the data produced for strategic and operations management. The link between the quality system and strategic and operations management is at a developing stage. 4.1 Linkage of the quality system with strategic and operations management Since the previous audit, LUT has given up specialised sub-strategies (e.g. for regional development) and has only one institutional strategy. In Together, the strategy to 2015, an ambitious set of strategic goals are identified. The University previously claimed inter alia to create solutions and produce problem solvers to help address contemporary ecological and energy challenges. Together sets out a vision for 2015 of LUT as an agile, international university combining technology with business, representing the top European level in its key areas of expertise. 19

23 The audit team found during the site visit that strategic ambitions, as well as the organisational structure of the University, are undergoing revision. The strategy work was started during the summer of 2014 when the management and members of staff were invited to join facilitated group sessions. Later, a web survey for the whole staff was opened. The audit team acknowledges that the University management has chosen an inclusive, interactive and iterative approach to the development of the new University strategy that involves staff, students and stakeholders. The University emphasises the view that strategic management, operations management and quality management have the same objective: the success of the University and the realisation of a shared vision. The targets of the strategy Together are specified in action plans for research, education and societal interaction. Action plans define targets, development measures and indicators to be used. The monitoring of the plans is the Provost s responsibility. Target values are set in the annual operations plans of the University and (at the time of the audit visit) of the faculties. Currently, the University-wide action plans are comprehensive documents, which contain numerous development measures. For example, the action plan for education lists altogether 15 measures grouped into three focus areas and the responsible actor for each measure is clearly defined. However, there is no information on timetables, monitoring or resources. Moreover, the action plan for societal interaction does not name any responsible actors for some of the measures. The audit team suggests defining responsibilities and timetables as well as follow-up activities for strategic measures more carefully. The interviews revealed that members of staff were often unaware of the action plans and their key points. The University has already recognised the role and follow-up responsibilities for action plans as areas to be developed. The audit team shares this view and recommends the management to consider carefully what kinds of detailed strategic development plans are actually needed and how they could be made more communicative for the University community. The quality system relies heavily on the use of strategic performance indicators and quantitative targets to bring about change. When the University Board gives its approval to the annual budget and operations plan, it sets quantitative targets for scientific research, academic education, societal interaction and internationalisation. The University-level operations plan for 2015 includes 20 performance indicators. Most indicators are common with the national universities funding model or with the University s performance agreement with the Ministry. The audit team welcomes the compact framework of LUT s strategic indicators, which were well understood and familiar to the staff. On the other hand, some strategic measures were not as well defined within quality processes in the Quality Manual. For instance, LUT has not defined the quality processes to support it in raising the proportion of students achieving a minimum of 55 ECTS credits per academic year; 20

24 given that at the time of the audit LUT was some way off reaching its own targets and the higher Ministry target the audit team considered that it was important that quality processes were developed to help the University achieve its goals. As an example of a strategic measure, LUT could make more visible the actions it is taking and set up a University-wide project to develop good practice to raise the share of students who achieve the annual target. Results-orientation has been strengthened in the system. Discussions revealed that budgeting and forecasting the economic situation have improved remarkably. The units employ the TTS planning system for budgeting and forecasting according to the annual planning cycle. LUT has built business intelligence reporting systems and has improved internal communications about the results of operations. Preparations to widen the use of the data warehouse LUT-DW to produce follow-up data on the education process are in progress. The interviews confirmed that LUT-DW is used by the management and their financial support staff as well as by the project managers for research activities. The new intranet contains performance indicator reports at the University level and at the level of faculties. The senior management saw the development of effective tools for financial management of the research portfolio as their biggest challenge. The Deans expressed the need for more detailed reports about students progress at the level of specialisations. For example, the Faculty of Technology has implemented a database tool which enables monitoring each doctoral student individually. Although initiatives for local reporting solutions are well-intentioned and based on the real needs of units, the audit team recommends combining all the data produced for strategic and operations management. In addition, in the self-assessment report the University has recognised the development of follow-up information systems and more efficient utilisation of follow-up data and feedback as a development target. Based on the audit visit, the audit team shares this view and encourages LUT to explore the possibilities of the chosen intranet technology to improve business intelligence reporting e.g. to produce dashboards. 4.2 Functioning of the quality system at different organisational levels The operations management and quality management are seen as a top-down process. At the top, the University Board holds four meetings per year. In every meeting, the Rector reports about the financial situation and performance results. In addition, the Board receives statistical comparisons with other universities and benchmarking reports. Moreover, the Board makes formal decisions about development issues and has a logbook for monitoring the progress of measures. The audit team considers the role of the Board as well balanced; although the emphasis is on strategic oversight, the Board also reviews results systematically and steers development. 21

25 The Rector is responsible for the quality management system, strategic management and operations management. The Rector chairs the University Steering Committee, which is composed of the faculty Deans. The Steering Committee regularly reviews the realisation of operational targets at the University and the faculty levels. On the basis of memoranda seen by the audit team, it was not apparent if development measures were actually defined at meetings. The Quality Management Committee controls the development of the quality management system and the execution of internal audits. Based on the audit visit, the commitment of the top management to quality and to the culture of continuous improvement is strong. The Rector is personally very involved in the strategy process and has clear insight into the need to develop research evaluation. The top management stressed the importance of international benchmarking in assessing how well the institution performs. The University management conducts performance and development discussions with the various organisational units annually. In preparation for the discussion, the unit prepares its operations plan, risk analysis and budget. The Deans and Heads of Department considered the annual discussions as useful. From the viewpoint of the top management, the performance discussions provide a forum where units can present what they would like to accomplish and this makes a good starting point for the discussion on the relationship between faculty plans and the overall University strategy. However, it was evident that ordinary staff members were not familiar with the annual operations plans of the faculties. At the faculty level, the Dean is responsible for performance guidance, budget and quality management. The Deans receive reports prepared for the Board on the financial situation, the realisation of targets and the feedback collected through the year, which is an example of a practical use of information. The responsibilities for producing reporting information are clearly defined: the Chief Financial Officer is responsible for financial monitoring information and the Provost is responsible for risk reports and information used to monitor the operations. The audit team acknowledges that quality management system produces a wide range of information for the top management and the Deans and is reasonably well linked to strategic planning, management and operations management. Both the Board and the Steering Committee follow and discuss the information in a regular manner. The role of the Board is strong in steering any necessary corrective action. However, the role of the Steering Committee in defining new approaches to improve performance regarding strategic targets is less clear. In the context of the organisational structure in 2014, the Dean chaired the faculty Steering Group, which had Heads of Department as members. The Heads of Department were responsible for communicating issues to the department staff. During discussions, the audit team found examples where staff were unaware of University-wide policy, including, for instance, the action plans that complement the University strategy. Despite this, it is evident that middle management and members of the staff are committed to the operation of the quality system and to joint quality work. 22

26 The performance guidance for academic staff members comes from instructions given at joint meetings of their department or from instructions given by the Faculty Council. Meetings and discussions within the department were seen as being important in order to share opinions and to keep a record of issues to be improved. 23

27 5 Development of the quality system LUT has systematic and well-functioning procedures for evaluating and developing its quality system. The central method is the annual amendment of the Quality Manual. In addition, internal audits and external evaluations are employed as tools for system development. On the basis of established evaluation procedures, LUT has been able to identify the quality system s strengths and those areas which need further development. Development of the quality system has been very systematic since the 2009 audit. Clear progress has been made in relation to most development areas raised in the previous audit. The University has a compact set of strategic performance indicators and relevant reporting and monitoring procedures. LUT has concentrated its system descriptions into one Quality Manual, which can be accessed via the staff intranet. As a result of its strategic ambition, LUT has strengthened the international dimension of external evaluations. In summary, the quality system works better than before. Development of the quality system is at a developing stage. 5.1 Procedures for developing the quality system Annual amendment of the Quality Manual forms a systematic procedure for the maintenance of the quality management system. Different bodies and individuals address development needs as part of their duties. The Quality System Manager is responsible for collecting and documenting amendment proposals, which are handled by the Vice-Rector and Provost and discussed at a Quality Management Committee meeting. The audit team recognises that the process is well-defined and participatory. During the interviews, the senior management saw keeping the documentation of practices up to date as a challenge in the course of change. Nevertheless, the Quality Manual has been amended eight times during Discussions revealed that the Quality Manual will be updated again after the new organisation has taken effect. 24

28 The audit team acknowledges that the maintenance of the Quality Manual as the system description and point of authority has been very systematic. However, it was not apparent if changes to processes and instructions were handled only once a year as part of the Manual amendment or whether the University was able to react more quickly when new instructions were needed. The LUT procedure for giving and officially approving instructions and guidelines could be defined in the Quality Manual. The University emphasises the importance of internal and external evaluations as tools for system development, in addition to the annual amendment of the Quality Manual. Internal quality audit procedures aim to assess quality management practices in different units and from different actors perspectives. The results of audits are handled in the Quality Management Committee meetings and in the meetings of the units subject to an audit. The audit team recognised that internal audit reports have often produced a comprehensive picture about the quality management of basic duties in the units. The audit team found examples of how the audits have produced information for identifying development targets or renewals. For example, as a consequence of the internal audit of the University s management system in 2012, the practice regarding University Steering Committee memoranda was improved. However, as LUT has identified a need to strengthen the effectiveness of internal quality audits, it should consider introducing a formal follow-up procedure for internal audits. LUT s self-assessment report for the audit reflected the University s significant ability to identify strengths and weaknesses in quality management. However, the audit team was surprised that the members of the Quality Management Committee who were interviewed did not raise the possible effects of the ongoing organisational restructuring and the new strategy on the quality system. On the other hand, the development needs of the quality system seemed relatively clear for the Rector. From her viewpoint, the Quality Manual will have to undergo a revision because the future organisation will also include new processes and functions. For example, an evaluation mechanism and selection criteria for the new solution-oriented research platforms will have to be developed and included in the quality system. The audit team encourages LUT to consider carefully how to take into account the requirements set by the new strategy within the quality system and how the system might be changed to support new strategic ambitions. 5.2 Development stages of the quality system Quality management at LUT has a long tradition. Before the establishment of the quality management system, performance guidance processes guided the development of the University s operations. The first official version of the Quality Manual was approved by the Board in In addition, faculties and other units compiled quality management descriptions for inclusion within their own manuals. The internal quality audit procedure was launched in After that, LUT participated in a FINHEEC quality system audit, which it passed in

29 In order to provide a sustainable basis for performance guidance and quality management, LUT has kept the universal principles of quality management as they were defined in The quality targets of core University operations have been updated due to revised strategic policies, new indicators and other performance guidance practices. The audit team was impressed by the systematic development work which has been carried out and also by the commitment to the given audit recommendations. Clear progress was identified by the audit team in relation to most issues raised in the previous audit. First, the University has built a compact set of performance indicators which are linked to strategic goals. The University and its units have adopted the same indicators for performance management. Moreover, LUT has developed a systematic monitoring procedure, which works especially well in the context of the Board s operation. The previous audit recommended that LUT should consider combining several quality manuals of small University units into one institution-wide Quality Manual. This task has been completed. LUT has also aimed to harmonise postgraduate education procedures, which are now coordinated by the LUT Graduate School (discussed in Chapters 6.1 and 6.2.3). In addition, LUT has launched the new intranet for staff, which includes information from the Quality Manual. Moreover, LUT has been successful in employing the SolePro project and time management system. The audit discussions revealed that the system is widely used among staff and managers. In the previous audit, several recommendations concerned feedback culture and practice. The University has managed to raise the response rates of electronic course feedback, but, as the University acknowledges, there is still room for improvement. The University has introduced new feedback practices such as the University-wide support service survey and a postgraduate survey. There is also a survey for employers and other stakeholders. The quality management of societal interaction, discussed in detail in chapter 6.4, has been improved by defining roles and responsibilities and by appointing persons in charge of coordination for societal relations. Also, a Societal Interaction Committee was appointed in 2012 to evaluate and develop cooperation with stakeholders. A remarkable new dimension in the development of LUT s quality management since the previous audit has been in the field of internationalisation, especially in relation to the use of external evaluations and benchmarking activities. This development has happened as a result of the University s own strategic ambition. The University has recognised that accreditations have brought international visibility and partners in cooperation. The audit team was impressed by the number of international accreditations of degree programmes and international benchmarking projects. The audit team encourages LUT to consider how benchmarking could be made an integral part of the quality management system as a tool for enhancement. 26

30 6 Quality management of the HEI s basic duties 6.1 Degree education The quality system produces qualitative and quantitative information that is used to develop degree education. The objectives for degree education are closely linked to the University strategy. Feedback is an integral part of quality management and is used both on the degree programme and course levels to develop education at the University. The system is heavily dependent on student feedback; further use of additional sources of information should be considered. Strong commitment to producing high quality education is evident and staff, students and external stakeholders participate in the development work in a meaningful manner. However, the present system does not sufficiently encourage innovation and diversity in teaching methods and therefore the audit team recommends adopting a more systematic approach to the development of the pedagogic competence of the teaching staff. Systematic feedback collection and utilisation procedures advance the functioning and enhancement of key support services. Quality management of degree education is at a developing stage The objectives for degree education The University strategy sets out the framework for degree provision under the Universities Act. Objectives and targets for degree programmes form part of the University s annual operations plan and are integrated with the targets agreed with the Ministry of Education and Culture. In the strategy 2015 Together, LUT sets a clear and ambitious target for attracting the most gifted students from both Finland and abroad; and for LUT graduates gaining employment in the most demanding and highly paid jobs of their field as well in international labour markets. Key objectives for degree education cover: competitiveness, measured by numbers of applicants and graduate employability; internationalisation as indicated by student mobility, international recognition and 27

31 accreditation; quality as seen through degree content, in research and in the skills developed in students; satisfaction, including student feedback; and performance, measured by students progress in their studies and by graduation rates. The objectives, quality targets and indicators are derived from the University strategy and are defined for each process of degree education in the Quality Manual. The tasks and responsibilities are also defined. Both quantitative and qualitative in nature, the quality targets are effectively aligned with the strategy. The Board sets and monitors annual objectives for education. If the University is falling behind on some of the objectives, the Board requires a plan with corrective actions to be presented Functioning of the quality management procedures Within LUT as a whole, undergraduate education is headed by the Vice-Rector for education who has University-wide responsibility for the curriculum. Degree education is based on defined learning outcomes with those at course level relating to overall programme learning outcomes. In addition, individual students will agree their personal study plans within a degree programme. Curriculum work, where the Head of Degree Programme and teaching staff evaluate the quality, professional relevance and development needs of a degree programme, is seen as a key process for ensuring high quality at the programme level. The Quality Manual outlines the annual cycle (Figure 4) for the review and development of the curriculum. The outcome of the curriculum cycle are the descriptions of degree programmes and courses, which are published annually on the University portal. The cycle is well-established and it was clear that staff were familiar with the cycle and its stages, and that it was widely used by the staff. LUT states that it is giving more responsibility to the heads of degree programmes, expecting programmes to be continuously developed in terms of quality and performance. The established curriculum cycle supports this ambition. However, it was unclear whether all degree programmes have procedures to ensure a variety of teaching and learning methods. The analysis of the contents and learning methods of a degree programme as a whole, which was conducted in the undergraduate sample programmes (Chapters and 6.2.2), are good examples of such procedures, and should be implemented in all programmes. 28