European quality audits in comparison: Different approaches to institutional diversity

Transcription

1 European quality audits in comparison: Different approaches to institutional diversity Alexander Kohler Managing Director Sirpa Moitus Project Manager Korkeakoulujen arviointineuvosto Rådet för utvärdering av högskolorna The Finnish Higher Education Evaluation Council (FINHEEC)

2 Quality Audit Network Network established and started functioning in 2008 Network aim An informal forum for exchanging experiences on audit methods Network meetings: Vienna 2008, Madrid 2009, Helsinki 2010 Participation criteria European quality assurance agencies implementing or having implemented audits Participating countries and agencies Austria: AQA Denmark: EVA England, Northern Ireland and Scotland: QAA Finland: FINHEEC Germany: GAC Norway: NOKUT Spain: ANECA and AQU Catalunya Switzerland: OAQ 2

3 Report on European audit models FIRST NETWORK PUBLICATION: Trends of Quality Assurance and Quality Management in Higher Education Systems. AQA Available at: 3

4 Overview of countries with audits BASIC CONCLUSIONS 1. HEIs are in the different stages of development of QA/QM systems 2. Countries are in different stages in implementing the ESG 3. The lenght of audit experiences varies between agencies Audits conducted from before 2003 Traditions England and Northern Ireland Audits since 2003 Experienced Finland Audit introduced after 2007 Youngsters Spain Scotland Switzerland Austria Norway Germany (Denmark ) Missing: Sweden, Iceland 4

5 Institutional diversity and autonomy reflected in audit models 1. Voluntary vs. mandatory audits Voluntary: Austria, Denmark (voluntary since 2007, earlier mandatory) Voluntary, but most of HEIs participate: Finland and Spain Voluntary, but HEIs must choose between audit and programme accreditation: Germany Mandatory: Switzerland, Norway Mandatory and a condition of funding: UK, Northern Ireland, Scotland 2. Institutional autonomy in design of its own QA system Areas of internal QA/QM systems to be evaluated are defined in the national audit guidelines HEIs are expected to make use of a wide variety of reference points Especially ESG Additionally professional and statutory guidelines ANECA provides guidelines and training for HEIs on how to build an internal QA system 5

6 Institutional diversity and autonomy reflected in design and implementation of audits 3. Input from HEIs in design of national audit models In many countries, HEIs have participated in the design of national audit models Many agencies have organised hearings, consultations and seminars Some agencies utilize systematically institutional and auditor feedback in the development of audit model 4. Implementation of an audit Some audit models evaluate if the QA systems are in place Some other models also look for evidence of the effectiveness of the QA/QM system For instance Germany: Samples of study programmes UK and Northern Ireland: Audit trails 6

7 Institutional diversity and autonomy reflected in audit topics and scope 5. The scope of audits varies In every country: Teaching and learning Additionally, in Austria, Finland and Denmark: Quality Assurance of Research/R&D, Innovation, Support services, Staff, Internationalisation... Some audit models consist only of mandatory themes In other audit models there are both common and optional themes for HEIs 6. General requirements to QM systems are similar Focus on internal QM processes and systems that guarantee a standard of quality (mission, excellence, minimum standard) Institutional ability to ensure and improve its own quality QM as a leadership task and as a basis for steering decisions Use of monitoring systems Documentation of QA, including QA responsibilities Involvement of internal (and external) interest groups The outcome aspect seems to be strengthening 7

8 Institutional diversity and autonomy reflected in audit reports and outcomes 7. Publication of audit results differ between agencies FINHEEC: Emphasis on transparency of audit outcomes and publicity of reports OAQ: Emphasis on the institutional privacy and data protection 8. Audit reports include both enhancement and accountability aspects Most reports include recommendations Some of reports also highlight the strengths and good practices of QA QAA reports include confidence judgements Confidence, limited confidence OR no confidence 8

9 Institutional diversity and autonomy reflected in audit consequences 9. Audit consequences Positive input to development of the internal QA system Usually no direct consequence on funding, except.. QAA audit outcomes are used as a condition of funding In Switzerland, if the HEI does not pass the re-audit, there is a theoretical possibility to a cut in federal funding (up to 25 %) Usually audits do not lead to closing down an institution, however.. In Norway HEI may loose the right to course provision until accredited In one case in Scotland, a limited confidence judgement was used one argument in the merger of two institutions 9

10 Institutional diversity and autonomy reflected in follow-up procedures 10. Follow-up procedures For those HEIs which passed the audit Usually, development of the internal quality assurance system is being evaluated in the next audit cycle QAA: One year after, formal response of HEI and series of meetings with the agency FINHEEC: After three years, a voluntary mid-term follow-up and benchmark seminar GAQ: After four years, a sample of study programmes are reviewed For those HEIs which received a re-audit/limited or no confidence decision NOKUT: After half an year, a re-audit is being organised QAA: Within 18 months, HEI must produce an action plan FINHEEC: After two years, a re-audit is being organised EVA, OAQ, AQA: There is a voluntary follow-up procedure available ANECA: Follow-up procedure is under discussion 10

11 Diversity of European audit models Barriers to a European dimension Audit/accreditation is a national competence Audit criteria are similar, but not the same As a result, audit reports are not comparable Forms of audit co-operation already on way AQA has used international observers Audit Network as a means to increase transparency of QA Challenges from a HEI point of view A need for international quality labels to support institutional cooperation and mobility A need to understand the status and scope of national audit/accreditation models in partnerships between HEIs Challenges from the agencies' point of view Mutual trust between agencies and networks International auditor pools?? Joint audit and accreditation models?? 11

12 Suggestions for discussion themes each group may choose 1-2 of the following themes 1. Meaning of creativity and institutional autonomy in the design of internal QA systems? 2. Support provided by the agency (before, during and after the audit)? 3. Expectations towards audit reports? 4. Audit consequences at their best and at their worst? 5. Diversity of European audit approaches advantages and disadvantages? 12

13 Instructions for the group work 1. Background of the participants 2. Division into 3-4 subgroups with 5-7 members in each 3. Please agree on the chair and the secretary 4. Start the group work with introducing yourselves and telling which discussion themes you prefer minutes for discussion, then a 15 minute break 6. Presentation and discussion of results from the subgroups 13